Scavenge data but don't save some syslogs?

Graylog Central (peer support)
1

General use question… with Graylog can I use data from syslogs to increment a variable but not retain the actual syslogs themselves?

A very basic example of this might be bandwidth usage from a firewall for which we might get 10,000 syslogs an hour. I would like to take the value, add it to the tally, then dump the syslog to minimize Graylog server resource consumption.

Another one might be number of failed VPN attempts.

We want to be able to see these stats on a dashboard but then if we need to research anything we would go to our primary system (which is forwarding these syslogs to Graylog). Our primary system retains everything and runs good reports but just can’t give us the short term dashboard we want.

Thank you.

2

Hey @CrocoGator,

You can accomplish this function with pipelines.

As an outline:

  1. Get the value(s) you need as variable(s)
  2. create a new message with the variable(s) as fields (create_message)
  3. send the new message into a stream (route_to_stream)
  4. Drop the original message. (drop_message)
  5. Profit :smile:

Greetings - Phil