I’m quiet new to Graylog. I’ve set up Graylog and configured Windows clients with sidecar and other devices to send Logs to Graylog. All working fine. I’ve also configured some streams for the types of devices and built some dashboards.
How should I proceed to achieve my goal (if it’s possible):
I want to have a Dashboard where only important messages show up (the ones that require you to do something) or “unknown” messages which would be displayed constantly on a monitor.
When I see a unknown message that I’m not interested in I want to sort it out for the future so it won’t be displayed anymore.
I can imagine that this could become quiet a long process connected with much work but this is what my boss wanted to have.
-> In the long run: Graylog tells me if I have to be worried
For Windows clients I know that you can filter non-interesting Logs already with Sidecar before sending to Graylog but I want to save all incoming Logs in the database in case I want to do troubleshooting.
As far as I understood you could maybe use Pipelines or streams to sort out messages you don’t want to display. But if I drop messages there with rulesets, are they still in the database?
the short answer - yes, you use pipelines to route messages to different streams and the streams underlying index. You can have multiple streams using a single elasticsearch index. Think of a stream as just a tag on a message. When you drop a message in a stream, it does not get stored by elasticsearch database.
I set up my windows messages to route all warn, error, critical to one stream and index for review and all informational to another stream/index with a short life. The underlying index for shortlife only holds those messages for a shorter time. I have a multi-staged pipeline to evaluate windows ID on incoming messages from winbeats and route/move them to an Alerting stream/index - where I am watching to execute the right alert when they arrive. This way the message is placed in a index relevant to how important it is and gets tagged as something that was alerted on.
Here is a content pack I put together a while ago that has the framework. it may need some tuning.
