Hello,
i’m newbie with SIEM and Graylog, so I have some questions about configuration and strategy. Im’ not sure to understrand correctly the input / pipeline / stream / extractors relations.
In order to do some tests, I set up a Graylog server running and getting some logs via syslog from another server.
So there is an input running and i see the syslog messages via All Messages stream and via a dedicated stream “MySyslogTest”.
Having 2 different users teams with different needs, I would like to keep a raw version of the syslog messages for the first team and I would like an extracted/parsed version of the same syslog messages for the second team.
What’s the best way to implement this ?
The extractor is only attached to the input ?
So I think about 1 input with a pipeline that sends syslog messages into 2 differents streams. This pipeline extracts data and populates new fields before sending the message to 2nd stream.
Is that possible and a good way ?
---> What’s the best way to implement this ?
Sounds like you have a good handle on how things work - your idea below is solid.
--->The extractor is only attached to the input ?
That is correct - extractors and pipelines have their different advantages (and community supporters) but hey essentially do the same thing.
----> So I think about 1 input with a pipeline that sends syslog messages into 2 different streams. This pipeline extracts data and populates new fields before sending the message to 2nd stream.
Is that possible and a good way ?
You could attach two streams to the input, one that results in the message not being changed and residing in one (elasticsearch) index, then in the second stream (Going to a different Elasticsearch index) attached to the input, you can connect a pipeline that will break out the message differently.
Note though, when you break out a message, the system generally leaves the original intact as “message” and simply adds on fields with data to that record. So if your “just the message” team doesn’t care about the extra fluff of things being broken out, they can still see the original and you don’t have to go through all the separation/duplication.
