I would like to configure my Graylog instance in such a way that for certain logs, there are two “views”: One in which personal data is anonymized (for your usual troubleshooting while keeping GDPR compliance etc.) and one that contains more information (for when, for example, there is evidence of any attack and we need to track down the “patient zero”-workstation).
I have been tinkering around on this for quite a bit. I have tried to route some logs into different streams (and different index sets), then manipulate one of the streams with processing pipelines. Every time I did this, the changes the processing pipelines performed on the logs also applied to the other stream which I meant to stay unchanged. My Message Processors Configuration is this:
- Message Filter Chain (active)
- Pipeline processor (active)
- AWS Instance Name Lookup (disabled)
- GeoIP Resolver (active)
I also tried to send the raw logs via output to a different input and put it into the different stream from there, but the result was the same.
Now, I could probably accomplish my goal by setting up a second Graylog instance to host only anonymized logs, but that seems overkill to me.
Can anyone tell me if there is a way to configure what I referred to as different “views” in Graylog? And if so, how should I go about that?
Any help would be greatly appreciated.