Hi
Partially inspired by LogPoint - I would like to create a series of extractors, that would ensure a uniform search to be sufficient on logs, no matter the source - eg. LogPoint will add “device_ip=”, “device_name=” etc. (will add a picture).
I know that would need a series of extractors for different logsources… but is it possible? realistic? and how?
Best regards
/Flemming
How does the message field actually look when it arrives in Graylog, the top example is key value pairs and the bottom in JSON. Either could be dealt with very easily in a pipeline rules, and it will just create a field for each key with the value in it, these structured logs are the best kind because they are so easy to parse.
Hi Joel
This message example is just a screenshot found online… the pipeline rules you are talking about, could we make that so the structured fields are added? so the original source log is unchanged? (yes… I know… I’m a noobie 
)
Best regards
/Flemming
Yes, generally when you do this kind of thing you will leave the message field alone, then you may need to have a pipeline rule with some regex that clean up the format first, often there is some text before the JSON or things like that. Then you can either use key values or flatten_json to, then lastly you will write that back into the message using set_fields, and you will have a whole bunch of new fields with corresponding values in it.
The equivalent in Graylog for device_name and device_ip are probably source and gl2_remote_ip (depeding how you collect logs).
Then you need extractors or pipeline rules to extract more fields.
If you want to automatically resolve IP/hostname you can create a lookup and create a pipeline rule or extractor that uses the lookup
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
