Creating fields from query (pipelines?)

Hi Everyone

So i have been testing Graylog’s capabilities with parsing complex URL’s
what i’m trying to achieve is relatively simple but getting a graylog pipeline to play ball is
harder than i thought it would be.

so I have a field that is created called query which extracts all information after the ? using grok patterns in the url in the log, so for my query i have this:

so for the pipeline I was thinking along these lines but i cannot seem to get it to be happy

rule "map query to fields"
    let data = parse_json(to_string($message.query));

Your problem not fully clear for me, but.
You wrote a simple sting,and a pipeline, what try to process json.
Your sting not in json format.

I suggest something like that

yeah I knew the parse json was a longshot,

but what im basically trying to do is parse every single parameter given into the url
to be mapped into its own field

Make sure that you have your logs in the correct format and then use a key value extractor using the & symbol to separate your fields on

Search the forums for several examples of using the key value extractor in a pipeline


There is no real good way to do this with regular expressions (or json parsing since it’s not json :D) but the key/value extractor may do the trick. Alternatively, change the way your application logs, if possible, to do it in full/proper json, then it’s easy enough to pull out with the json parser.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.