Extract information out of message field

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

Hi! I am trying to extract information coming from aws flowlogs in message filed as:

${srcaddr}    ${flow-direction} ${action} ${traffic-path} ${dstaddr}   ${region}  ${packets} ${protocol}          ingress         ACCEPT       -      us-east-1       23         6

in order to extract the ${srcaddr} ${flow-direction} ${action} ${traffic-path} from the whole message what can I do? What rule can be made please share an example to extract the above fields only and drop rest of the information.
the message field after filtration will look like this: ingress ACCEPT -


I have create extractors for these, either use regex or GROK

DestinationIP Regular expression
extract data from message into dstip.
    regex_value: dstip=+?((?:\d+\.){3}\d+).+




grok_pattern: %{IP}

Thanks for the update @gsmith
I am sorry I think I didn’t clarify the requirements. You are trying to add extractor based on only 1 parameter which is destination Ip but instead I want to extract few inputs (like ${srcaddr} ${flow-direction} ${action} ${traffic-path}) from the message filed

and store it in the message field and drop the rest of the fields that I am not interested in, ideal message filed after extraction should look like this:

I was trying to play with extractors and came across with this:

turns out I can’t store my extracted data into message field. Can you please let me know if there is any workaround?

Also by calling an api from 1 day time stamp of message field, is it possible to have it extracted via api call?

Hi Folks,

you could also create a Crok Pattern for it, click on the message > Create extractor then choose Grok Pattern:

Select Named Captures Only

Example : ingress REJECT - us-east-1 1 6

%{IPV4:srcaddr} %{WORD:flow-direction} %{WORD:Action} \- %{IPV4:destip} %{GREEDYDATA:location}

@Aksel when I click “try against example” I am getting this against your example,

Please use the line i sent to you, and load the same message as you posted before

1 Like

I used the same line as yours and the example message is also correct.

Sorry. It is working as I had new line space at the bottom that was hindering the execution.

can you please let me know where the extracted message will be going? How can I view them?

When new same like messages comes in, the Fields will be extracted and these Fields will be shown before or after the message field: srcaddr, flow-direction, action etc …

see this example:

you can also use the Graylog search engine and search for: srcaddr: * AND Action: “REJECT” AND flow_direction: “ingress”

Btw go back to the created grok pattern and change flow-direction to flow_direction and save


thanks a lot! btw which api can I use to extract these fields from last 24hours timestamp?
Please note that I have tried /api/messages/index/messageID but it retrieves only one result. I need to have results for last 24 hours

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.