Hi Team,
I have messages coming in without being parsed hence I wrote my custom Grok parsers but then wanted to apply in Graylog for those messages.
Can someone help me with this?
TIA
Blason R
Did you wrote a Grok pattern that does not work? Did you have a pattern that should only match to specific messages?
Your request is not clear!
My bad for not being specific.
Let me start, I have below logs coming in Graylog from filebeat
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] connect
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: GET /malware.doc HTTP/1.1
[2018-11-12 08:39:33] [2250] [http_80_tcp 2642] [192.168.5.103:49499] connect
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: Host: 192.168.5.43
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: Connection: keep-alive
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: Upgrade-Insecure-Requests: 1
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: DNT: 1
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: Accept-Encoding: gzip, deflate
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] recv: Accept-Language: en-GB,en-US;q=0.9,en;q=0.8,hi;q=0.7
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] info: Request URL: http://192.168.5.43/malware.doc
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] info: Sending fake file configured for extension ‘doc’.
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] send: HTTP/1.1 200 OK
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] send: Content-Length: 26624
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] send: Date: Mon, 12 Nov 2018 03:09:33 GMT
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] send: Content-Type: application/word
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] send: Connection: Close
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] send: Server: Microsoft-IIS/8.0
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] info: Sending file: /var/lib/inetsim/http/fakefiles/sample.doc
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] stat: 1 method=GET url=http://192.168.5.43/malware.doc sent=/var/lib/inetsim/http/fakefiles/sample.doc postdata=
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] disconnect
Of course those are not being parsed hence I picked the message which is most relevant to me which is
[2018-11-12 08:39:33] [2250] [http_80_tcp 2641] [192.168.5.103:49498] info: Request URL: http://192.168.5.43/malware.doc
And wrote a parser for this. However wanted to know how do I apply my custom extractor/parser only to this specific log since there are other around 20-25 other lines as well. I save my custom extractor in extractor section and now I wanted to apply that so that can build dashboard on
- URI Pattern [http://192.168.5.43/malware.doc]
- Client host [[192.168.5.103:49498]
- And port client has connected [[http_80_tcp 2641]
TIA
Blason R
without knowing your setup - but from what i can see I would advice using the processing pipelines to run that only on the given messages.
When formatting your post ( https://community.graylog.org/faq#format-markdown ) in a propper way you make it more easy for other to help you.
Surely thanks. Will take a note of it.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.