Nginx grok parser dont work in pipeline, works in extractor

Graylog Central (peer support)
1

this line works in grok extractor but not in grok in pipeline
should it not be the same syntax for get it working ? i guess its the " in the pattern but still…

%{IPORHOST:clientip} (?:-|(%{WORD}.%{WORD})) %{USER:ident} [%{HTTPDATE:timestamp}] “(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})” %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{QS:forwarder}

// Anders

2

It would help if you posted the rule you were trying to get to work so we could see it in context.

That being said you are on the right track, you have to escape “special” things like quotes to have them in the pipeline.

"
Should be:

\"

3

hi yes but the thing is that the rule works in extractor that but not in the pipeline… :slight_smile:
and thanks for the info that i need to esc it out… :+1:

4

rule “[NGINX] parse nginx access”
when
has_field(“message”)
then

let mess = to_string($message.message);

let parsed = grok(pattern: “%{IPORHOST:clientip} (?:-|(%{WORD}.%{WORD})) %{USER:ident} [%{HTTPDATE:timestamp}] “(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})” %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{QS:forwarder}”, value: mess, only_named_captures: true);

set_fields(parsed);

end

5

i added the pattern in Grok patterns
NGINX_ACCESS %{IPORHOST:clientip} (?:-|(%{WORD}.%{WORD})) %{USER:ident} [%{HTTPDATE:timestamp}] “(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})” %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{QS:forwarder}

then added it in the pipeline now it works

rule “[NGINX] parse nginx access”
when
has_field(“message”)
then

let mess = to_string($message.message);
let parsed = grok(pattern: “%{NGINX_ACCESS}”, value: mess, only_named_captures: true);

set_fields(parsed);

end

1 Like
6

you have quotes that in the message - and other characters that need to be quoted in the processing pipeline.

That is the reason - AFAIK

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.