Problems with Grok Pattern for Apache 2.4 Error logs

Graylog Central (peer support)
1

Hello,

this is my first message but I’ve used a lot of all the resources of the Community, so thank you, everyone.

I am trying to parse Error Logs from Apache 2.4 using a Grok Pattern but I cannot do it. I am using this Grok Pattern:

[%{HTTPDERROR_DATE:timestamp}] [%{WORD:module}:%{LOGLEVEL:loglevel}]
[pid %{POSINT:pid}(:tid %{NUMBER:tid})?](
(%{POSINT:proxy_errorcode})%{DATA:proxy_message}:)?( [client
%{IPORHOST:clientip}:%{POSINT:clientport}])?( %{DATA:errorcode}:)?
%{GREEDYDATA:message}

Here you can see some log error entries:
[Fri Aug 10 05:21:26.511633 2018] [core:error] [pid 58722] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET /…/…/…/…/…/…/…/ HTTP/1.1
[Fri Aug 10 05:21:29.472006 2018] [core:error] [pid 53626] [client xx.xx.xxx.xxx:xxxxx] AH00126: Invalid URI in request GET @www.q5u4a3l2y1s.com HTTP/1.0
[Fri Aug 10 05:21:29.505785 2018] [core:error] [pid 58988] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET @www.q5u4a3l2y1s.com HTTP/1.0
[Fri Aug 10 05:21:29.665022 2018] [core:error] [pid 58722] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET qualys:@www.q5u4a3l2y1s.com HTTP/1.0
[Fri Aug 10 05:21:29.699173 2018] [core:error] [pid 53626] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET qualys:@www.q5u4a3l2y1s.com HTTP/1.0
[Fri Aug 10 05:21:39.767464 2018] [core:error] [pid 58837] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET /…/…/…/…/…/…/…/…/…/ HTTP/1.1
[Fri Aug 10 05:21:59.131454 2018] [core:error] [pid 58152] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET /%22%3e%3cscript%3ealert(document.domain)%3c/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd HTTP/1.1
[Fri Aug 10 05:21:59.651522 2018] [core:error] [pid 58002] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET /admin/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd HTTP/1.1
[Fri Aug 10 05:22:00.327068 2018] [core:error] [pid 57471] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET /install/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd HTTP/1.1
[Fri Aug 10 05:23:13.339605 2018] [core:error] [pid 58775] [client xx.xx.xx.xx:xxxxx] AH00126: Invalid URI in request GET /%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd HTTP/1.1

I’ve used Grok Constructor to test the grok pattern and the results are OK but when I am trying to use this Grok Pattern as Graylog Extractor I always get the same error: “Attention
We were not able to run the grok extraction. Please check your parameters.” I’ve checked on graylog nodes logs but nothing else appeared."

I’ve also checked that all the patterns that are used on the extractor are under System - Grok Patterns.

Anyone has used this Grok Pattern and was able to make it work?

Thank you very much!

Alex.

2

Alex:

I see you use the pattern HTTPDERROR_DATE. That does not appear to be a default pattern loaded in Graylog. Did you make a custom pattern? If so, can you provide that pattern?

3

Hello Megan, thank you very much for your answer. I’ve used http grok patterns that I’ve found here: http://grokconstructor.appspot.com/groklib/httpd and here: logstash-patterns/files/grok-patterns at master · hpcugent/logstash-patterns · GitHub

I’ve tested the grok_pattern with some apache error log entries and works perfectly. I’ve also import these grok patterns to Graylog but always I try to use them I’ve got an error: “Attention
We were not able to run the grok extraction. Please check your parameters.”

Regards,

4

Have you tried escaping the brackets? Put a \ in front of all [ and ]

5

Ye, I’ve tried escaping the brackets, I’ve also tried escaping the escape characters (double \) but it does not work either.
I’ve tried even to replace [ ] with ? characters in the grok pattern, but it does not work

6

Okay, I was having some success with escaping the brackets when I was testing but had to get back to work. I will try to look at it again later. I highly recommend testing with https://grokdebug.herokuapp.com/ . Make sure you add your custom pattern (the HTTPDERROR_DATE). I’ll see what I can do later if you haven’t figured it out yet.

7

Hello Megan,

I’ve tested the grok expression (without escaping []) in https://grokdebug.herokuapp.com/ and also in http://grokconstructor.appspot.com/do/match and works well.

The problem happens when I try to use the Grok Expression directly as a Graylog Extractor.

I keep doing tests.

Thank you very much!

8

you might want to create a custom GROK Pattern in Graylog and assign a name. Then use that name in the extractor.

When I did not make it work when using the complete string this is my personal work-around.

9

Hello Ian, thank you for your answer.

The problem is that I don’t know which is the problem with the expression. I’ve tried to modify it and customized it, but I always get the same result: when I tested on GrokDebug or GrokConstructor works, but when I added to Graylog as a new Grok Pattern it doesn’t work, showing the message: "“Attention: We were not able to run the grok extraction. Please check your parameters.” but no other information, neither on log files.

The weird thing is that I used also Grok Patterns to parse Apache access logs (with very similar Grok expressions that the error one) and works perfectlly I don’t understand why thesee other Grok Patterns are not working.

Regards,

10

try the mentioned step.

Save your custom Pattern in Graylog (System > Grok-Pattern) and use in the extractor only your single word.

11

Hello Jan,

I’ve tried but with the same results: "“Attention: We were not able to run the grok extraction. Please check your parameters.”

Thank you very much,

Alejandro

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.