Apache sending 3 logs in one. Can Graylog split them?


(João Ciocca) #1

I tried some google fu and, once again, came up emtpy. So, the guys at production are forwarding Apache logs to my Graylog (I think via syslog), but it seems I’m receaving 3 logs in each log. Is there a easy way to split them on Graylog? Production guys says they can’t/don’t know how to solve this on their end.


(Jan Doberstein) #2

Hej @joaociocca

what does it mean 3 logs in each log?

Are you receiving the logs of three virtual hosts in the same stream of messages?


(João Ciocca) #3

nop, the same one is condensing 3 logs by log sent. I get 3 [timestamp][log], separated each in one line. The logs are not repeated, they’re different ones.


(Jan Doberstein) #4

could you please sample that? I’m still not getting what you receive at Graylog and what you like to archive.


(João Ciocca) #5

The log received by Graylog has all of this on the message field:

[Thu Jul 20 19:09:08 2017] [error] [client ip] PHP Notice:  error message
[Thu Jul 20 19:09:08 2017] [error] [client ip] PHP Notice:  error message
[Thu Jul 20 19:09:08 2017] [error] [client ip] PHP Notice:  error message

All three logs differ in “error message”, but have the same client ip. This is a sample of a single log entry’s message field received by Graylog.


(Scampuza) #6

hi Joao ,

It seems to be a problem with the Apache/PHP logger. Could you please share with me the Apache log configuration?


(João Ciocca) #7

Thanks @scampuza! The production guys sent me this:

#ErrorLog logs/error_log
ErrorLog "| tee -a  /var/log/httpd/error_log | nc -u -j <graylog_ip> 514"

(Scampuza) #8

Joao,

I would recommend using a Collector Sidecar for capturing the messages from the Log Files, for me, its a more reliable way for doing it. I use “nc …” for testing purposes, but not in production.

If you need any help in installing and configuring the Sidecar Collector, let me know


(João Ciocca) #9

I’ll look into it, thanks @scampuza!


(Billy Vierra) #10

This is almost always caused by buffering of the logs… basically [apache|tee|nc] isnt trying to write the error log immediately but batch them up until it hits X bytes or Y time and then write them (to increase performance). (iirc all 3 support buffering, not sure if enabled by default)

Since you are piping directly to nc all 3 lines are written at once and thus graylog receives them as a single message.


(João Ciocca) #11

I see. thanks for the explanation! =D

oh, I forgot to give you guys an update! We installed the sidecar and everything is smoothly now <3


(system) #12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.