OK, I am still trying to get our set up to work with Graylog and am having trouble with the sidecar-collector. Here is what I have:
server send logs from: Red Hat Enterprise Linux Server release 7.3, running graylog-collector-sidecar with nxlog. The collector is generating a nxlog.conf file like this:
define ROOT /usr/bin
<Extension gelf>
Module xm_gelf
</Extension>
User nxlog
Group nxlog
Moduledir /usr/libexec/nxlog/modules
CacheDir /var/spool/collector-sidecar/nxlog
PidFile /var/run/graylog/collector-sidecar/nxlog.pid
define LOGFILE /var/log/graylog/collector-sidecar/nxlog.log
LogFile %LOGFILE%
LogLevel INFO
<Extension logrotate>
Module xm_fileop
<Schedule>
When @daily
Exec file_cycle('%LOGFILE%', 7);
</Schedule>
</Extension>
<Input 59fcda86ccba8e2573422cb4>
Module im_file
File '/var/log/httpd/*_access_log'
PollInterval 1
SavePos True
ReadFromLast True
Recursive True
RenameCheck False
Exec $FileName = file_name(); # Send file name with each message
</Input>
<Output 59fcda56ccba8e2573422c80>
Module om_tcp
Host graylog.library.vanderbilt.edu
Port 5044
OutputType GELF_TCP
Exec $short_message = $raw_event; # Avoids truncation of the short_message field.
Exec $gl2_source_collector = '485f3ca7-ca1e-4959-be00-117a50e2b1db';
Exec $collector_node_id = 'graylog-collector-sidecar';
Exec $Hostname = hostname_fqdn();
</Output>
<Route route-0>
Path 59fcda86ccba8e2573422cb4 => 59fcda56ccba8e2573422c80
</Route>
But in my nxlog.log on this server, I am seeing these messages:
2017-11-06 12:25:12 ERROR om_tcp detected a connection error;End of file found
2017-11-06 12:25:13 INFO connecting to graylog.library.vanderbilt.edu:5044
2017-11-06 12:25:24 INFO reconnecting in 1 seconds
2017-11-06 12:25:24 ERROR om_tcp detected a connection error;End of file found
2017-11-06 12:25:25 INFO connecting to graylog.library.vanderbilt.edu:5044
2017-11-06 12:26:08 INFO reconnecting in 1 seconds
2017-11-06 12:26:08 ERROR om_tcp detected a connection error;End of file found
2017-11-06 12:26:09 INFO connecting to graylog.library.vanderbilt.edu:5044
And on my Graylog server, I am seeing the following errors:
2017-11-06T12:28:50.670-06:00 ERROR [NettyTransport] Error in Input [Beats/59e8c322ccba8e11c8e9f514] (channel [id: 0x4ab85d41, /129.59.95.20:39170 => /129.59.151.61:5044])
So it appears my issue is with my INPUT setup on my Graylog server, but I do not know what to change or how to fix it. What I am trying to do is to send all of my Apache access log files to Graylog for analysis (and want to do this across all of our servers), but I just need help getting started. Please advise. Thank you!
Current INPUT configuration:
Name = apachein
Forward to (Required) = apacheout [nxlog]
Type = [NXLog] file input
Path to Logfile = /var/log/httpd/*_access_log
Poll Interval = 1
Save read position = checked
Read since start = checked
Recursive file lookup = checked
Rename check = unchecked
Enable multiline = unchecked
Start pattern of multiline = /^-./ (I am assuming that since “Enable multiline” is unchecked that this will be ignored)
Stop pattern of multiline =
Additional Fields =
Jamen McGranahan