How to save all incoming syslog to Graylog-server

atul · September 26, 2017, 5:01am

I install graylog-2.3.0-1 OVA @ VM-Ware all things are fine they are working properly. I created a input for switches and routers at 514 port messages are received and working properly, but i want to store all incoming logs in different directory with host name or by their IP’s As in Syslog server. While i configured the rsyslog.conf to store message to store logs that input is not working (May be due to port ) How can configure so that they both work simultaneously.

Logs

2017-09-22_07:42:44.91562 INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now FAILED%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now STOPPING%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now STOPPED%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now TERMINATED%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now STARTING%ERROR [NettyTransport] Error in Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] (channel [id: 0xa37e180c])% java.net.BindException: Address already in use
2017-09-22_07:42:44.91840       at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_131]
2017-09-22_07:42:44.91878       at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_131]
2017-09-22_07:42:44.92479       at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) ~[?:1.8.0_131]
2017-09-22_07:42:44.92548       at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) ~[?:1.8.0_131]
2017-09-22_07:42:44.92760       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129) [graylog.jar:?]
2017-09-22_07:42:44.93107       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77) [graylog.jar:?]
2017-09-22_07:42:44.93316       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) [graylog.jar:?]
2017-09-22_07:42:44.93638       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) [graylog.jar:?]
2017-09-22_07:42:44.93773       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) [graylog.jar:?]
2017-09-22_07:42:44.94010       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [graylog.jar:?]
2017-09-22_07:42:44.94320       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) [graylog.jar:?]
2017-09-22_07:42:44.94543       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) [graylog.jar:?]
2017-09-22_07:42:44.94789       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) [graylog.jar:?]
2017-09-22_07:42:44.95014       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [graylog.jar:?]
2017-09-22_07:42:44.95229       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) [graylog.jar:?]
2017-09-22_07:42:44.95440       at org.jboss.netty.channel.Channels.bind(Channels.java:561) [graylog.jar:?]
2017-09-22_07:42:44.95729       at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:197) [graylog.jar:?]
2017-09-22_07:42:44.95979       at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:198) [graylog.jar:?]
2017-09-22_07:42:44.96295       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) [graylog.jar:?]
2017-09-22_07:42:44.96509       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) [graylog.jar:?]
2017-09-22_07:42:44.96809       at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
2017-09-22_07:42:44.97056       at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
2017-09-22_07:42:44.97373       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_131]
2017-09-22_07:42:44.97606       at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
2017-09-22_07:42:44.97903       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
2017-09-22_07:42:44.98122       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
2017-09-22_07:42:44.98410       at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-09-22_07:42:44.98594 ERROR [InputLauncher] The [org.graylog2.inputs.raw.udp.RawUDPInput] input with ID <59c4be52896f3a039cddbbc6> misfired. Reason: Address already in use.% org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:514
2017-09-22_07:42:44.98946       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:156) ~[graylog.jar:?]
2017-09-22_07:42:44.99199       at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
2017-09-22_07:42:44.99552       at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
2017-09-22_07:42:44.99840       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_131]
2017-09-22_07:42:45.00148       at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
2017-09-22_07:42:45.00387       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
2017-09-22_07:42:45.00677       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
2017-09-22_07:42:45.00877       at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-09-22_07:42:45.01275 Caused by: org.graylog2.plugin.inputs.MisfireException: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:514
2017-09-22_07:42:45.01506       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:155) ~[graylog.jar:?]
2017-09-22_07:42:45.01770       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) ~[graylog.jar:?]
2017-09-22_07:42:45.01978       ... 7 more
2017-09-22_07:42:45.02257 Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:514
2017-09-22_07:42:45.02421       at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:204) ~[graylog.jar:?]
2017-09-22_07:42:45.02717       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) ~[graylog.jar:?]
2017-09-22_07:42:45.02879       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) ~[graylog.jar:?]
2017-09-22_07:42:45.03216       ... 7 more
2017-09-22_07:42:45.03450 Caused by: java.net.BindException: Address already in use
2017-09-22_07:42:45.03848       at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_131]
2017-09-22_07:42:45.04071       at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_131]
2017-09-22_07:42:45.04395       at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) ~[?:1.8.0_131]
2017-09-22_07:42:45.04583       at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) ~[?:1.8.0_131]
2017-09-22_07:42:45.04884       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129) ~[graylog.jar:?]
2017-09-22_07:42:45.05063       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77) ~[graylog.jar:?]
2017-09-22_07:42:45.05365       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) ~[graylog.jar:?]
2017-09-22_07:42:45.05525       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
2017-09-22_07:42:45.05854       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
2017-09-22_07:42:45.06087       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
2017-09-22_07:42:45.06471       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) ~[graylog.jar:?]
2017-09-22_07:42:45.06652       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
2017-09-22_07:42:45.06928       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
2017-09-22_07:42:45.07153       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
2017-09-22_07:42:45.07460       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) ~[graylog.jar:?]
2017-09-22_07:42:45.07636       at org.jboss.netty.channel.Channels.bind(Channels.java:561) ~[graylog.jar:?]
2017-09-22_07:42:45.07903       at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:197) ~[graylog.jar:?]
2017-09-22_07:42:45.08085       at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:198) ~[graylog.jar:?]
2017-09-22_07:42:45.08361       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) ~[graylog.jar:?]
2017-09-22_07:42:45.08545       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) ~[graylog.jar:?]
2017-09-22_07:42:45.08855       ... 7 more

jochen · September 26, 2017, 6:48am

You’ll have to use a different port for either your rsyslog server or the Graylog input.
They cannot bind the same socket.

atul · September 26, 2017, 11:16am

If using different port how can we collect the log. All incoming logs are 514 so i should be run both Input as well as Rsyslog on same port to collect the logs and input to accept at 514.

jochen · September 26, 2017, 11:50am

Why do you need to have rsyslog listen on port 514?
It’s perfectly fine if only Graylog listens on port 514 to collect the syslog messages sent to this port.

atul · September 27, 2017, 4:14am

So how can i collect incoming logs in different directory with their host name or IP.

jochen · September 27, 2017, 8:47am

Do you need to collect the syslog messages with rsyslog additionally to Graylog?

atul · September 27, 2017, 10:28am

Yes so that i can read them easily. Coz in my environment They are not stored any where so i want to store them as raw as they received at graylog server.

jochen · September 27, 2017, 10:36am

In this case you have to ingest the logs with rsyslog and forward them from there to Graylog.
The Graylog syslog input has to listen on another (unique!) port, i. e. not 514.

See https://github.com/Graylog2/graylog-guide-syslog-linux/blob/master/README.md for details.

atul · September 28, 2017, 9:58am

Hi Jochen, Now i receive the logs and can store them in different directory below is my rsyslog.conf. Now tell me how can i send them to gray log all message received at 514. As you can see i am able to receive graylog server logs at 5140 input configured but not those log which are configured to receive at 514 from switch and LB.

rsyslog.conf

#  /etc/rsyslog.conf    Configuration file for rsyslog.
#
#                       For more information see
#                       /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
#  Default logging rules can be found in /etc/rsyslog.d/50-default.conf


#################
#### MODULES ####
#################

$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support
#$ModLoad immark  # provides --MARK-- message capability

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

# Enable non-kernel facility klog messages
$KLogPermitNonKernelFacility on

###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf
#*.* @10.248.14.144:5140;
#$template TmplAuth, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplAuth, "/var/log/rsyslog_custom/%FROMHOST-IP%/%PROGRAMNAME%.log"
#$template TmplMsg, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplMsg, "/var/log/rsyslog_custom/%FROMHOST-IP%/%PROGRAMNAME%.log"
authpriv.*   ?TmplAuth
*.info,mail.none,authpriv.none,cron.none   ?TmplMsg
*.* @10.248.14.144:5140

atul · September 28, 2017, 10:05am

Just tell me that how can i send these “/var/log/rsyslog_custom/%FROMHOST-IP%/%PROGRAMNAME%.log” received log to gray log (They are all resides graylog server only). How can create a input to parse these received log.

jochen · September 28, 2017, 10:05am

Please refer to the aforementioned syslog guide for how to configure forwarding syslog messages to Graylog:

github.com

Graylog2/graylog-guide-syslog-linux/blob/master/README.md#rsyslog

## Sending syslog from Linux systems into Graylog

The two most popular syslog deamons (the programs that run in the background to accept and write or forward logs) are [rsyslog](https://www.rsyslog.com/) and [syslog-ng](https://syslog-ng.com/). One of these will most likely be running on your Linux distribution.

Please refer to the documentation of your distribution if you are not sure about this.

### ⚠️ Warning ⚠️

**These instructions configure rsyslog and syslog-ng to send log messages _unencrypted_ over the network. This is generally not recommended on public networks.**

### rsyslog

Forwarding syslog messages with rsyslog is easy. The only important thing to get the most out of your logs is following
[RFC 5424](http://www.ietf.org/rfc/rfc5424.txt). The following examples configure your `rsyslog` daemon to send
RFC 5424 date to Graylog syslog inputs:

##### UDP:

    *.* @graylog.example.org:514;RSYSLOG_SyslogProtocol23Format

This file has been truncated. show original

The used message format is relevant (i. e. you’re missing the template in your forwarding directive).

atul · September 28, 2017, 10:26am

But i am able to parse the message generated by the graylog server see below directory contain the all logs received at 514 with local graylog ser as IP 127.0.0.1 …and my others logs localhost logs are parsed in my input which i configured to receive at 5140 port in graylog web interface. But others are not parsing inside that input.

root@graylog:/var/log/rsyslog_custom# ll
total 76
drwxr-xr-x 19 syslog syslog 4096 Sep 28 15:38 ./
drwxrwxr-x 12 root syslog 4096 Sep 28 13:41 …/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.120/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.121/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:48 10.248.0.13/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:51 10.248.0.14/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.157/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.159/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:13 10.248.0.5/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:31 10.248.22.181/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 13:41 10.248.22.32/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 13:41 10.248.22.33/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:37 10.248.22.41/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:37 10.248.22.42/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:12 10.248.22.43/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:40 10.248.22.44/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:38 10.248.22.45/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:38 10.248.22.46/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:56 127.0.0.1/
root@graylog:/var/log/rsyslog_custom# pwd
/var/log/rsyslog_custom
root@graylog:/var/log/rsyslog_custom#

The bold letter are the logs of local server logs they are captured in that input but rest logs are not.

system · October 12, 2017, 10:26am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Send syslog messages from Graylog host server to Graylog itself Graylog Central (peer support)	4	4267	May 31, 2022
Can't send rsyslog Graylog Central (peer support)	14	2864	September 9, 2017
Issues with Syslog input Graylog Central (peer support)	21	7708	August 22, 2017
Syslog input from another host to Graylog server Graylog Central (peer support)	2	896	July 2, 2019
Graylog inputs do not seem to be working Graylog Central (peer support)	5	2915	September 7, 2018

How to save all incoming syslog to Graylog-server

Related topics