How to save all incoming syslog to Graylog-server


(atul) #1

I install graylog-2.3.0-1 OVA @ VM-Ware all things are fine they are working properly. I created a input for switches and routers at 514 port messages are received and working properly, but i want to store all incoming logs in different directory with host name or by their IP’s As in Syslog server. While i configured the rsyslog.conf to store message to store logs that input is not working (May be due to port ) How can configure so that they both work simultaneously.

Logs

2017-09-22_07:42:44.91562 INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now FAILED%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now STOPPING%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now STOPPED%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now TERMINATED%INFO  [InputStateListener] Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] is now STARTING%ERROR [NettyTransport] Error in Input [Raw/Plaintext UDP/59c4be52896f3a039cddbbc6] (channel [id: 0xa37e180c])% java.net.BindException: Address already in use
2017-09-22_07:42:44.91840       at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_131]
2017-09-22_07:42:44.91878       at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_131]
2017-09-22_07:42:44.92479       at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) ~[?:1.8.0_131]
2017-09-22_07:42:44.92548       at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) ~[?:1.8.0_131]
2017-09-22_07:42:44.92760       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129) [graylog.jar:?]
2017-09-22_07:42:44.93107       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77) [graylog.jar:?]
2017-09-22_07:42:44.93316       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) [graylog.jar:?]
2017-09-22_07:42:44.93638       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) [graylog.jar:?]
2017-09-22_07:42:44.93773       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) [graylog.jar:?]
2017-09-22_07:42:44.94010       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [graylog.jar:?]
2017-09-22_07:42:44.94320       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) [graylog.jar:?]
2017-09-22_07:42:44.94543       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) [graylog.jar:?]
2017-09-22_07:42:44.94789       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) [graylog.jar:?]
2017-09-22_07:42:44.95014       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [graylog.jar:?]
2017-09-22_07:42:44.95229       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) [graylog.jar:?]
2017-09-22_07:42:44.95440       at org.jboss.netty.channel.Channels.bind(Channels.java:561) [graylog.jar:?]
2017-09-22_07:42:44.95729       at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:197) [graylog.jar:?]
2017-09-22_07:42:44.95979       at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:198) [graylog.jar:?]
2017-09-22_07:42:44.96295       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) [graylog.jar:?]
2017-09-22_07:42:44.96509       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) [graylog.jar:?]
2017-09-22_07:42:44.96809       at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
2017-09-22_07:42:44.97056       at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
2017-09-22_07:42:44.97373       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_131]
2017-09-22_07:42:44.97606       at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
2017-09-22_07:42:44.97903       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
2017-09-22_07:42:44.98122       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
2017-09-22_07:42:44.98410       at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-09-22_07:42:44.98594 ERROR [InputLauncher] The [org.graylog2.inputs.raw.udp.RawUDPInput] input with ID <59c4be52896f3a039cddbbc6> misfired. Reason: Address already in use.% org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:514
2017-09-22_07:42:44.98946       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:156) ~[graylog.jar:?]
2017-09-22_07:42:44.99199       at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
2017-09-22_07:42:44.99552       at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
2017-09-22_07:42:44.99840       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_131]
2017-09-22_07:42:45.00148       at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
2017-09-22_07:42:45.00387       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
2017-09-22_07:42:45.00677       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
2017-09-22_07:42:45.00877       at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-09-22_07:42:45.01275 Caused by: org.graylog2.plugin.inputs.MisfireException: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:514
2017-09-22_07:42:45.01506       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:155) ~[graylog.jar:?]
2017-09-22_07:42:45.01770       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) ~[graylog.jar:?]
2017-09-22_07:42:45.01978       ... 7 more
2017-09-22_07:42:45.02257 Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:514
2017-09-22_07:42:45.02421       at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:204) ~[graylog.jar:?]
2017-09-22_07:42:45.02717       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) ~[graylog.jar:?]
2017-09-22_07:42:45.02879       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) ~[graylog.jar:?]
2017-09-22_07:42:45.03216       ... 7 more
2017-09-22_07:42:45.03450 Caused by: java.net.BindException: Address already in use
2017-09-22_07:42:45.03848       at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_131]
2017-09-22_07:42:45.04071       at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_131]
2017-09-22_07:42:45.04395       at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) ~[?:1.8.0_131]
2017-09-22_07:42:45.04583       at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) ~[?:1.8.0_131]
2017-09-22_07:42:45.04884       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129) ~[graylog.jar:?]
2017-09-22_07:42:45.05063       at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77) ~[graylog.jar:?]
2017-09-22_07:42:45.05365       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) ~[graylog.jar:?]
2017-09-22_07:42:45.05525       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
2017-09-22_07:42:45.05854       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
2017-09-22_07:42:45.06087       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
2017-09-22_07:42:45.06471       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) ~[graylog.jar:?]
2017-09-22_07:42:45.06652       at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
2017-09-22_07:42:45.06928       at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
2017-09-22_07:42:45.07153       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
2017-09-22_07:42:45.07460       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) ~[graylog.jar:?]
2017-09-22_07:42:45.07636       at org.jboss.netty.channel.Channels.bind(Channels.java:561) ~[graylog.jar:?]
2017-09-22_07:42:45.07903       at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:197) ~[graylog.jar:?]
2017-09-22_07:42:45.08085       at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:198) ~[graylog.jar:?]
2017-09-22_07:42:45.08361       at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) ~[graylog.jar:?]
2017-09-22_07:42:45.08545       at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) ~[graylog.jar:?]
2017-09-22_07:42:45.08855       ... 7 more

(Jochen) #2

You’ll have to use a different port for either your rsyslog server or the Graylog input.
They cannot bind the same socket.


(atul) #3

If using different port how can we collect the log. All incoming logs are 514 so i should be run both Input as well as Rsyslog on same port to collect the logs and input to accept at 514.


(Jochen) #4

Why do you need to have rsyslog listen on port 514?
It’s perfectly fine if only Graylog listens on port 514 to collect the syslog messages sent to this port.


(atul) #5

So how can i collect incoming logs in different directory with their host name or IP.


(Jochen) #6

Do you need to collect the syslog messages with rsyslog additionally to Graylog?


(atul) #7

Yes so that i can read them easily. Coz in my environment They are not stored any where so i want to store them as raw as they received at graylog server.


(Jochen) #8

In this case you have to ingest the logs with rsyslog and forward them from there to Graylog.
The Graylog syslog input has to listen on another (unique!) port, i. e. not 514.

See https://github.com/Graylog2/graylog-guide-syslog-linux/blob/master/README.md for details.


(atul) #9

Hi Jochen, Now i receive the logs and can store them in different directory below is my rsyslog.conf. Now tell me how can i send them to gray log all message received at 514. As you can see i am able to receive graylog server logs at 5140 input configured but not those log which are configured to receive at 514 from switch and LB.

rsyslog.conf

#  /etc/rsyslog.conf    Configuration file for rsyslog.
#
#                       For more information see
#                       /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
#  Default logging rules can be found in /etc/rsyslog.d/50-default.conf


#################
#### MODULES ####
#################

$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support
#$ModLoad immark  # provides --MARK-- message capability

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

# Enable non-kernel facility klog messages
$KLogPermitNonKernelFacility on

###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf
#*.* @10.248.14.144:5140;
#$template TmplAuth, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplAuth, "/var/log/rsyslog_custom/%FROMHOST-IP%/%PROGRAMNAME%.log"
#$template TmplMsg, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplMsg, "/var/log/rsyslog_custom/%FROMHOST-IP%/%PROGRAMNAME%.log"
authpriv.*   ?TmplAuth
*.info,mail.none,authpriv.none,cron.none   ?TmplMsg
*.* @10.248.14.144:5140

(atul) #10

Just tell me that how can i send these “/var/log/rsyslog_custom/%FROMHOST-IP%/%PROGRAMNAME%.log” received log to gray log (They are all resides graylog server only). How can create a input to parse these received log.


(Jochen) #11

Please refer to the aforementioned syslog guide for how to configure forwarding syslog messages to Graylog:

The used message format is relevant (i. e. you’re missing the template in your forwarding directive).


(atul) #12

But i am able to parse the message generated by the graylog server see below directory contain the all logs received at 514 with local graylog ser as IP 127.0.0.1 …and my others logs localhost logs are parsed in my input which i configured to receive at 5140 port in graylog web interface. But others are not parsing inside that input.

root@graylog:/var/log/rsyslog_custom# ll
total 76
drwxr-xr-x 19 syslog syslog 4096 Sep 28 15:38 ./
drwxrwxr-x 12 root syslog 4096 Sep 28 13:41 …/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.120/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.121/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:48 10.248.0.13/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:51 10.248.0.14/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.157/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:16 10.248.0.159/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:13 10.248.0.5/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:31 10.248.22.181/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 13:41 10.248.22.32/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 13:41 10.248.22.33/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:37 10.248.22.41/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:37 10.248.22.42/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:12 10.248.22.43/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:40 10.248.22.44/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:38 10.248.22.45/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 15:38 10.248.22.46/
drwxr-xr-x 2 syslog syslog 4096 Sep 28 08:56 127.0.0.1/
root@graylog:/var/log/rsyslog_custom# pwd
/var/log/rsyslog_custom
root@graylog:/var/log/rsyslog_custom#

The bold letter are the logs of local server logs they are captured in that input but rest logs are not.


(system) #13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.