Can't listen messages on Syslog TCP via rsylog or syslog-ng

Zurek · November 22, 2017, 9:58am

Hi i have some problems with listening syslog messages on TCP. On UDP everythig works fine, but on TCP i receive noting. I uses examples from https://github.com/Graylog2/graylog-guide-syslog-linux/blob/master/README.md in rsyslog conf file and syslog-ng. Graylog can read netcat messages on any port with Raw/Plaintext TCP. But i need to read logs like …/log/message. I’m using Centos7.

sachin · November 22, 2017, 1:12pm

Can you check if the linux firewall is blocking it.

Zurek · November 22, 2017, 6:21pm

Firewall is turned off.

jochen · November 22, 2017, 6:22pm

Please post the configuration of the Syslog inputs in Graylog and the full configuration of your syslog clients.

Zurek · November 23, 2017, 7:27am

Graylog input:

allow_override_date:
true
bind_address:
0.0.0.0
expand_structured_data:
false
force_rdns:
false
max_message_size:
2097152
override_source:
<empty>
port:
1250
recv_buffer_size:
1048576
store_full_message:
false
tcp_keepalive:
false
tls_cert_file:
<empty>
tls_client_auth:
disabled
tls_client_auth_cert_file:
<empty>
tls_enable:
false
tls_key_file:
<empty>
tls_key_password:
********
use_null_delimiter:
false

rsyslog

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
$ModLoad immark  # provides --MARK-- message capability

*.*@192.168.0.1:1251;RSYSLOG_SyslogProtocol23Format    #UDP
*.*@@192.168.0.1:1250;RSYSLOG_SyslogProtocol23Format   #TCP

syslog-ng

source s_client { system(); internal(); };

destination d_network { 
syslog("192.168.0.1" port(1250));
};

log { source(s_client); destination(d_network); };

jochen · November 23, 2017, 8:05am

I think there’s a whitespace missing between the message qualifier and the remote address in your rsyslog configuration:

*.* @@192.168.0.1:1250;RSYSLOG_SyslogProtocol23Format

What’s the output of the following commands on the machine running Graylog?

sudo ip addr show
sudo netstat -tplen
sudo iptables -L -n

Zurek · November 23, 2017, 8:21am

ip addr show:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:8a:83:d4 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3
   valid_lft 86211sec preferred_lft 86211sec
inet6 fe80::f98e:6db:9f0d:febb/64 scope link 
   valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:32:f8:44 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.1/24 brd 192.168.0.255 scope global enp0s8
   valid_lft forever preferred_lft forever
inet6 fe80::4790:2d02:afa9:b7ac/64 scope link 
   valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
link/ether 52:54:00:28:48:c4 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
   valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
link/ether 52:54:00:28:48:c4 brd ff:ff:ff:ff:ff:ff

netstat -tplen

Proto Recv-Q Send-Q Local Address           Foreign Address         State            User       Inode      PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      0          14348      1/systemd           
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      0          20941      1449/dnsmasq        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          18908      970/sshd            
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      0          18394      960/cupsd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      0          20051      1293/master         
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      990        20174      1300/mongod         
tcp6       0      0 :::111                  :::*                    LISTEN      0          14347      1/systemd           
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      989        23202      972/java            
tcp6       0      0 ::1:9200                :::*                    LISTEN      989        23201      972/java            
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      989        22860      972/java            
tcp6       0      0 ::1:9300                :::*                    LISTEN      989        21988      972/java            
tcp6       0      0 :::22                   :::*                    LISTEN      0          18910      970/sshd            
tcp6       0      0 ::1:631                 :::*                    LISTEN      0          18393      960/cupsd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      0          20052      1293/master         
tcp6       0      0 :::1250                 :::*                    LISTEN      988        29018      2835/java           
tcp6       0      0 192.168.0.1:9000        :::*                    LISTEN      988        29010      2835/java

iptables -L -n:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:53
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:53
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:67
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:67

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24     ctstate RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:68

After adding space between * and @ nothing changes, UDP syslog works TCP syslog not.

jochen · November 23, 2017, 8:51am

Try changing the Bind Address (bind_address) of your Syslog TCP input to 192.168.0.1.

Zurek · November 23, 2017, 9:22am

Yeah, I tried it before and still nothing, on 192.168.0.1 like on 0.0.0.0.

jochen · November 23, 2017, 10:04am

What’s the output of the previously mentioned commands when binding the Syslog TCP input to 192.168.0.1?

Zurek · November 23, 2017, 10:27am

ip addr show:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:8a:83:d4 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3
   valid_lft 86212sec preferred_lft 86212sec
inet6 fe80::f98e:6db:9f0d:febb/64 scope link 
   valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:32:f8:44 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.1/24 brd 192.168.0.255 scope global enp0s8
   valid_lft forever preferred_lft forever
inet6 fe80::4790:2d02:afa9:b7ac/64 scope link 
   valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
link/ether 52:54:00:28:48:c4 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
   valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
link/ether 52:54:00:28:48:c4 brd ff:ff:ff:ff:ff:ff

netstat -tplen

Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      0          13956      1/systemd           
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      0          21280      1457/dnsmasq        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          20759      968/sshd            
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      0          20748      965/cupsd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      0          20870      1207/master         
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      990        21051      1298/mongod         
tcp6       0      0 :::111                  :::*                    LISTEN      0          13955      1/systemd           
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      989        23716      989/java            
tcp6       0      0 ::1:9200                :::*                    LISTEN      989        23715      989/java            
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      989        23633      989/java            
tcp6       0      0 ::1:9300                :::*                    LISTEN      989        22400      989/java            
tcp6       0      0 :::22                   :::*                    LISTEN      0          20761      968/sshd            
tcp6       0      0 ::1:631                 :::*                    LISTEN      0          20747      965/cupsd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      0          20871      1207/master         
tcp6       0      0 192.168.0.1:1250        :::*                    LISTEN      988        29572      2686/java           
tcp6       0      0 192.168.0.1:9000        :::*                    LISTEN      988        28635      2686/java

iptables -L -n

Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp – 0.0.0.0/0 0.0.0.0/0 udp dpt:53
ACCEPT tcp – 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
ACCEPT udp – 0.0.0.0/0 0.0.0.0/0 udp dpt:67
ACCEPT tcp – 0.0.0.0/0 0.0.0.0/0 tcp dpt:67

Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all – 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all – 192.168.122.0/24 0.0.0.0/0
ACCEPT all – 0.0.0.0/0 0.0.0.0/0
REJECT all – 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all – 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp – 0.0.0.0/0 0.0.0.0/0 udp dpt:68

Zurek · November 27, 2017, 8:59am

So, i can do nothing now?

jochen · November 27, 2017, 12:01pm

What’s the result when you try to send a manually crafted message to Graylog’s Syslog TCP input?

# echo '<0>1 2017-11-27T15:00:00.000Z myhost example message' | nc 192.168.0.1 1250

Zurek · November 28, 2017, 9:09am

Okay, seems to work on TCP syslog for nc commad now too, so i think it’s problem on clients side then? But funny, UDP and TCP on rsyslog have almost same config and UDP works TCP not :V

system · December 12, 2017, 9:09am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
TCP Syslog does not work Graylog Central (peer support)	3	1329	May 28, 2019
RSYSLOG UDP works but not SYSLOG TCP Graylog Central (peer support)	9	12246	October 6, 2017
[solved] How i get rsyslog messages in graylog? misunderstanding Graylog Central (peer support)	5	9057	February 1, 2019
No message sources found Graylog Central (peer support)	6	1717	April 5, 2017
Syslog UDP and Raw/Plaintext UDP not accepting input, TCP works Graylog Central (peer support)	2	1911	July 18, 2018

Can't listen messages on Syslog TCP via rsylog or syslog-ng

Related topics