Rsyslog cannot connect - Permission denied 2027 on Graylog server

Graylog Central (peer support)
1

1. Describe your incident:

Good Morning,

I have a centos stream release 8 server and my rsyslog does not connect to my graylog server.

image
image1412×439 197 KB

2. Describe your environment:

  • OS Information:
    graylog server: debian 10
    host connection error: Centos Stream 8

image

  • Package Version:
    graylog server: 10 buster
    host connection error: Stream 8

  • Service logs, configurations, and environment variables:

host connection error:

image
image1413×182 65.4 KB

3. What steps have you already taken to try and solve the problem?

I’ve already disabled firewall-cmd on both machines, I’ve changed it to low port and high port, reviewed the rsyslog settings, updated the rsyslog version and tried to connect to another linux server on the same port and the connection worked.

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

Hello @mxroot

Need to ask a couple question about this environment. Could you show how Rsyslog is configured and the configurations for the Graylog INPUT ? Is the log shipper ( Rsyslog) on the same node as Graylog? or is it on a remote node? Is the correct INPUT for Rsyslog being used, meaning the log/s sent are Syslogs and the Input being used is Syslog UDP?

3

hi gsmith, thanks for the reply.

Both INPUT in graylog and rsyslog configuration are compliant.

The server is on the same track, 10.1.102.0/24

strange, because I have 3 other linux servers (x1 centos 7 and x2 debian 11) but this is the only centos 8 server I have in my environment and they are all configured for the same INPUT and the entry is Syslog TCP.

I did a test pointing to UDP, however, it did not work.

INPUT_01
INPUT_01527×952 56.2 KB

INPUT_02
INPUT_02532×955 60.8 KB

rsyslog_config
rsyslog_config761×407 126 KB

4

Hello,

I see , so there are other Linux servers that are working correct and its only from CentOS 8 server your having a issue with, is this correct?

Not sure what test you performed but have you tried a tcpdump on Graylog server to see if any messages are arriving?

If they are not then check Selinux/Apparmor, Services or firewall/s ?

If they are, for troubleshooting I would create a new INPUT using Raw/Plaintext and some new port like 5044, etc… to see if its a INPUT or log format issue. I would tail my Graylog and elasticsearch file for anything pertaining to this problem.

EDIT:
Here is an example of what you should see with a tcpdump. Depending on the server in question, you may have to wait for logs to be shipped.

[root@graylog log]# tcpdump  host gsmith.domain.com and port 51412
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:43:16.696752 IP gsmith.domain.com.25089 > graylog.domain.com.51412: Flags [P.], seq 433158213:433159020, ack 1056193768, win 8207, length 807
17:43:16.696789 IP graylog.domain.com.51412 > gsmith.domain.com.25089: Flags [.], ack 807, win 52883, length 0
17:46:30.806513 IP gsmith.domain.com.25089 > graylog.domain.com.51412: Flags [.], seq 807:3727, ack 1, win 8207, length 2920
17:46:30.806568 IP graylog.domain.com.51412 > gsmith.domain.com.25089: Flags [.], ack 3727, win 52792, length 0

Also did you try changing these setting that would correspond to the INPUT in use?

UDP:
*.* @graylog.example.org:12201;RSYSLOG_SyslogProtocol23Format

TCP:
*.* @@graylog.example.org:12201;RSYSLOG_SyslogProtocol23Format
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.