Issues receiving TCP Syslog & TCP Raw

Hi there

I currently have an A10 networks device sending Syslog messages (RFC5424) via UDP to a 3-node Graylog cluster on a Global UDP input (UDP 1514). This seems to work fine, but I am looking to make everything inherently more ‘reliable’ and would like to move the comms over to TCP instead.

Graylog refuses to process messages received from this device via TCP though. Syslog TCP Inputs do not ingest any messages at all, and Raw TCP Inputs only successfully ingest messages from this device upon shutting down the Input (as if it’s flushing a buffer or journal when doing so) - I have tried this on TCP 1514 and TCP 5555 with the same results.

I have run packet captures and can confirm that the actual content of the Syslog message is the same regardless of TCP/UDP transport methods - as one would expect.

I do not seem to be able to get any information from Graylog’s debug/trace functionality about the ignored/discarded data, and also have not observed any of the common solutions applying (future-dated timestamps, journal full, disk full, etc). The input statistics in Graylog’s web interface do recognise the network traffic hitting the Input in terms of Network input measurement. I’m not quite sure how to investigate further.

I have tested having the A10 device send TCP syslogs to a simple rsyslog collector, which works flawlessly with minimal configuration. Additionally, I have also successfully sent and ingested TCP Syslog messages to the Graylog cluster from a test machine that isn’t the A10 device.

All Graylog nodes are running on Ubuntu 18.04.3 LTS with UFW disabled and AppArmor disabled (for ease of debugging), Graylog 3.1.4+1149fe1 with MongoDB 4.0.15 and ElasticSearch 6.8.6. Was previously having this same problem on Graylog 3.0.2 with MongoDB 4.0.x and ES 6.8.6 too.

he @jakeb

just one idea - you could start a netcat on one system and use that as target for your network device.

My current idea is that this device might not send a \n or similar after the message that Graylog does not see it as new message at all. But that is just my :crystal_ball:

This was my first idea also.
If you can increase the amount of the messages on the sender device you should see one message when your TCP input cache is full, and it should contains multiple logs in one row.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.