GELF UDP Input switch between Global and Local


#1

Good evening. I’ve got a busy input - GELF UDP. Services sending logs for this input set remote host as an A record for the IPs of the three Graylog server nodes in the cluster I have inherited. A couple of days ago I changed this input from ‘Global input’ to run on one of three Graylog server nodes. Later on that day I changed it back to Global. Now looking back I see, during the period where the input was set to one Graylog server, messages missing from services that log via this input. These are services that are in constant use and log messages all the time but for which there are zero messages recorded during the hours where the input was not global.

I’m trying to explain the missing messages.

It occurs to me that I did not make a change to the A record to remove the two IPs for the Graylog servers that did not have this input running. As this is UDP we don’t have much to trace from the owners of the services that are missing messages. There is nothing much in the Graylog server logs. Detail for the single node that was running the input continued to record incoming and outgoing messages all through the time where it was the only node taking the input.

In addition when I changed the input back to global the input didn’t appear active on the other two Graylog nodes. I needed to sudo graylog-ctl restart to make that happen. Is that expected or should we be able to switch between without a restart?

Thanks very much.

Graylog version 2.3.2


(Jan Doberstein) #2

When you shutdown the Input - but did not change the A-Record for the round-robin dns loadbalancing and you use UDP the messages are lost.

What you described that the input does not came up again, might be a bug. But would you please update first to the current stable release and test again?


#3

Good morning @jan Thanks very much for confirming that.

Sure, will plan in an upgrade and check it again.


(system) #4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.