I am new to Graylog and I am having some issues getting all of my UniFi syslog traffic working with Graylog. I have the Dream Machine Pro sending syslog via udp/5140. I have a local input configured and running for it and I am averaging 72 msg/s. So far so good. I am assuming that I next need to configure a stream so that I can search against the stream? I created a stream called UniFi with a simple rule: A message must match all of the following rules: “source must contain UDM-Pro”. I saved it and started it w/o any errors; however, it doesn’t appear to be working properly as if I try to do search against the UniFi stream, there is no data shown in the stream. I went back to the stream and tried Step 1 “Load a message to test rules”, select input = "UniFi Syslog Input (org.graylog2.inputs.syslog.udp.SyslogUDPInput), and then selected “Load Message”. This results in this error: “Input did not return a recent message.” This is confussing as when I look at the UniFi input it is showing an average of 72 messages per second. Any ideas what I did, or am doing, wring?
I’m assuming you’ve already verified that your message source matches the rule. That being the case, do you see messages in the All Messages stream? Or did you prior to this rule change?
This might be an issue with the message timestamp. If there is a mismatch these messages could be populating in the streams hours ahead or behind the search timeframe. Try generating a specific test message into Graylog and then searching for it, and if you don’t find it but you’re sure you should be then try expanding the search time frame to see if it has been stored with a timestamp in the future or past.
I did that, but it doesn’t look like I am getting anything in the All Messages stream. If I do a search with no streams selected (all streams) and set it to update every 1 second, the message count never goes above 0. In the upper right corner I see messages coming in and going out as the numbers keep changing.
IIRC debug logging might produce something. I run Unifi/Ubiquiti gear myself, and have noticed the same. I have it dropping some miniupnpd messages because those get noisy.
Since I haven’t put a lot of work into this yet, I am thinking about trying a clean install…
Well dang… just all of a sudden, I am now seeing messages in the All Messages stream. I didn’t do anything, honest. All I had to do was make a new Ubuntu installer and all of a sudden: messages! I am not going to mess with a new stream just now. I am going to let it run as-is all night and see if it is still working in the morning. If so, I will create a new stream and see what happens.
Any chance it could be h/w related? Since I am only testing this, I am not using server-grade h/w. It is a 3.2 GHz quad core CPU with 32 GB of RAM and an old 1TB, 5200 rpm, 2.5" drive that I had laying around.
I would be very surprised if it was hardware related on Graylog’s end. From what I’m seeing in Ubiquiti forums, there seem to be a number of issues with getting logs to remote systems. I think if anything, it’s more likely something on the Ubiquiti/Unifi end.
@irishjd in any case the disk seems to be the weak link there so you should be able to diagnose a hardware issue pretty easily by examining disk performance. You’ll also be able to see it manifest in Elasticsearch, which would ultimately be visible as a backup into the Graylog buffers/disk queue.
Since performance has come up, this is worth a read.
