Specific Logs not saving in Graylog

(Taylor Duncan) #1

Hello. This is my first time posting in this community. First off, thank you for taking the time to read this post. I have been using Graylog for a little over a year and it is an absolutely wonderful product. I have an issue with messages not being saved in Graylog.

I am sending messages from haproxy via rsyslog to graylog over UDP. These messages are all in JSON format. I am using a JSON extractor to extract the messages. I noticed some time ago that a few specific website logs were not making it to Graylog, so I began to investigate. I started by doing a packet capture on the Graylog Server. I navigated to the website, and immediately I was able to see the message in JSON from the TCP dump. So the message is making it to the Graylog Server.

I guess my question is how can the message make it to Graylog, but not be saved? I originally thought the message was just not extracting properly, but it turns out the message is not being saved at all by Graylog. Does anyone have any thoughts or suggestions? Thanks again in advance.

(Ben van Staveren) #2

I’ve seen this happen when Graylog expects the message to be in a certain format, if it can’t parse it (or it’s considered empty) it’ll be dropped quietly - for example if you send a non-syslog formatted message to a syslog input, it won’t make it past the input .

Seems a bit unlikely since I figure rsyslog will “do the right thing” in all cases, but that’s the only thing I can think of. Any chance you could post one of the messages that doesn’t make it, and one that does? That may allow us to get a better idea of what may be going on :slight_smile:

(Taylor Duncan) #3

Ok, so I hope someone can learn from this in the future. I was assuming the messages were not saved because I attempted to search for them and nothing appeared in the results. It turns out I was not searching properly for a substring on my “host_header” input, so no results were returning.

I just need to read up on proper searching in Graylog. Thanks again for responding!

(Ben van Staveren) #4

Haha okay. yeah, that would also do it :smiley: Glad to hear it all worked out in the end :slight_smile: