I’m having an issue with one of my graylog servers not showing incoming data within the search results.
I can see that there are incoming messages, but it almost like the messages are not being processed.
I tried performing es queries directly and I have the same results using direct queries as I do making queries from within the GUI.
Is there a way to tail the RAW data coming into the graylog server?
Please elaborate on your Graylog setup and provide more details about your problem (e. g. what kind of messages you’re ingesting, the configuration of the inputs, what messages you would expect to find).
No, that’s not possible as not all information is “text” but might be encoded in some binary format.
Right now we are just ingesting Syslog TCP messages.
I’ve configured a single Syslog TCP input listening on port 1514.
I forwarding data using rsyslog on ~ 800 clients.
The above setup is working fine for my other graylog instances…
When I forward data to the other instances the log messages show up almost immediately when I search for them in the GUI.
Rsyslog is forwarding things like /var/log/{messages,secure,cron,mail} so no binary log are being forwarded.
I have a script that deploys graylog and have deployed ~ 15 other instances with it. They all seem functioning normally. I’m deploying the latest version of graylog in combination with the latest mongodb and latest es 5.x version. Even with 800 clients I’m not generating much data so having a single node instance at each site seems to be working fine except for this one instance.
I’m confused and was looking or an advanced debug tool because I can see messages going in/out but the data within the search does not see to be updating.
I think I know whats happening now. The server that I’m seeing issues on is within India.
I have a few machines in Australia so I didn’t want to setup a graylog server just for Australia so I’m forwarding logs from those few machines to the server in India.
In the GUI the timezone for the admin user is set to UTC, the server itself is set to use IST as it’s timezone but graylog is converting time stamps to UTC. However the data being sent from Australia has time stamps that are ~ 4 hours ahead of the server time of the India graylog server.
When I go to search for the data being sent from Australia (say data in the last 5 mins) the India graylog servers fails to show the data sent from Australia because of the time difference.
Is there anyway to deal with this from within the GUI? What would you recommend I do?
This has the potential to be a pretty big issue, because I’m forwarding all data from ~ 20 graylog servers around the world to a server on the East Coast in the US which will process the collected data and send it to a Splunk instance. This central server will not work well if it can’t process data correctly from different timezones.
The ingested messages ideally have to contain a timezone information in their timestamps.
If you’re using rsyslog or syslog-ng, take a look at graylog-guide-syslog-linux/README.md at master · Graylog2/graylog-guide-syslog-linux · GitHub for information how to configure them correctly.
Other than that, you’ll have to fix the timestamps using a pipeline rule: http://docs.graylog.org/en/2.3/pages/pipelines.html
Thanks, adding RSYSLOG_SyslogProtocol23Format to rsyslog.conf resolved all my issues. Thanks for pointing me to that github page 
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
