Search over Web Interface not delivering messages within 2 hours

[Vic]

I have installed the OVA 3.0.2-1 and have also configured 3 Inputs for sending logs from 3 different wildfly instances using the GELF-Logger from here: https://logging.paluch.biz/examples/wildfly.html

Logs are coming into the Graylog server but when I tried to configure my first extractor and clicked on “Load message” in order to load the most recent message received within the last hour (which is the only way to create the first extractor apart from importing one), I got the message: “Error. Input did not return a recent message”

When I go to the input and click on “Show received messages” I can see that there are many messages within the last hour, being the last one 5 minutes ago.

If I go to the Search-Tab and search for message within 1 or 2 hours no messages are shown. Only when I select the 8-hour-search I see all the messages (showing first the last one 5 minutes ago).

If I use the absolute search with a period of time within the last hour, messages are shown.

So I suppose this is a problem related to the timezone settings of the graylog server? I checked under System/Overview and I can see:

User admin: 2019-07-09 13:34:16 +02:00
Your web browser: 2019-07-09 13:34:16 +02:00
Graylog server: 2019-07-09 13:34:16 +02:00

They are all showing the right time with the right timezone.

Under server.config the root_timezone is set to CET (I updated this)
The VM of graylog has the folllowing timezone: CET (also updated by myself)

The original System/Overview was:

User admin: 2019-07-09 11:34:16 +00:00
Your web browser: 2019-07-09 13:34:16 +02:00
Graylog server: 2019-07-09 11:34:16 +00:00

When the messages received were showing the wrong timestamp.

Any ideas? Thanks in advance!

[jan (Jan Doberstein)]

as written in the configuration file:

• root_timezone is the timezone of the user root_username not the timestamp that is used in Graylog. (most time the timezone for the user admin, if not renamed from default)

So is the actually the timestamp of the message itself correct? Or is that 2 hours off?

[Vic]

Hi Jan, thanks for your quick reply to begin with!

This is the steps I made:

1. Created my first inputs
2. Realized that messages coming through these inputs had a timestamp - 2 hours
3. Checked “System/Overview”.
4. Realized only web browser time was correct. User admin and Graylog server time was showing - 2 hours
5. Changed root_timezone from UTC to CET
6. Checked messages again and timestamp was now being shown correctly
7. Tried relative search from “last 5 minutes” till “last 2 hours” and message were not being shown (there were message for sure)
8. Tried absolute search selecting a time frame for the last 5 minutes and messages were shown.
9. Checked “System/Overview”.
10. Graylog server was showing - 2 hours
11. Changed timezone of VM from UTC to CET
12. Checked “System/Overview”.
13. All components were now showing the same timezone (same timestamp and same +2:00)

[jan (Jan Doberstein)]

you never mentioned what you ingest … if your messages do not contain any timezone information syslog input in graylog will assume it is utc … if that is not the case you need to adjust this with a processing pipeline.

[Vic]

Messages from the wildfly instances are being sent using the GELF-Logger from https://logging.paluch.biz/examples/wildfly.html

Version is set to “1.1”
Timestamp ist set to “yy-MM-dd HH:mm:ss,SSS ZZZZ”

Messages are given a timezone by the log handler in wildfly and sent via udp to graylog

This is a screenshot of the messages received with the default Timestamp field and the fields Time and timestamp rendered in my timezone:

If I make a search for the last 15 minutes no messages are found.

If I make an absolute search for the last 15 minutes, messages are found.

Problem is ONLY related to the relative search including the “recent message searchs” of the stream rules and the extractos for the inputs

[Vic]

Hi Jan, any feedback?

[jan (Jan Doberstein)]

nope - no idea. Might be a bug.