Still having timestamp issues

Graylog Central (peer support)
1

I still have issues with timestamps. I don’t understand why it gives me +2. I just want the original timestamp to show and the system should not modify anything.
I have created a (working) pipeline because of previous topic i created. This is the result.

image

But i still see:
image

Why would the system show me my timezone timestamp? That is so not logical…
I want to see the timestamp of the message. Not the timestamp of the message modified to my current timezone.
Rule on my pipeline:

rule "fortigate timestamp"
when
has_field("devname") && has_field("date") && has_field("time")
then
let build_message_0 = concat(to_string($message.date), " ");
let build_message_1 = concat(build_message_0, to_string($message.time));
let new_timestamp = parse_date(value:to_string(build_message_1), pattern:"yyyy-MM-dd HH:mm:sss", timezone:"Europ/Amstedam");
set_field("timestamp", new_timestamp);
set_field("Timestamp", new_timestamp);
set_field("test_timestamp", new_timestamp);
End

Maybe i need to wait for 2 hours for it to catch up but changing to absolute time and going 2 hours or more in the future still shows wrong messages.
According to my old topic, this should have been fixed in 3.0

But im on Graylog 3.0.2+1686930
-edit-
Since the devices in question are already in TZ Europe/Amsterdam, should i change the new timestamp to UTC first? And then i get correct timestamps?.. thats not logical either but…

2

Have been trying with a new rule to use the eventtime field (epoch time). It does convert it but i cant change the timestamp field.

rule "Epoch Convert"
when
has_field("devname") && has_field("date") && has_field("time") && has_field("eventtime")
then
let epoch = parse_date("1970-01-01T00:00:00.000Z", "yyyy-MM-dd'T'HH:mm:ss.SSSZ");
let ts_seconds = seconds(to_long($message.eventtime));
set_field("epoch_timestamp", epoch + ts_seconds);
set_field("timestamp", epoch + ts_seconds);
End

image
image.png1124×68 7.19 KB

3

I think this is my solution (created new Indice to put the messages in). Also read somehwere that using RAW TCP input might help, that wasn’t the case. Using SYSTLOG TCP port 30000.

  1. Created a stream for my fortigates (SYSLOG TCP 30000)
  2. Created a pipeline for this stream
  3. Set “Continue processing on next stage when” to any
  4. Use the rule Epoch Convert
    rule “Epoch Convert”
    when
    has_field(“devname”) && has_field(“date”) && has_field(“time”) && has_field(“eventtime”)
    then
    let epoch = parse_date(“1970-01-01 00:00:00.000Z”, “yyyy-MM-dd’ 'HH:mm:ss.SSSZ”);
    let ts_seconds = seconds(to_long($message.eventtime));
    set_field(“epoch_timestamp”, epoch + ts_seconds);
    set_field(“timestamp”, epoch + ts_seconds);
    End

Now my messages show up correct for my timezonze. FortiGates are configured for Europe/Amsterdam, so is my timezone:

image
image.png1198×207 19.4 KB

4

You should always store the timestamps as UTC, for all of your systems regardless of the local timezone - This gives you consistent data.

The console will then show you the timestamp in your local zone, based on your profile preferences - Click on your name, then Edit Profile then edit your timezone.

image

What you can see from mine is the @timestamp (which is what is actually written to the index) is UTC - But the “timestamp” column is calculated, based on my local timezone (+10) - If i wanted to see/search on that in +2, UTC or any other timezone, I just need to edit my profile.

That way, when you search for “what happened in the last 8 hours globally”, you will see results from all your devices based on the true last 8 hours, regardless of what timezone they are in.

3 Likes
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.