Timestamps mismatch

Graylog Central (peer support)
1

Hello,

I have checked several topics in this community page how to fix my issue with different timezones in the input devices but I couldn’t manage to have a working solution.

I am receiving the messages from Cisco ISE like that using tcpdump:

Msg: Apr 17 15:36:09 SF1-ISE01 CISE_RADIUS_Accounting 0000222563 1 0 2018-04-17 15:36:09.165 +03:00 0002486358 3001 NOTICE Radius-Accounting

The Graylog Timestamp is going to be 18:36:09 and I will receive this message with 3 hours delay in the GUI due to EET timezone set on the ISE and that’s not what I want. Changing the timezone in ISE and all other devices which we plan to integrate with Graylog is not an option. We managed to fix it as we changed the system timezone but that bugged another device with was already sending in UTC…

I have tried with an Input extractor without success:

{
  "title": "Timezone_change",
  "extractor_type": "copy_input",
  "converters": [
    {
      "type": "date",
      "config": {
        "time_zone": "Etc/UTC",
        "date_format": "yyyy-MM-dd HH:mm:ss.SSS"
      }
    }

I have tried with a pipeline rule but honestly, I am far from programming and I have no idea how to handle it :roll_eyes:

I couldn’t find a way to just leave the timestamp similar with the time when the syslog message has been received.

Thanks in advance :slight_smile:

2

The date converter only works if the date string is the only content in the specified field, so you’ll have to run a Regex extractor or Grok extractor first, which isolates the date string you want to parse.

3

I managed to make a Grok extractor with:
%{TIMESTAMP_ISO8601:Timestamp}
and it is working correctly with the source format but it’s not able to re-write the pre-defined Timestamp field with the correct time. On the other hand, the date converter doesn’t work with Grok extractors:
extractor

The issue is that Graylog is not processing the messages instantly as it thinks them as set in the future with 3 hours… that’s what I want to fix.

4

You can convert recognized Grok patterns in-place.

See http://docs.graylog.org/en/2.4/pages/extractors.html#using-grok-patterns-to-extract-data (pretty much at the bottom) for details.

5

Thanks, the extraction is working fine but it’s not re-writing the original field for Timestamp and the messages are still shown with 3 hours delay (2nd is the correct date/time):

messages

6

What did you do exactly?
Also, did you provide the correct timezone for the timestamp? Otherwise Graylog assumes it’s UTC.

7

@jochen, I have already explained that the problem is not with extraction but with the time of processing the message after 3 hours due to different timezone of the input device.
I need to somehow make the server process messages with a timestamp of the time when they were received whether or not they have stayed in the journal.

8

@babameca3575 I’m not sitting in front of your machine and I don’t have any insight in your Graylog cluster and its configuration.

If you don’t provide the requested information, we’re unable to help you.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.