GROK pattern for timestamp not working in pipeline

Graylog Central (peer support)
1

1. Describe your incident:
Hello,

I get lots of gl2_processing_error:

gl2_processing_error

Replaced invalid timestamp value in message <b3833781-8331-11ec-ab86-0050568779b2> with current time - Value <2022-02-01T08:36:41.609+01:00> caused exception: Invalid format: "2022-02-01T08:36:41.609+01:00" is malformed at "T08:36:41.609+01:00".

The problem seems to be a GROK pattern which does not match:
%{TIMESTAMP_ISO8601:timestamp}%{SPACE:UNWANTED}%{LOGLEVEL:loglevel;string}%{SPACE:UNWANTED}\[%{NOTSPACE:puppetid}\] \[%{NOTSPACE:puppetprocess}\] %{GREEDYDATA:puppetmessage}

It looks to me, that the GROK pattern TIMESTAMP_ISO8601 throws the error.
But this should match, the date stamp works on a GROK online Pattern test like this
https://grokconstructor.appspot.com/do/match#result
The (default) GROK pattern TIMESTAMP_IS8601 should be able to digest the given date:
2022-02-01T08:36:09.584+01:00

Either I’m getting something totally wrong, or there is a bug in the GROK Pattern matcher.

I run Graylog 4.2.5.

2. Describe your environment:
SLES 12 SP4
Graylog 4.2.5

2

Hello && Welcome

Need at ask a couple questions.

  • What version of Elasticsearch do you have?
  • What type of input are you using?
  • The type of logs sent from the remote device are they compatible with the input?

To be honest I haven’t had a problem with the default GROK patterns, but I normally don’t use them to change the timestamp. I either use a pipeline or regex.

3

Hello,
the ElasticSearch version is 7.10.2
The input type is Beats, that is Filebeats.
The type of logs is compatible. We send most logs with filebeats and this works fine.
The strange thing here is, that the GROK Pattern should match, but does not.

Thanks for your help.

Dietmar Schurr

4

Hello

Curious about this…

Are you using some type of extractor/pipeline?

5

Hello,

yes, this is part of a pipeline:

rule "Puppetserver"
when
	has_field("filebeat_fields_puppet")
then
	let matches = grok("%{TIMESTAMP_ISO8601:timestamp}%{SPACE:UNWANTED}%{LOGLEVEL:loglevel;string}%{SPACE:UNWANTED}\\[%{NOTSPACE:puppetid}\\] \\[%{NOTSPACE:puppetprocess}\\] %{GREEDYDATA:puppetmessage}", to_string($message.message), true);
	set_fields(matches);
end

This given pipeline rule throws an gl2_processing_error when applied on this message:
2022-02-03T07:21:45.124+01:00 WARN [qtp1934661193-27919434] [puppetserver] Puppet /etc/puppetlabs/code/environments/non_prod/gkvi_modules/sap_hostagent/data/common.yaml: file does not contain a valid yaml hash

The error is:

gl2_processing_error

Replaced invalid timestamp value in message <906754f1-84b9-11ec-a3de-005056877d52> with current time - Value <2022-02-03T07:21:45.124+01:00> caused exception: Invalid format: "2022-02-03T07:21:45.124+01:00" is malformed at "T07:21:45.124+01:00".

To me, the date format seems ok, that means it should match the given GROK Pattern of TIMESTAMP_ISO8601.
But, still I get this error.

May be I am overlooking something, or there might be a bug with the GROK pattern.

Thanks in advance for your help.

Regards,
Dietmar Schurr

6

I think the issue is that the GROK pattern for TIMESTAMP_ISO8601 has a space char after the T.

%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?

your message doesn’t include that space. I would copy the GROK Pattern and save it as a new one without the space char and then modify your pipeline to include that GROK.

TIMESTAMP_ISO8601_NO_SPACE
%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?

1 Like
7

Hello cawfehman,

hmm, but i thought that [T ] means T or space, one of the two characters is expected.

So, to my understanding this should work.

And, in an online GROK pattern testing tool like this one Test grok patterns
it does work.

That is why I think there might be a bug in Graylog.

Regards,
Dietmar Schurr

8

Hello,

I managed to solve this problem by replacing
%{TIMESTAMP_ISO8601}
with
%{YEAR}-%{MONTHNUM}-%{MONTHDAY}T%{HOUR}:%{MINUTE}:%{SECOND}%{ISO8601_TIMEZONE}

in the GROK pattern.

I still have no idea why the TIMESTAMP_ISO8601 leads to gl2_processing_errors.

Regards,

Dietmar Schurr

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.