Pipeline rule not printing debug extracted fields

[rwarden]

Hi,

I’m trying to create a function in a pipeline using fields that have been extracted from an Extactor via GROK pattern matching. For starters I am just trying to print the fields using pipeline debug but I just get this log line:

2020-01-27T12:58:26.361Z INFO [Function] PIPELINE DEBUG: Passed value is NULL.

My fields are being correctly parsed as expected…

And I have also set the message processor order to use “Message Filter Chain” before the “Pipeline Processor”.

I have the following simple rule set:

rule “debug vars”
when
has_field(“message”)
then
debug($message.ats_timestamp);
end

If I change
debug($message.ats_timestamp);
to
debug($message.source);
then it prints the source correctly.

Any help appreciated.

Thanks

[shoothub]

I usually use this snippet for debug:

let debug_message = concat("ats timestamp: ", to_string($message.ats_timestamp));
debug(debug_message);

[rwarden]

Thanks, but the field $message.ats_timestamp still prints as null using that method

[jan (Jan Doberstein)]

he @rwarden

where did you create the field ats timestamp?

[rwarden]

Hey @jan,

I’m creating the field from a GROK pattern Extractor, here’s my GROK:

(?m)(?<ats.timestamp>^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]*)%{SPACE}%{LOGLEVEL:ats.loglevel}%{SPACE}%{WORD:ats.class}%{SPACE}%{GREEDYDATA:ats.message}

Although it seems to be converting “ats.timestamp” to “ats_timestamp” when displaying the field.
I have of course tried printing “ats.timestamp”, “ats_timestamp” and additionally even changing my GROK pattern to be “atstimestamp” instead to try rule out any special character issues but it still does not print the value expected.

[jan (Jan Doberstein)]

dots are not allowed in field names @rwarden that is the root cause of your problem. When you change that to something else all will work.

[rwarden]

Thanks, yes removing the dot from the field names fixed the issue.

Thanks for your help.