Pipeline rule breaks feed

Graylog Central (peer support)
1

1. Describe your incident:
pipeline rule:

rule “iptoname”
when
has_field(“src_ip”)
then
let update_source = lookup_value(“ip_device_name”, to_string($message.src_ip));
set_field(“devicename”, update_source);
end

using the rules simulator, this rule appears to work, for given ip it provides name from lookuptable…i also verified that im get hits on my look up cache.

when i apply the rule to pipeline, messages that should be processed by the rule appear to disappear…oddly when i disable/remove the stage the rule in on appears the messages backfill, so it seems like some kind of a issue with adding the field maybe…

2. Describe your environment:

  • OS Information: linux

  • Package Version: 5.4.0-170 (docker)

2

determined that this:

rule “iptoname”
when
has_field(“src_ip”)
and
contains(to_string($message.src_ip), “192.168”)
then
let update_source = lookup(“ip_device_name”, $message.src_ip);
set_field(“devicename”, update_source);
end

changing the “lookup_value” to “lookup” seems to work however the output is ugly:

devicename
{“value”:“kevinwindows10”}

how can that be cleaned up to just show just the value kevinwindows10

3

found this post , https://community.graylog.org/t/using-a-csv-lookup-table-within-a-pipeline/3707/13 , unsure why this is but, remove the rule from stage, went in maintenance on index, rotated then the reapplied original rule:

rule “iptoname”
when
has_field(“src_ip”)
then
let update_source = lookup_value(“ip_device_name”, to_string($message.src_ip));
set_field(“devicename”, update_source);
end
now its showing just the value:
devicename
homeassistant

4

When rotating the index fixes a problem it is normally a field type mismatch (sending the wrong kind of data to a field) because the auto detection of what the field type should be is guessed using the first message to the new index.

If some value is a string it’s always good practice to force it using to_string just in case when you do set_field.

Lookup looks up multiple values at once, and do that’s why you are seeing key value pairs being returned rather than just the value. So you could have used update source.value in your set_field whatever is after the dot needs to match whichever key you want to access.

If you are on 5.2 you can also manually set the field types now by clicking the field name in the search view.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.