Lookup table, soruce, gl2_remote_ip


(-) #1

Hi guys,

I have a lookup table include two columns, key == source (this is an IP list, what I can found in gl2_remote_IP), value == host (this is a host name what I would like to see in “source” field). I use this as decorator on gl2_remote_IP now (attached the picture).

k%C3%A9p

So I would like to create a pipeline, for this process, because if I use this list as decorator, I can not search in the value.

I created this, but is not working…I’m very beginner in pipelines, so pls help me in this problem,


rule “lookup”
when true
then
let source = lookup_value(“src_hst”, $message.source); #the lookup table name is src_hst.csv
set_field(“host”, source);
end

Thanks,


(Philipp Ruland) #2

Hey @_gh05st,

you’ve got two issues in your code:

Field name

you have to distinguish between the field source and the field gl2_remote_ip.

The field source is written based on conventions of the input (e.g. the Syslog header in case of the Syslog Input) while the gl2_remote_ip is written by Graylog itself and is always set to the IP where the message came from (more precise: The “Source Address” in the IP-header of the incoming packet.)

You used gl2_remote_ip in your decorator but source in the pipeline. Are these values the same? Or is your source field the hostname instead of the IP of a device (for example)?

The lookup function

Have a look at http://docs.graylog.org/en/2.4/pages/pipelines/functions.html#lookup-value. The function lookup_value(lookup_table: string, key: any, [default: any]) expects the name of a lookup table, not the filename of the file this lookup table is based on. According to your screenshot, IRM_Sources should be the value of the lookup_table parameter.

This code should work:

rule “lookup”
when true
then
  let source = lookup_value(“IRM_Sources”, $message.gl2_remote_ip);
  set_field(“host”, source);
end

Greetings,
Phil


(-) #3

Thanks for your help, it is works :slight_smile:


(-) #4

Hi,

I have another problem, I do a cron job in my server what ping my all source every day and It create a csv file based on result.
Two columns in the output. “source”,“result”. (source==gl2_remote_ip, result==result of the ping)

I created a pipeline base on your idea, but I do not know why, but It is not working.

rule “ping”
when true
then
let source = lookup_value(“IRM_Ping”, $message.gl2_remote_ip);
set_field(“ping”, source);
end

Working fine as decorator, I attached.
k%C3%A9p

Thanks,


(Philipp Ruland) #5

What happens if you use the Pipeline as a decorator via the “Processing Pipeline Decorator”? :slight_smile:

Greetings,
Phil


(-) #6

Thanks, I solved it :slight_smile:


(Jan Doberstein) #7

sharing is caring …

please post your solution that someone else will see what had happened and learn from it.


(-) #8

Solutions:

rule “lookup”
when true
then
let source = lookup_value(“Lookup_table_name”, $message.gl2_remote_ip);
set_field(“source”, source);
end


(system) #9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.