Port Definitions Pipeline

[alphafilter]

I apologize if this has been covered I could not find a specific example based on my google searches. What we are trying to do is insert a field into messages that uses a lookup table to define the port name (i.e 443=https) so that this information can be displayed in a dashboard. This seems like something simple and maybe we are completely going about this the wrong way.

We have this working as a decorator in a stream so I know the lookup table data is valid and working.

We are trying to now use this same lookup table in a pipeline to add this field to the message so it can be used in a dashboard. Below is what I have for my pipleline script but it does not seem to be working.

rule "port-name"
when
  has_field("sourceTransportPort")
then
  // Do the lookup and add "Not available" to field remote_addr_whois
  let lookup_result = lookup("port-map", $message.sourceTransportPort, 1);
  set_field("sourceTransportPortName", lookup_result["name"]);
end`Preformatted text`

any guidance would be greatly appreciated. I am sure one we get the hgn of this we can continue on with what we are looking to accomplish.

Thank you

[tmacgbay]

Hard to tell without seeing your table - my first guess is assuming it’s a simple table with a key and a value, lookup_result will contain the value so your set_field() line need only be:

set_field("sourceTransportPortName", lookup_result);

[alphafilter]

tmac, thanks we will give that a shot. As far as the lookup table goes it is referencing a csv file and the contents of that file look like below (with more lines of course)

port,name,comment
1,tcpmux,TCP port service multiplexer

[tmacgbay]

I don’t think Graylog has their tables doing multi-value yet… I know it was an issue in the past but they may have fixed it (haven’t looked it up recently…) there is an ugly workaround if it doesn’t work and you need it.

[alphafilter]

Thanks for the help thus far. I redid the file so it only has 2 values.

port,name
1,tcpmux
5,rje
7,echo
9,discard
11,systat
13,daytime
17,qotd
18,msp
19,chargen
20,ftp-data
21,ftp

etc

when I use the code above now and just have “lookup_result” I stop getting all logs inside the stream

[tmacgbay]

Looking in the docs further, perhaps multi-value works… its not clear but it looks like you just needed a different format… you could do the following with the orig multi-value table:

  set_field("sourceTransportPortName", lookup_result.name);
  set_field("sourceTransportPortCOMMENT", lookup_result.comment);

single values use the lookup_value() function…