Is it possible to make a stream with GeoIP filter?


#1

Hi,

We’re running the GeoIP Resolver to get the additional fields XXX_country_code etc.
According to the docs, the GeoIP Resolver needs to be last in order to get those additional GeoIP fields. (I noted that if the GeoIP Resolver was first in the list, only the “source” field would get the additional GeoIP data)

Unfortunately, it doesn’t work to create a stream with filter based on the additional GeoIP fields, like:

  • Field ssh_accepted_ip_country_code must not match regular expression DK

as the GeoIP Resolver seems to executed after the stream filtering is done.

Is there any fix/workaround to get that working?


(Jan Doberstein) #2

hej @frpet1

it highly depends on the order of your processing. But you could for example create a processing pipeline that does the geo_ip lookup with the new lookup tables and route that after to a stream.

just as an idea


(Jochen) #3

That’s not correct. The GeoIP Resolver simply has to run after the stage which creates the message fields it’s supposed to process. So if the messages contain the relevant fields (consisting only of an IP address and nothing else), it can be placed as the first filter.

Could you please point me to the documentation where you’ve read that?

Messages are assigned to streams in the “Message Filter Chain” message processor (StreamMatcherFilter). If you want to create a stream rule which requires that field, you have to run the GeoIP Resolver before the Message Filter Chain.


#4

See the text under the picture:
http://docs.graylog.org/en/2.3/pages/geolocation.html#configure-the-message-processor


#5

Right,
the fields I want to do GeoIP lookup upon are extracted (with a extractor) from a “input” source, so the extractor needs to be run before otherwise no custom fields with IPs will be available…

  • Run the Extractor on input source
  • Do GeoIP lookup
  • Stream filtering

Is that possible?


#6

Ok, I’ll read up on how it works…

Summaring the above, I guess what I wanted to do today isn’t possible with the current workflow, as both the extractor(s) filtering & stream filtering occurs in the same “Message Processors Conf” component, namely in the “Message Filter Chain”.

How could the pipeline processor help out ?
sorry, I meant to ask for an pipeline processor example


(Jan Doberstein) #7

Hej @frpet1

I personal would solve all your issues within a processing pipeline. That pipeline would do the following:

  • extract all wanted information from the message
  • use a GEO IP Lookup Table
  • route the message to the wanted stream

that would be the best solution IMHO


(Jochen) #8

For reference:
https://github.com/Graylog2/graylog2-server/pull/4186


#9

Great work! thank you!


(system) #10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.