Graylog 4.0.5 - GeoIP does not work

[Tomcat7 (Nursultan)]

Hello.
I am trying to set up GeoIP, but am a little confused about Pipelines.
I have a firewall and collect logs using Graylog. The logs are very different. I wanted to display on the world map the blocked IP addresses (users, remote users) that were blocked when trying to get illegal access to the firewall.
Steps taken:

1. Downloaded the GeoLite2-City.mmdb database;
2. Copied to the /etc/graylog/server/;
3. Configured Data Adapters
   System → Lookup Tables → Data Adapters button → Create data adapter:
   Data adapter type → Geo IP - MaxMindTM Databases

Title: GeoIP
Description: GeoIP Lookup Table
Name: geoip
File Path: /etc/graylog/server/GeoLite2-City.mmdb
Database type: City database

4. Configured Caches
   System → Lookup Tables → Caches button → Create cache button

Cache Type: Node-local, in-memory cache
Title: GeoIP
Description: GeoIP Cache
Name: geoip

5. Configured Lookup Tables
   System → Lookup Tables → Lookup Tables (active by default) → Create Lookup Table

Title: GeoIP
Description: GeoIP Lookup
Name: geoip
Data Adapter: GeoIP (geoip)
Cache: GeoIP (geoip)

6. Setting up the Pipeline
   System → Pipelines → Manage rules button → Create Rule button

Description: Incoming connections

Rule source:
rule “GeoIP lookup: remote_addr”
when
has_field (“remote_addr”)
then
let geo = lookup (“geoip”, to_string ($ message.remote_addr));
set_field (“remote_addr_geo_location”, geo [“coordinates”]);
set_field (“remote_addr_geo_country”, geo [“country”]. iso_code);
set_field (“remote_addr_geo_city”, geo [“city”]. names.en);
end

Now the pipeline:

System → Pipelines → button Manage pipelines → button Add new pipeline

Title: GeoIP
Description: Incoming connections

Click the Edit connections button, connect:
All messages

And also in Stage 0, click Edit and add a rule to it:
GeoIP lookup: remote_addr

I cannot find further in the field: ip, geo_ip.

The Message Processors Configuration settings were corrected:
1 AWS Instance Name Lookup
2 GeoIP Resolver
3 Message Filter Chain
4 Pipeline Processor

It seems to me that something needs to be added to the stream. Stuck on the sixth point and in the stream settings.

Thank You for attention.
Regards, Tomcat7.

[shoothub]

1. First check if lookup table works by enter internet ip address
2. Check if your field remote_ip contains correct internet ip address
3. Try to use pipeline function debug() and check graylog logs file
   concat("remote ip: ", to_string($message.remote_addr));
   and then
   sudo tail -f /var/log/graylog-server/server.log

[gsmith]

@Tomcat7
Hello,
Just chiming in, I have successfully start and configured GeoIP from this documention below.

How to Set Up Graylog GeoIP Configuration | Graylog

I’m not sure if you followed those instruction but I noticed you had some different configurations.
Your pipeline

rule “GeoIP lookup: remote_addr”
when
      has_field (“remote_addr”)
then
      let geo = lookup (“geoip”, to_string ($ message.remote_addr));
      set_field (“remote_addr_geo_location”, geo [“coordinates”]);
      set_field (“remote_addr_geo_country”, geo [“country”]. iso_code);
      set_field (“remote_addr_geo_city”, geo [“city”]. names.en);

end

Here is a visual on my configuration.

Hope that helps.

[Tomcat7 (Nursultan)]

Hello Shoothub.

1. I tried to check lookup table, i can’t find. I read documentation, but I don’t understand nothing.
2. I read documentation, but I don’t understand nothing.
3. In the logs, messages only about the absence of a license for the database.

I will re-read the documentation, I seem to have done it according to the instructions.

[Tomcat7 (Nursultan)]

Hello Gsmith

Corrected Pipeline rule and Message Processors Configuration, but the field with the name srcip_geolocation does not appear and cities do not appear on the map.
The firewall blocks users and adds the prefix “Blacklisted” to the log, and then everything is sent to the Graylog.
It seems to me that we need to create a stream and a rule for it. So that he extracts the public IP address from the message and then searches the database and displays it on the map. While reading the documentation and looking at examples of creating rules for a stream.

[shoothub]

You probably don’t undestand how geoip with pipeline rule works. You need to extract IP address in separate field before using pipeline rule and GeoIP. So first check how to extract this IP using either extractor (GROK, regex) or pipeline rule to separate field. Then use this field as input in pipeline rule.

https://docs.graylog.org/en/4.0/pages/extractors.html

[Tomcat7 (Nursultan)]

@shoothub

You probably don’t undestand how geoip with pipeline rule works. You need to extract IP address in separate field before using pipeline rule and GeoIP. So first check how to extract this IP using either extractor (GROK, regex) or pipeline rule to separate field. Then use this field as input in pipeline rule.

Hello
Ok, I will try to do that. As soon as it turns out, I will definitely write how it turned out to be done. I think it will help other people.

[Tomcat7 (Nursultan)]

Hello
It turned out to be done.

1. Created Index (Indices) for firewall;

2. Created Stream for firewall;

3. Created Grok Pattern for input firewall;
   %{IPV4: src_ip}

4. Created Pipeline and rules;
   rule “GeoIP lookup: src_ip”
   when
   has_field(“src_ip”)
   then
   let geo = lookup(“geoip”, to_string($message.src_ip));
   set_field(“src_ip_geo_location”, geo[“coordinates”]);
   set_field(“src_ip_geo_country”, geo[“country”].iso_code);
   set_field(“src_ip_geo_city”, geo[“city”].names.en);
   end

5. Message Processors Configuration
   |1| Message Filter Chain
   |2| Pipeline Processor
   |3| GeoIP Resolver
   |4| AWS Instance Name Lookup