GeoIP Setup - Need help

Graylog Add-ons
1

Hi,

I’m new to Graylog. I use Graylog to receive events that are written to Elasticsearch. Here I now also have entries that are visible from Sysmon on Windows devices with data_win_eventdata_sourceIP and data_win_eventdata_destinationIP.
A stream has already been created.
I also followed these instructions to activate GeoIP: https://graylog.org/post/how-to-set-up-graylog-geoip-configuration/

In the pipeline configuration I created the following rule that points to my stream

rule "GeoIP lookup"
when
  true
then
set_field("data_win_eventdata_sourceIp",$message.src_ip);
set_field("data_win_eventdata_destinationIp",$message.dest_ip);

let geo = lookup("geoip", to_string($message.src_ip));
set_field("src_ip_geo_location", geo["coordinates"]);
set_field("src_ip_geo_country", geo["country"].iso_code);
set_field("src_ip_geo_city", geo["city"].names.en);

let geo2 = lookup("geoip", to_string($message.dest_ip));
set_field("dest_ip_geo_location", geo2["coordinates"]);
set_field("dest_ip_geo_country", geo2["country"].iso_code);
set_field("dest_ip_geo_city", geo2["city"].names.en);
end

The arrangement of the message processor is as follows:

  • AWS Instance Name Lookup
  • Pipeline Processor
  • Stream Rule Processor
  • Message Filter Chain
  • GeoIP resolver

Unfortunately, the additional fields with the geo information are not created for me.

Maybe someone can help me out.

Thanks
David

2

Have you tried manually performing a lookup for one of the IPs on the system/lookup tables page?

3

Manually is working

{
  "single_value": "37.751,-97.822",
  "multi_value": {
    "continent": {
      "code": "NA",
      "geoname_id": 6255149,
      "names": {
        "de": "Nordamerika",
        "ru": "Северная Америка",
        "pt-BR": "América do Norte",
        "ja": "北アメリカ",
        "en": "North America",
        "fr": "Amérique du Nord",
        "zh-CN": "北美洲",
        "es": "Norteamérica"
      }
    },
    "country": {
      "confidence": null,
      "geoname_id": 6252001,
      "is_in_european_union": false,
      "iso_code": "US",
      "names": {
        "de": "USA",
        "ru": "США",
        "pt-BR": "Estados Unidos",
        "ja": "アメリカ合衆国",
        "en": "United States",
        "fr": "États-Unis",
        "zh-CN": "美国",
        "es": "Estados Unidos"
      }
    },
    "traits": {
      "autonomous_system_number": null,
      "autonomous_system_organization": null,
      "connection_type": null,
      "domain": null,
      "ip_address": "8.8.8.8",
      "is_anonymous": false,
      "is_anonymous_proxy": false,
      "is_anonymous_vpn": false,
      "is_hosting_provider": false,
      "is_legitimate_proxy": false,
      "is_public_proxy": false,
      "is_satellite_provider": false,
      "is_tor_exit_node": false,
      "isp": null,
      "organization": null,
      "user_type": null
    },
    "city": {
      "confidence": null,
      "geoname_id": null,
      "names": {}
    },
    "represented_country": {
      "confidence": null,
      "geoname_id": null,
      "is_in_european_union": false,
      "iso_code": null,
      "names": {},
      "type": null
    },
    "coordinates": "37.751,-97.822",
    "location": {
      "accuracy_radius": 1000,
      "average_income": null,
      "latitude": 37.751,
      "longitude": -97.822,
      "metro_code": null,
      "population_density": null,
      "time_zone": "America/Chicago"
    },
    "postal": {
      "code": null,
      "confidence": null
    },
    "registered_country": {
      "confidence": null,
      "geoname_id": 6252001,
      "is_in_european_union": false,
      "iso_code": "US",
      "names": {
        "de": "USA",
        "ru": "США",
        "pt-BR": "Estados Unidos",
        "ja": "アメリカ合衆国",
        "en": "United States",
        "fr": "États-Unis",
        "zh-CN": "美国",
        "es": "Estados Unidos"
      }
    },
    "subdivisions": []
  },
  "string_list_value": null,
  "has_error": false,
  "ttl": 9223372036854776000
}```
4

Here is my configuration

image
image1450×486 73.4 KB

image
image1508×750 51.1 KB

image
image952×818 39.5 KB

image
image1482×654 33.5 KB

image
image1486×1330 114 KB

image
image2958×1258 291 KB

Incoming Message by Stream

message
<134>device_name="SFW" timestamp="2024-01-25T13:45:09+0100" device_model="XXXXXX" device_serial_id="XXXXXXXX" log_id="010101xxxxx1" log_type="Firewall" log_component="Firewall Rule" log_subtype="Allowed" log_version=1 severity="Information" fw_rule_id="x" fw_rule_name="LAN-to-WAN" fw_rule_section="Local rule" nat_rule_id="x" nat_rule_name="XXXXXX IPv4" fw_rule_type="USER" gw_id_request=x gw_name_request="XXXXX-1G" web_policy_id=xx ips_policy_id=xx app_name="DNS" app_risk=1 app_technology="Network Protocol" app_category="Infrastructure" ether_type="Unknown (0x0000)" in_interface="Port1" out_interface="Port6" src_mac="xx:xx:xx:xx:xx:xx" dst_mac="xx:xx:xx:xx:x:xx" src_ip="192.xxx.xxx.xx" src_country="R1" dst_ip="194.xx.x.xxx" dst_country="DEU" protocol="UDP" src_port=5xxx9 dst_port=xx src_trans_ip="62.xxx.xxx.xxx" src_zone_type="LAN" src_zone="LAN" dst_zone_type="WAN" dst_zone="WAN" con_event="Start" con_id="1565570657" hb_status="No Heartbeat" app_resolved_by="Signature" app_is_cloud="FALSE" qualifier="New" in_display_interface="LAN" out_display_interface="IF01-1G" log_occurrence="1"
5

Simulation Results

image
image2956×1312 364 KB

6

Seems to me the message fields are not being parsed. Do you see a field dst_ip in the message table?
The simulation says “not satisfied”, indicating that has_field("dst_ip") is false.

7

I see the dst_ip field. Is it possible that it cannot process this because the fields are separated by a space?

image
image1606×252 105 KB

8

The data is present in the log message. But it is apparently not being parsed out and assigned to a message field.

9

I created a json extractor for the input that breaks down the entire message into individual fields. I also get the field fw_dst_ip. This field contains the value of the dst_ip field of the message. But even if I use this field in my pipeline rule, I get the same reply at the simulation.

image
image1178×548 44.5 KB

10

It’s hard to remotely debug pipeline rules.
Use the debug statement and/or create fields to display intermediate results in your rule to narrow down where things are not working as expected.

11

ERROR [PipelineRuleParser] Unable to retrieve expression for variable fw_dst_ip, this is a bug

12

What is the order of your message processors (system/configuration/message processors) ?

13

image
image2038×500 29.9 KB

14

Please also post your updated pipeline rule again.
Here is a pretty detailed discussion of pitfalls with geoip:

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.