# Pipeline for creating a GeoIP dashboard

**URL:** <https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205>\
**Category:** Graylog Central (peer support)\
**Tags:** pipeline-rules, dashboards\
**Created:** [August 9, 2022, 1:01pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205 "2022-08-09T13:01:45Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![tco](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@tco](https://community.graylog.org/u/tco)\
**Post date:** [August 9, 2022, 1:01pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/1 "2022-08-09T13:01:46Z")

</div>

Hello to all,

I would like to create a worldmap dashboard to visualize where the user connections come from on our stormshield firewall

I followed this procedure : [How to Set Up Graylog GeoIP Configuration | Graylog](https://www.graylog.org/post/how-to-set-up-graylog-geoip-configuration)  
But I’m not really comfortable with pipelines despite the documentation I have trouble to understand and make it work

Here is the technical problem:  
 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/3/3f10d51efd8d011d581174f7bd6ab5ea54c7bbcf.png)

I would like to extract the ip address from the src=x.x.x.x message field to put it in a new field to create my dashboard on this field

already is it a pipeline rule, an extrator that I need?  
can someone explain the profound difference? i don’t understand

This is my rules pipeline for now… i found it on this forum but i don’t know how use it…

rule “function ExtractIPv4Pairs”  
when  
regex(“._^0-9a-zA-Z[^0-9a-zA-Z]._^0-9a-zA-Z[^0-9a-zA-Z]._“,to\_string($message.message)).matches  
then  
let pair = regex(”._^0-9a-zA-Z[^0-9a-zA-Z]._^0-9a-zA-Z[^0-9a-zA-Z]._”,to\_string($message.message),[“src”,“dst”]);  
set\_field(“IP Source”,to\_ip(pair.src));  
end

Thx all …

---

<div class="post-metadata">

**Author:** ![gmorin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gmorin/32/12744_2.png) [@gmorin](https://community.graylog.org/u/gmorin)\
**Post date:** [August 9, 2022, 2:50pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/2 "2022-08-09T14:50:07Z")

</div>

Hi @tco,

From what I see, you have at least two IPs in your messages.

You are using regex, but you can also use GROK patterns, there are preconfigured GROK for IPv4 & IPv6.

I’ve done a similar thing that you’re trying to achieve, and here is the element I can provide you, hoping that it will help.

In my case, I used the %{IP} GROK extractor on the input of the message to create two seperate fields :

```auto
(something here) %{IP:Source_IP} (something here) %{IP:Destination_IP} (something here)

```

Then, I use the pipeline provided in the Graylog documentation on the “Source\_IP” field in order to get the map working.

Maybe if you post the entire log (with the addresses anonymized) in an editable format (like using the block code here) I might be able to help you more ! 🙂

---

<div class="post-metadata">

**Author:** ![tco](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@tco](https://community.graylog.org/u/tco)\
**Post date:** [August 9, 2022, 3:02pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/3 "2022-08-09T15:02:12Z")

</div>

thank you for your quick answer

I have to create an extractor using the grok pattern ip?  
how can i create my extractor, on the input that receives the logs from my firewall?

do I have to import an extractor?

sorry for these questions, i have never created one and i am not very comfortable with the subject

---

<div class="post-metadata">

**Author:** ![gmorin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gmorin/32/12744_2.png) [@gmorin](https://community.graylog.org/u/gmorin)\
**Post date:** [August 9, 2022, 3:06pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/4 "2022-08-09T15:06:25Z")

</div>

You’re welcome @tco.

Well, that’s quite easy to create the extractor. You go on the message, and on the field you wanna extract you click on the little arrow pointing down, create extractor. Then, select “Grok pattern” for the type and submit.

On the page that opens, you can write your extractor and try it against the message you’ve selected to see if it works or not ! 🙂

Again, feel free to ask if you have difficulties, but again, it’lle be hard to help you without the entire message that you’re working on ^^

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [August 9, 2022, 10:27pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/5 "2022-08-09T22:27:37Z")

</div>

Hello @tco

if you conformable with making a regex extractor, I have done this all ready with this…

```auto
srcip=+?((?:\d+\.){3}\d+).+

```

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/e/ef3475fb5574bd20af8ac2a5a5a8c6fcbc038044.png)

That matches my GeoIP Pipeline

```auto
rule "GeoIP lookup: srcip"

when

  has_field("srcip")

then

let geo = lookup("geoip", to_string($message.srcip));

set_field("src_ip_geo_location", geo["coordinates"]);

set_field("src_ip_geo_country", geo["country"].iso_code);

set_field("src_ip_geo_city", geo["city"].names.en);

end

```

HowTo [here](https://docs.graylog.org/docs/extractors#:~:text=This%20is%20how%20you%20can,value%20to%20the%20same%20field.)

---

<div class="post-metadata">

**Author:** ![tco](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@tco](https://community.graylog.org/u/tco)\
**Post date:** [August 10, 2022, 12:17pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/6 "2022-08-10T12:17:26Z")

</div>

Thanks to both of you for your answer,

I didn’t manage to use the grok pattern… but the regular expression worked. I have a new IPSource field that appeared, thanks

I created the pipeline rule by adapting it:

rule “GeoIP lookup: IPSource”

when

has\_field(“IPSource”)

then

let geo = lookup(“geoip”, to\_string($message.IPSource));

set\_field(“src\_ip\_geo\_location”, geo[“coordinates”]);

set\_field(“src\_ip\_geo\_country”, geo[“country”].iso\_code);

set\_field(“src\_ip\_geo\_city”, geo[“city”].names.en);

end

Is there any way to verify that this works?  
Also, I’m French, does changing the “names.en” to “[names.fr](http://names.fr)” in my rule work?

I also have my lookup\_tables with my GeoLite2-City.mmdb database to set the location of source ip addresses

---

<div class="post-metadata">

**Author:** ![gmorin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gmorin/32/12744_2.png) [@gmorin](https://community.graylog.org/u/gmorin)\
**Post date:** [August 10, 2022, 1:12pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/7 "2022-08-10T13:12:15Z")

</div>

Hi @tco,

> [@tco](#):
>
> I didn’t manage to use the grok pattern… but the regular expression worked. I have a new IPSource field that appeared, thanks

Great if it works ! 😛

> [@tco](#):
>
> Is there any way to verify that this works?

I think that there might be a cleaner solution than just test it against a real stream, but that’s how I personnaly proceed when I write a new pipeline.

> [@tco](#):
>
> Also, I’m French, does changing the “names.en” to “[names.fr](http://names.fr)” in my rule work?

French too, the pipeline I use also have the “names.en”… I didn’t try changing it by “[names.fr](http://names.fr)” because it works flawlessly for me.

---

<div class="post-metadata">

**Author:** ![tco](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@tco](https://community.graylog.org/u/tco)\
**Post date:** [August 10, 2022, 4:48pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/8 "2022-08-10T16:48:14Z")

</div>

Sorry I can’t get the geoip feature to work  
To summarize: I created my lookup table and it works when I do the geolocation tests:

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/4/4e39203b9f5ad83332fd083b295e0598772ebf9f.png)

I created my pipeline rule : (the name of my lookup\_table is “geoip”)

```auto
//
rule "geoip"

when

  has_field("IPSource")

then

let geo = lookup("geoip", to_string($message.IPSource));
set_field("ipsrc_location", geo["coordinates"]);
set_field("ipsrc_country_name", geo["country"].names.en);

end
//

```

I created my pipeline then

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/b/b9a3c3e0be65ba61464453a25af28bd9b1954e80.png)

The problem :  
When I generate a log on my firewall, I don’t have the “ipsrc\_location” and “ipsrc\_country\_name” fields that my pipeline should create, I have the IPSource field that appears

Log :

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/f/f5ba92511e8e513ff105292d3383b46a66ea8a16.png)

Thank’s all 🙂

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [August 10, 2022, 6:41pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/9 "2022-08-10T18:41:49Z")

</div>

You can use the debug() function in your pipeline rule to see what is going on (see below for example) Also, check to see if your Message Filter Chain is set BEFORE your pipeline processor (See further below) in system/configurations…

```auto
rule "geoip"

when

  has_field("IPSource")

then
    // tail -f /var/log/graylog-server/server.log to see results of debug message
    debug(concat("=========== IPSource is: ", to_string($message.IPSource)));
    
    let geo = lookup("geoip", to_string($message.IPSource));
    set_field("ipsrc_location", geo["coordinates"]);
    set_field("ipsrc_country_name", geo["country"].names.en);

end

```

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/0/0ef7d871905876532944535df144b88617f835f3.png)

---

<div class="post-metadata">

**Author:** ![tco](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@tco](https://community.graylog.org/u/tco)\
**Post date:** [August 11, 2022, 8:36am UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/10 "2022-08-11T08:36:26Z")

</div>

THANK YOU IT WORKS!  
I understand the pipeline system better now… thanks a lot

Last thing, I would like to make a data table with the identification information  
I would have to create a “user” extraction and put it in a new field  
I understand the principle of extractor now but I don’t understand JSON languages, regex, etc…  
Is there an easier way to extract the user field into a new field?

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/a/a6bc68b9282ab85359cc1be25e8336cd778e2f0e.png)

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [August 11, 2022, 9:41pm UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/11 "2022-08-11T21:41:24Z")

</div>

Hello,

REGEX extractor should work.

```auto
user="(.*?)\"

```

---

<div class="post-metadata">

**Author:** ![tco](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@tco](https://community.graylog.org/u/tco)\
**Post date:** [August 12, 2022, 7:14am UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/12 "2022-08-12T07:14:38Z")

</div>

thank you very much!!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [August 26, 2022, 7:15am UTC](https://community.graylog.org/t/pipeline-for-creating-a-geoip-dashboard/25205/13 "2022-08-26T07:15:04Z")

</div>

Closed on August 26, 2022, 7:15am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.