Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question. Don’t forget to select tags to help index your topic!
1. Describe your incident:
After setting up the geolocation processor as in Geolocation, and adding an extractor for the IIS logs to have a field (called source_ip), I still cannot find the geolocation fields to use.
Package Version:
Graylog 4.3.9+e2c6648 on graylog01 (Private Build 17.0.4 on Linux 5.4.0-131-generic)
ii elasticsearch-oss 7.10.2 amd64 Distributed RESTful search engine built for the cloud
ic graylog-4.2-repository 1-4 all Package to install Graylog 4.2 GPG key and repository
ii graylog-4.3-repository 1-6 all Package to install Graylog 4.3 GPG key and repository
ii graylog-enterprise-integrations-plugins 4.3.9-1 all Graylog Enterprise Integrations plugins
ii graylog-enterprise-plugins 4.3.9-1 amd64 Graylog Enterprise plugins
ii graylog-integrations-plugins 4.3.9-1 all Graylog Integrations plugins
ii graylog-server 4.3.9-1 all Graylog server
ii mongodb-org 4.0.28 amd64 MongoDB open source document-oriented database system (metapackage)
ii mongodb-org-mongos 4.0.28 amd64 MongoDB sharded cluster query router
ii mongodb-org-server 4.0.28 amd64 MongoDB database server
ii mongodb-org-shell 4.0.28 amd64 MongoDB shell client
ii mongodb-org-tools 4.0.28 amd64 MongoDB tools
Service logs, configurations, and environment variables:
3. What steps have you already taken to try and solve the problem?
I upgraded from 4.2 to the latest 4.3.9 and followed the instructions at Geolocation
Old topic Geolocation not working tried to cover the same problem but no solution was posted
Older topics use a different way of adding geolocation fields to log entries using lookup tables, but it was not described in the instructions, so I assumed it is no longer valid for v3.5.9
4. How can the community help?
Please share/advise what can be wrong
Thank you for your response. I didn’t create any pipeline or LUT… It was not mentioned in the Geolocation document!!!
In fact, the 1st paragraph on the geolocation using pipeline and LUTs document mentions “Geolocation can be automatically built into the Graylog platform by using the “GeoIP Resolver” plugin with a MaxMind database. However, you can further improve your ability to extract meaningful and useful data by leveraging the functionality of pipelines and lookup tables”, so I was expecting that by setting up only the geolocation processor, it will be able to create the geolocation fields automatically without the need of adding additional pipelines or LUTs…
Hi @gsmith, I just tried that but I still cannot see the geolocation fields. please note that I removed the enterprise add-ons as it was giving many log errors caused by lack of license.
My current Message Processors Configuration is as follows:
@gsmith , As I mentioned earlier, I didn’t create a pipeline or LUTs. As the geolocation document mentioned, I just added the processor plugin information. Also, I added a message extractor to the input to make sure that there will be a valid IP field (I named it source_ip) to be used for geolocation…
I’m familiar with this technique, but I was hoping that I can use the GeoIP addon as is… this document does not depend on the addon, it creates a LUT to be used in a pipeline. I appreciate you sharing your configuration though… I shall disable the Geolocation plugin and start building the LUT and its pipeline…
Its probably for the better. I understand the plugin is easier but later on if you upgrade it may not work. This way if you build it, it should stay working without any problems. My setup went through GL version 2.5 thru 4.3 and its still working good.
rule "GeoIP lookup: srcip"
when
has_field("somefield")
then
let geo = lookup("geoip", to_string($message.srcip));
set_field("src_ip_geo_location", geo["coordinates"]);
set_field("src_ip_geo_country", geo["country"].iso_code);
set_field("src_ip_geo_city", geo["city"].names.en);
debug(geo);
end
Make sure the correct naming convention is the same.
Specially this part, noticed it states "geoip "
let geo = lookup("geoip", to_string($message.srcip))
@er213 this log is for IIS. If I understood your question correctly, your problem is showing the real IP in the IIS log, then you need to add the X-Forwarded-For as a custom field in IIS. This has nothing to do with Graylog
Hi @gsmith … Thank you SO MUCH for your help. It worked now for me. The problem was in the name of the LUT. In all examples it was geoip, but when I created it, I created the table with the name GeoIPtbl
