Client address: 127.0.0.1 and not the real IP

1. Describe your incident:

Remote users’ IP addresses are always shown as 127.0.0.1 in the “Client Address” field on GL v.4.3.3:

and not as the real IP as on GL 3.2.6:

2. Describe your environment:

  • OS Information: Ubuntu 20.04 LTS

  • Package Version:
    Old GL: 3.2.6
    New GL: 4.3.3

  • Service logs, configurations, and environment variables:

All standard config but some passwords

3. What steps have you already taken to try and solve the problem?

Access the old GL and check the client IP → see they match with the real client IP
Access the new GL and check the client IP → see they are shown as 127.0.0.1 all the time

4. How can the community help?

Is there any way to fix this?

Thanks!

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

Hello,

Its probably the way this Graylog node was configured, Unfortunately I don’t have this issue. I’m using GL 4.3.

NOTE: 127.0.0.1 is a real IP address its called a loop-back address for the localhost.

it is quite possible that however you are parsing the incoming messages, it is picking up the wrong field? may looking at the path the message takes through your extractors and/or pipeline? you would need to provide a sample message and some detail on how you are breaking out fields… specifically the loopback and the required IP field and it’s placement in the message.

Also, did you upgrade Elasticsearch or switch to OpenSearch? To what version if you did? Did you upgrade Sidecar?

NOTE: 127.0.0.1 is a real IP address its called a loop-back address for the localhost.

:slight_smile:
I know that.

I also use GL 4.3.3 and haven’t modified/found anything on the config files related to the “Client Address” field…

Sorry if I sound too blonde here but I don’t see the relationship between parsing messages and where users login from… O_o You can see which users are logged in the system without ingesting a single line of logs…

FYI - we use OpenSearch 1.3.3 + GL 4.3.3 from day #1.

Thanks!

no - I wasn’t paying enough attention - I was thinking this was a table you had built from incoming data rather than the Users and Teams page. :crazy_face:

On Graylog 4.3.2 they are reporting correctly for me. I will be upgrading to 4.3.3 next week - will check back if it’s off for me…

Hello,

Out of curiosity , since MongoDb hold metadata have you check there? ( i.e. Logs, database).

EDIT I just noticed this…

How did you go about this upgrade?

Hi,

are you using a proxy like nginx? If so try to configure the proxy_set_header; this is from my nginx.conf:

location / {
proxy_pass http://graylog:9000;

proxy_set_header Host $host; # required for docker client’s sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client’s IP
proxy_set_header X-forwarded-host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}

Additional I configured the trusted_proxies setting in my graylog.conf.

This might explain more: Logging user activity

2 Likes

Hi @gsmith

I didn’t upgrade.
Both GL servers run in parallel :slight_smile: and we’re in the process of configuring the new one to offer the same services as the old one…
Once the new GL-cluster is finished, we’ll sunset the old server…

Cheers

Hi @valhaim

Thanks for your answer.
Following your advice, I updated the

proxy_set_header

stuff as follows:

        location / {
          proxy_set_header Host $http_host;
          proxy_set_header X-Real-IP $remote_addr;  <-- added it
          proxy_set_header X-Forwarded-Host $host;
          proxy_set_header X-Forwarded-Server $host;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header X-Graylog-Server-URL https://$server_name/;
          proxy_set_header X-Forwarded-Proto $scheme;  <-- added it
          proxy_pass       http://127.0.0.1:9000;
        }

and also added:

trusted_proxies = 127.0.0.1/32, ip-of-the-load-balancer/32

to the Graylog config file, then restarted both nginx and Graylog.

Still, no changes and 127.0.0.1 is still shown in the web interface for remote users…

Do I have to change

proxy_pass http://127.0.0.1:9000;

to

proxy_pass http://graylog-hostname:9000; on each node; e.g: “node-1:9000”, “node-2:9000”

or

proxy_pass http://fqdn-for-the-load-balancer:9000;; e.g.: “graylog.domain.tld:9000”

?

Thanks!

Quick update: it seems like the remote IP addresses are slowly being populated; still there are several entries for 127.0.0.1.

I guess it will be all fine after some more time…

Thanks @valhaim for the suggestions :slight_smile:

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.