Apache forwarded for address

[the_tree (Steve Farmer)]

Hi all,

We have 4 apache servers behind an nxginx load balancer, all hosts are running RHEL 8

I have set graylog up using sidecar and filebeats with success

I followed the instructions below (we have a maxmind account so I am familiar with their system)

How to Set Up Graylog GeoIP Configuration | Graylog

All good until I got to the end and configured the pipeline rules.

We have no src_ip field and need to extract it from the log messages

Currently we use standard apache log formats

ErrorLog /www/logs/xxx.mydomain.com-error_log
CustomLog /www/logs/xxx.mydomain.com-access_log common

The problem is extracting the forwarded_for address which is embedded in the messages

The options seem to be the filebeat apache module or an extractor or custom apache log format

Which leads me to think that nxlog may be a better option if I need a custom apache log format anyway

Suggestions most welcome

Thanks
Steve

[tmacgbay]

There are a variety of ways you can pull the src_ip from the message you can use extractors on the Input or you can use functions in your pipeline such as key_value() or regex() or grok() … It is possible to do it at the filebeat module or switch to nxlog and do it there but I rarely see configurations doing that in these forums.

[the_tree (Steve Farmer)]

Thanks mate

I have an extractor working, it needs to be refined.

I think i will replace %h with %{X-Forwarded-For}i in the apache log format so i always get the forwarded for IP address

Thanks again