Source doesn't always show host or IP

[tuaris (Daniel Morante)]

Description of your problem

I have an input " System Log Syslog UDP" on Graylog and a few different kind of appliances (ie. router/firewall) and hosts setup to send the syslog to that input.

    allow_override_date:
     true
    bind_address:
     0.0.0.0
    expand_structured_data:
     false
    force_rdns:
     false
    number_worker_threads:
     4
    override_source:
     <empty>
    port:
     5444
    recv_buffer_size:
     262144
    store_full_message:
     false

The problem is that it’s difficult to tell which host or appliance sent the message. Not all the log entries have the hostname or IP:

Description of steps you’ve taken to attempt to solve the issue

I am unable to find any configuration option in Graylog to show or capture the originating IP or hostname.

Environmental information

Operating system information

• FreeBSD

Package versions

• Graylog: Graylog v4.1.5+01c9198
• MongoDB: mongodb40-4.0.27
• Elasticsearch: elasticsearch6-6.8.16

[tuaris (Daniel Morante)]

Here’s another example. This time the source just shows a “1”

[tmacgbay]

Would be helpful if you can give some full_message examples as well as showing all extracted fields. It may be that you have to change something at the sending syslog host or parse fields to inject a host/IP based on what you find in the message once it’s in Graylog. Worst case you can create a separate input where needed… but that seems inefficient…

[gsmith]

Hello @tuaris

I agree with what @tmacgbay mentioned about creating a separate INPUT. I noticed in our environment some firewalls/routers logs are best suited for Raw/Plaintext UDP INPUTs then with Syslog UDP.

[tuaris (Daniel Morante)]

Here’s a few examples.

NOTE: Since I can’t post more than a single image or 2 links, I will refer to the images by name in here: Index of /downloads/unibia/screenshots/Graylog

This one corresponds to my second screenshot. The hostname of the system is storage. the FQDN is storage.morante.com:

Nov 21 00:02:50 storage 1 2021-11-21T00:02:50.543046-05:00 storage.morante.com mountd 4020 - - mount request from 10.8.8.249 for non existent path /mnt/CDImages
Nov 21 00:02:50 storage 1 2021-11-21T00:02:50.543071-05:00 storage.morante.com mountd 4020 - - mount request denied from 10.8.8.249 for /mnt/CDImages

This one that refers to dhcpd was hard to tell where it originated from:

dhcpd_graylog.png

However based on the message, it’s from this specific firewall appliance with a hostname protostar:

dhcpd_original.png

A good example of one impossible to determine where it from are these syslogd messages that are common to all FreeBSD systems:

syslogd_graylog.png

I know it’s a FreeBSD system log message, here’s what it normally looks like in /var/log/messages. Where saturn is the system’s hostname

Nov 21 03:38:41 saturn syslogd: last message repeated 3 times

Finally we have this kernel message, for which if I didn’t already know which system was having this exact problem problem, I wouldn’t have now where to look.

kernel_graylog.png

Nov 20 23:10:47 europa kernel: mps0: Controller reported scsi ioc terminated tgt 11 SMID 2063 loginfo 3112011a

So it looks like I need to add a field to Graylog to include the source hostname/IP address?

[tmacgbay]

I agree @gsmith - try a raw/plaintext input it may result in an easier time capturing your source.

[gsmith]

Hello,

dhcpd_original.png - FreeBSD systems looks like it is sending multi-line logs. Only thing I can think of to fix that would be a pipeline to group the individual lines together. Since that’s your firewall/router that may require a separate input and/or maybe a pipeline to group the separate lines together. The pipeline would then route all logs to defined stream for this purpose. You could also attach its own index. Just a thought.

Have you tried testing a new raw/plaintext input? This doesn’t mean you need to configure every source just one. Try it out for testing/troubleshooting.