DNS resolution failing for one host

[Fabou78 (Fabrice)]

One of my devices is not showing its hostname in Graylog 3.2.2 but its IP. It is an ASA device that has been configured exactly as all my other cisco ASA that works correctly i.e. showing their hostname rather than IP addresses.

The only difference I can think of is that the device has been configured to send log before the corresponding entry in DNS was added. So for the first few logs coming in for that device the Graylog server was not able to resolve the host.

Now DNS and reverse DNS entries are configured correctly for that host and I have verified that the Graylog server can indeed DNS resolve that host both ways. However the logs front that host still come unresolved

Anyone know a way to force Graylog to resume query DNS for that host? Once a device is known as an IP does it mean that it will always be known as is?

Please note that I only have this problem with this one particular host and I have red the following posts with no clear steering

Source coming in as IP address and not resolved DNS name

here’s some additional information: I ran the following command: to see what files were in use: lsof -p $(pgrep -f syslog) -a +D /var/log COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME rsyslogd 1923 root 1w REG 249,0 6295945 793792 /var/log/secure rsyslogd 1923 root 2w REG 249,0 2735194 793791 /var/log/messages rsyslogd 1923 root 4w REG 249,0 2365 793790 /var/log/maillog rsyslogd 1923 root 8w REG 249,0 244734 793785 /var/log/cron

Resolve IP - DNS resolver

What exactly do you want to achieve? If you’re desperate, you could take a look at the (old and deprecated) DNS resolver plugin.

Enrich log stream with local DNS for fields other than source

Is there a way to enrich log data with local DNS names for a specific field? An example field is orig_h, which has an IP address in it. I would like to be able to grab an internal DNS name if one exists within a pipeline rule-set. Regards, Brandon

Issues with reverse DNS and picky certs

Hi! I’m having issues getting graylog to use DNS to resolve hostnames. The error message is as follows: 2018-08-02T21:15:17.270-04:00 WARN [ProxiedResource] Unable to call https://XXX.XXX.XXX.XXX:9000/api/system/metrics/multiple on node <31861ddf-05a5-434f-8d19-dc00e3c33d9a> javax.net.ssl.SSLPeerUnverifiedException: Hostname XXX.XXX.XXX.XXX not verified: Everything else seems to work fine. The cert our organization uses is a wildcard that only allows the form of *.cs.universityname.edu whi…

Souce name resolution OR change name of sources

I’m changing the names of the servers in our infrastructure, and I need the new names in graylog “source” field. I registered the names in /etc/hosts and I put the search for “files dns” in /etc/nsswitch.conf, and already created a entry A in my DNS, but the messages are still using the old source name. Can you guys give me a hint to force Graylog to use the hosts file to register the sources or another method? Can I rename Sources in Graylog? Thanks! Rafael Carvalho

Whois o DNS resolution on IP's inside the log message

Dear people, I have Graylog v2.0.3 and I’m receiving a lot of log messages from several own servers. These messages have different IP addresses inside, such as: full_message <190>Dec 12 16:14:21 proxy (squid): 1544642061.016 0 179.0.15.180 TAG_NONE/503 0 CONNECT pipe.skype.com:443 moliva HIER_NONE/- - Is it possible to add a feature that let me execute whois or a DNS resolution if a put the mouse pointer on the IP, in this case 179.0.151.80 ??? Special thanks !!!

[Ponet (Jesse Hills)]

If you are talking about the ‘source’ field being set as the IP address instead of the hostname that will be due to the host providing that in the logs.

AFAIK, Graylog does not perform DNS resolution in order to populate the ‘source’ field with a hostame/FQDN. Graylog will populate the ‘source’ field with whatever the value is that the host provides in its logs.

[Fabou78 (Fabrice)]

If you are talking about the ‘source’ field being set as the IP address instead of the hostname that will be due to the host providing that in the logs.

Are you sure about this? I have configured all my ASA the same way and they all show up as full DNS name and those devices have no knowledge of the domain (FQDN) they are in.

There must be some resolution going on here otherwise how would you explain all the other devices showing their source as FQDN?

Another things I am thinking of if that all logs are coming from rsyslog on the same box i.e. it is rsyslog that is forwarding all log to graylog so I might look in there to see if it’s not rsyslog the issue

[Ponet (Jesse Hills)]

Check the logs on the host in question. What is being written there?

@Fabou78

Another things I am thinking of if that all logs are coming from rsyslog on the same box i.e. it is rsyslog that is forwarding all log to graylog so I might look in there to see if it’s not rsyslog the issue

So, you are forwarding the syslog messages from the ASAs to another host and then forwarding them into Graylog from there?

[Fabou78 (Fabrice)]

rsyslog and graylog is the same server. All host send to rsyslog and then rsyslog forward to graylog on the same machine.

I have just checked on rsyslog and indeed the ip is not resolved there so I will try to to resolve the issue there.

So for my information, graylog will never try to resolve source ip then?

[Ponet (Jesse Hills)]

Not for the source field no. That is populated with data parsed out of the received log message.

Out of curiosity… Why are you forwarding the logs to an rsyslog server on the GL host and then forwarding to an input on Graylog? Why not just send the logs directly into an input in GL?

[Fabou78 (Fabrice)]

Out of curiosity… Why are you forwarding the logs to an rsyslog server on the GL host and then forwarding to an input on Graylog? Why not just send the logs directly into an input in GL?

Because:

1. My graylog skills are low
2. Could not start input for port lower than 1024 i.e. 514 and didn’t want to poke with the system
3. I would have to reconfigure all hosts (still achievable but some work)

I have just added . @@locahost:1514;RSYSLOG_SyslogProtocol23Format at the end of my /etc/rsyslog.conf file and job done.

Do you think it is a bad setup?

I used to think that is was bad setup but now I realized that rsyslog do the DNS resolution I start to think that it is good because I can see the meaningful FQDN as source in the logs rather than IP address and this without customizing anything in graylog.

The downside of it is that I have double logs on the devices (hight disk usage) which is partially under control in rsyslog via logrotate.

[Ponet (Jesse Hills)]

I would configure your systems to log their hostnames properly and have them send directly into a Graylog input. The extra rsyslog part is just unnecessary.

[Fabou78 (Fabrice)]

Ok,

I will consider this seriously. Another reason was if for any reason graylog breaks (configuration, upgrades etc…) you still have access to rsyslog. So it might be that once the graylog peaople skills become better we will move out from rsyslog and build a full redundant graylog solution.

PS: I have solved my original problem by restarting rsyslog so I assume it was doing some caching.

Thanks for your help on this