Force rDNS and Slow Processing

Graylog Central (peer support)
1

Couldn’t find any solution to this so here I am asking you guys. Basically, I have a UDP input for all my network devices and it can only process out about 3.5K msg/s. After that, the process buffer starts filling up so it was probably an extractor or some other parsing process causing this. Turns out that if I disable the force rDNS option, the input can process up to 14K msg/s

Question is, how can I improve this? It looks like I have to use rDNS since some syslog messages get erroneous source names if I don’t.

2

You need to deploy a local DNS resolver which caches the requests and answers fast enough.

3

Thank you for the reply. Is this what you’re referring to?

Otherwise, would you be kind enough to provide a link to documentation that puts me in the right track?

TIA.

4

No, I mean something like Unbound or dnsmasq.

1 Like
5

Just to follow up on this. I installed and configured Unbound and added both A and PTR records for my sources. I confirmed that Graylog is using these entries to parse the source of the messages by the meands of using rDNS.

But still, it looks like it’s rated at ~3K msg/s.

Is there anything else that can be done to improve this?
Would using pipelines or extractors to replace the source with a specific string, based on the gl2_remote_ip field be faster? (a quick example on how the extractor would look would be nice)

I’m open to more suggestions, thank you.

6

Given that your Unbound instance is already optimized and cannot be made faster anymore, you could only disable reverse DNS lookups.

Maybe, it depends on what you want to do specifically.

7

Let’s say I’d like to parse all messages coming into a certain input (Syslog UDP) and set the ‘source’ field to some string that depends on the source IP (would be mimicking what DNS does).

For instance, this is the pseudo code as I don’t know the syntax of Pipeline rules yet:
if message field ‘gl2_source_ip’ == 10.0.0.1
then set message field ‘source’ to ‘host1’
if message field ‘gl2_source_ip’ == 10.0.0.2
then set message field ‘source’ to ‘host2’

(…)

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.