Use information from earlier log events to extend information in later events

Graylog Central (peer support)
1

1. Describe your incident:
I’m sending logs from our DNS server to Graylog and these logs contain the requested domain and replies (CNAME entries, A entries, etc).
Is there any chance I can load these into a lookup table or something similar? I’d like to reuse them later with our firewall logs that only show destination IP addresses right now but troubleshooting would be so much easier if I would see destination IP addresses and domain name of the ip address.
So can I use information from earlier events to add information to later events?

2. Describe your environment:
Graylog 4.3.15
Ubuntu 20.04.6 LTS

3. What steps have you already taken to try and solve the problem?
Google, Forum search

4. How can the community help?
Give me a hint or guide me to manual/posts that I’m not smart enough to find when I searched.

2

For resolving DNS (both forward {getting the IP of a hostname}, and reverse {getting the hostname of an ip}) i recommend using the DNS Lookup Adapter.

The longer answer to your question is there are a couple of ways to achieve what you are asking:

  • Dynamic Lookup Tables
    • This is a licensed feature so may not be applicable or helpful, but the way it works is you can use pipeline rules to dynamically store pieces of text in a lookup table that can later be referenced or “looked up”
  • Scripting and automation
    • This is a somewhat advanced solution, but the gist is you would have a script that, on a schedule, could query graylog, and then store the result in a way that graylog can use as a lookup table. This could be writing a CSV file, or possibly served via HTTP and used with the HTTP JSONPath data adapter

Hope that helps.