Disclaimer - Very new to Graylog. Everybody starts somewhere
1. Describe your incident:
Need help correlating logs from an Axigen e-mail server.
The logs from the e-mail server are broken into four distinct categories.
SMTP-IN (in bound smtp transactions) field: smtp_in_id
2025-06-28T19:51:10.683Z source=192.168.0.224 2025-06-28 19:51:10 +0000 08 axigen SMTP-IN:00000038: Set max data size to 20480 KB
SMTP-OUT (outbound smtp transactions) field: smtp_out_id
2025-06-28T19:51:18.123Z source=192.168.0.224 2025-06-28 19:51:18 +0000 08 axigen SMTP-OUT:00000005: Relay mail 2C9FDD: connected to 17.42.251.62:25
PROCESSING (internal processing of the e-mail) field: processing_id
2025-06-28T19:51:18.086Z source=192.168.0.224 2025-06-28 19:51:18 +0000 08 axigen PROCESSING:002C9FDD: Processing finished
DNR (DNS resolution activity) field: dnr_id
2025-06-28T19:51:10.851Z source=192.168.0.224 2025-06-28 19:51:10 +0000 08 axigen DNR:00000038: Sending query (1/1) to 192.168.0.1:53
Using Pipeline rules, I can create fields for each of the four elements, and thus they are each independently able to be queried. What I need to figure out is a way to correlate all of the log entries using these four fields to be able to display the logs that are relevant to a single mail transaction.
This server is a relay, so SMTP connections are made, mail is delivered, processed, and then redirected out to another server. The end goal is to be able to provide a list of some sort on the Dashboard of “transaction_ids” that when selected will show all of the logs across these four categories for that single inbound —> outbound transaction.
Some of the log entries have what I would call correlation points. An example would be the SMTP-OUT example I show above. In that message, you have the SMTP-OUT ID hex (00000005) and the hex (2C9FDD) that correlates to the PROCESSING ID (002C9FDD).
I believe there are enough points of correlation through a single transaction to derive the necessary values for each of the four fields.
I just have no idea how to do that. I have looked at Lookup Tables, and all of the various rules and filters, but I’m not finding a clear path to how this can be accomplished.
2. Describe your environment:
-
OS Information:
For testing I am running Graylog stack in Docker on a Raspberry Pi 5 8GB - 1TB SSD -
Package Version:
Version: 6.2.4 Enterprise -
Service logs, configurations, and environment variables:
3. What steps have you already taken to try and solve the problem?
As described above, I’ve built the necessary regex rules to build four fields that can be used to select the necessary details. I’ve created a fifth field called queue_id, that grabs the hex name of the queue when an e-mail is accepted for processing. This can be used to derive the appropriate processing_id. I’m working on creating other fields that can be used for points of correlation.
4. How can the community help?
I’m looking for any help folks are willing to give. I won’t lie, it would be nice if somebody just dropped the answer and how to implement it, but I’m happy with suggestions on direction, things to try, and areas of exploration.
Thank You
Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]