Getting more info from backlog messages in SMTP notifications

[theconqueror (Brian)]

Just setup a new and my first Graylog instance. I am currently sending message from Cowrie, an SSH honeypot, to Graylog. The messages that come into Graylog are all of the commands someone typed in the SSH session. If I look at all of the messages in Graylog you can see this.

I’d really like to get a single e-mail with all of those commands in there. I configured the event definition to have a message backlog of 10 messages. With the default e-mail notification I get a lot of weird data that I’m not really sure I’m comprehending.

{index=graylog_1, message=New connection: 10.93.36.50:27145 (10.93.4.229:22) [session: ea294ff6aad4], fields={eventid=cowrie.session.connect, gl2_accounted_message_size=349, gl2_receive_timestamp=2024-07-24 16:07:24.002, level=1, gl2_remote_ip=10.93.4.229, session=ea294ff6aad4, gl2_processing_error=Replaced invalid timestamp value in message with current time - Value <2024-07-24T16:07:23.735322Z> caused exception: Invalid format: “2024-07-24T16:07:23.735322Z” is malformed at “T16:07:23.735322Z”., gl2_remote_port=53564, gl2_message_id=01J3JQYKX900001NJ9JZ8D9A0W, gl2_source_input=668c50840653422bacf2aa6c, dst_ip=10.93.4.229, src_ip=10.93.36.50, src_port=27145, gl2_processing_timestamp=2024-07-24 16:07:24.331, protocol=ssh, system=cowrie.ssh.factory.CowrieSSHFactory, dst_port=22, gl2_source_node=9315c78a-db32-42f2-b372-a2f82a7a0967, sensor=backup, time=1.7218372437353218E9, gl2_processing_duration_ms=329}, id=d0906e10-49d6-11ef-b964-0242ac120004, timestamp=2024-07-24T16:07:24.329Z, source=backup, stream_ids=[000000000000000000000001]}

I tried tweaking the message body from the default

${foreach backlog message}
${message}
${end}

to

${foreach backlog message}
${message.message}
${end}

and that seemed to get me one of the messages that I saw in the search screenshot I posted above, but it was just one and not all of them. How do I get all of the backlog messages in the e-mail notification?

I tried looking though the docs, but there wasn’t enough there for me to totally understand and the JMTE doc isn’t rendering properly which makes it a little difficult to read. Can you at least point me in the right direction? Thanks

[gsmith]

Hey @theconqueror

For an email template you can do something like this.

Adding Stream URL to Notification Template

I didn’t realize the need for a URL in my notification until receiving a large amounts of alert emails. Even thou I did make configurations to the Notification template and can find who/what/and where something happened. I did find it nice, if need be, to click a link that would take me right to the alert message for greater understanding on what happened. The default settings for HTML only showed the ${event.source_streams} that would still require me to search and let it be known, I’m kind…

If you need all those message , lets say for an hour you need to adjust you setting in event definition/alerting section.

[theconqueror (Brian)]

Hmm, I’m not sure I totally follow here. I don’t want to add a URL to the e-mail to view the rest of the messages as I want to be able to see these alerts on my phone when I’m off the company network. I could see how that would help if I was onsite, but it’s not always the case.

When you say to adjust the settings in the event definition/alerting section - do you know what settings I would need to adjust? The time that these messages come in are very quick, they get sent for every command entered, well under 5 minutes. Its set to search within the last 5 minutes every 5 minutes. I also have message backlog enabled to include 10 messages. Anything else I would need to tweak in there?

[gsmith]

hey @theconqueror

@theconqueror

I don’t want to add a URL

I understand, but the template shown there has configuration you may need.

As for backlog messages.

@theconqueror

backlog messages in the e-mail notification?

Create a event definition with Filter & Aggregation condition Type. your settings for “Search within the last” && “Execute search every” is what you will need. you can modify what you want to see/get by enabling “Aggregation of results reaches a threshold”. there you can group by fields etc… the Notification template is only for configuring message to be sent. For example if you want only Event field you can configure your notification template to show only that field and not the whole message.

[theconqueror (Brian)]

I appreciate that you’re hanging with me here through my ignorance. Just to clarify, when these messages come in, I want all of them to be sent to me via e-mail whether it’s one message or multiple I dont really mind at this point, I just want the data.

This is all I get in the backlog messages

**Backlog (Last messages accounting for this alert)**
Remote SSH version: SSH-2.0-PuTTY_Release_0.81

I have the event definition set as Filter & Aggregation and the search query is anything from this host. I have “Search within the last” set to 5 minutes and “Execute search every” set to 5 minutes. “Create event for definition if…” is set for “Filter has results”

Do I need to use this aggregation feature? It seems to offer the ability to put conditions around when the event is created, like if I wanted there to be at least 20 messages before an event was created, but I don’t want that. I just want everything to be sent out. I tried setting it to if count of message is greater than 3, but now I’m not getting any alerts.

I took a look at the original post you referred to again and I’m still struggling to see how I can make use of it.

Thanks for your help

PS - is there any other reading good reading material than the docs on the website?

[Wine_Merchant]

Hello @theconqueror,

You can sign up here and take the Alerts, Events and Notifications course for free.. It will help give a broader context on the subject.

You could also script an API call to collect these messages, docs for that here.

[theconqueror (Brian)]

Thanks for that. I just signed up.

script an API call? That sounds overly complicated. I’m starting to worry that I’m not describing things properly. I just want those search results in an e-mail. Any time something meets that criteria, send me all the data in an e-mail.

[theconqueror (Brian)]

Just keeping this open so I can post my solution when I find it