Post your pipeline rules here with comments or how-to documentation.`-- VARIABLE $$Ports_Seen_Threshold$$ String
– Ignoring log entries with src_port 53 (DNS) due to Unity Media Connect Box reset issues
WITH IP_Ports AS (
SELECT distinct device_serial_id AS Device_ID, src_ip AS Source_IP, dst_port, in_interface AS Interface, CAST(timestamp AS DATE) AS Scan_Date
WHERE log_type = 'Firewall'
AND log_component = 'Appliance Access'
AND src_port <> 53
ORDER BY dst_port
IP_Port_Count AS (
SELECT Device_ID, Source_IP, COUNT(dst_port) AS Port_Count, array_join(array_agg(dst_port), ',') AS Port_List, Interface, Scan_Date
GROUP BY Source_IP, Interface, Scan_Date, Device_ID
'PORTSCAN DETECTED' AS Detection, Scan_Date, Source_IP, Port_Count, Port_List, Interface, Device_ID
WHERE Port_Count >= $$Ports_Seen_Threshold$$
ORDER BY Scan_Date DESC`
Hey, guys I recently detected an ongoing scan deep in the logs we collect.
Found this code but I can’t see some functions that would help translate the above code correctly. Any ideas as to how I can go about it.
All of those parts of the message (Device_ID, src_ip etc.) can be broken out using extractors or the pipeline once you have the data broken out to fields, they can be acted upon based on Alerts to send you a message via e-mail (or… Slack, Teams, Pagerduty, http…). Does that make sense?
(NOTE: I marked the code as such with the </> forum tool to make it more readable and moved the question the “Graylog Central” area where questions are asked/answered.)
By chance is this Graylog 2.4?
We are using Graylog 3.1.4.
What @tmacgbay suggested is the way to go, this would depend what you want to use. Pipeline would be you best option using regex expression then create the fields for each. I would like to suggest that only create the fields that are need because it will take up more space.
Here are a couple examples I have used since the logs look similar to one of my devices.
Maybe something like this in a pipeline.
rule "Source Port"
let batman = regex("srcport=(\\d\\))",to_string($message.message));
Actually testing out my pipeline above I found it was incorrect.
This should work better.
rule "Extract fields"
let siteName = regex("(srcport=\\s*(\\S+))", to_string($message.message));
rule "Extract multiple fields"
let srcport = regex("(srcport=\\s*(\\S+))", to_string($message.message));
let protocol = regex("(proto=\\s*(\\S+))", to_string($message.message));
let sourceInterface = regex("(srcintf=\\s*(\\S+))", to_string($message.message));
@tmacgbay It was tuff because my Extractor REGEX did not work in the pipeline
I’m not sure they pay you enough for posts like this @gsmith!
I thought about using the
key_value() function with the data but only specific fields are desired.
(On a side note it helps a LOT if you post the text of the message rather than a picture, it gives others something to play with for the answer…)
With regex it would be more efficient to have a single search and use the grouping to pull out the things you want… something like this:
rule "Extract multiple fields"
let BatSignals = regex("srcport=(\\S+).*proto=(\\S+).*srcintf=(\\S+)", to_string($message.message));
Thanks @tmacgbay Me likey
I was working on trying to make it simpler.
Speaking of key_value, did this ever get fixed?
I don’t think so… I think it is dying a slow death of neglect.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.