Is it possible to use Graylog to parse Syslog messages and compare devices connecting to the network with some sort of list to determine if they are known trusted clients? And then alert if the device isn’t in that list?
You can use Graylog tables if you have a fixed list. Here is an older post on how those work but it is still relevant. You can also set up a table connecter to see if the device exists in your internal DNS. Here is the latest docs on Tables.
I follow along just fine until the pipeline >> Manage Rules section
how do I create a rule that checks the field “MAC” to see if it is in my lookup table and send an alert if it isn’t found?
or how would I create a query that would check if the “MAC” field is not in the lookup table?
Should I assume you have created the table? Can I assume you know how to create rules and pipelines? Did you look at the lookup_value() function and try it in a rule attached to a pipeline attached to a stream? If so you could post the code you tried (Using the forum tools like </> to make it readable)
Create a table with all MAC addresses, then create an input to receive the logs, using the input extractors or a pipeline to break out the fields, then take the field containing the MAC address and use the lookup_value() function in your rule to see if it exists in the table… not sure where you are in that process…
I have created the lookup table from a csv file. I called it " MAC Address Table" and the test lookup succeeds so its working.
however I am at a loss as to the language for the queries and how to actually use that table.
I under stand the basics from this article, Search query language — Graylog 4.1.0 documentation
but haven’t seen any good examples or resources explaining how to use functions or do advanced queries. for example the lookup_value() function you mention. I can’t find any info the explains what this function does or how to use it (or what other functions exist)
I am new to GrayLog and have only been using it for a couple of days, so pardon my lack of knowledge on the product.
Here is the short - Some device sends log data to Graylog that hits the Input you have set up. The log data (record) is broken into fields with an extractor if you have set it up (you can also do this later in a pipeline - don’t forget to review your message processor configuration!). Data then flows to streams based on stream rules you have set up (streams have underlying indexes usually other than the default index… indexes are the separate databases in Elasticsearch) if you don’t create rules to route, the record goes to the default stream/index.
This is the point where your question comes into play. You can attach a pipeline to a stream that can manipulate the record data and fields (via rules) including things like deleting the record or routing it to a separate stream. Pipelines contain rules that you build for the actions you want - in a pipeline rule you can say compare a field to data in a table using the lookup_value() function. If you read through the blog post from my original reply it goes into detail on that. Going the other way - functions reside in rules set up to manipulate data flowing through a pipeline that is attached to a stream (which has an underlying index) that receives data from an input.
There is also alerting and reporting, etc. but you didn’t ask about that.
right,
I have read through that article. Everything is good up to this part
which is where it goes off the rails… LOL The article seems to make the assuming I know the query language used, which I don’t. as I said above there is no definition of the lookup_Value() function, what it is doing or how to use it. just a brief example that likewise doesn’t explain itself.
That is where I am stuck. I need to under stand the query language in order to be able to make any rules to do anything.
There is really not all that much to the language - it is here in the documentation. There are no loops, its held to a relatively simple “if this then that” structure (or technically when… then… end) When you are writing rules, there is a searchable reference area with all the functions on the right side of the page - you can click on any function to find out simple details about it.
Here is an example of one that includes table lookup that I wrote to massage data based on finding a windows event_id:
rule "AP3-WinSec-BadPw"
when
// If the record you are running against has and event id field containing 4625
to_string($message.winlog_event_id) == "4625"
then
// Create the "subject" of the e-mail alert we will send later in a field called called e-mail_subject
let subject_0 = concat("FAIL: ", to_string($message.winlog_event_data_TargetUserName));
let subject_1 = concat(subject_0, " connecting to ");
let subject_fin = concat(subject_1, to_string($message.winlog_host_name));
set_field("e-mail_subject", subject_fin);
//
// create detail of (e-mail) alert
let LogonTypeNumber = to_string($message.winlog_event_data_LogonType);
// -----It is below where I am referencing a table I previously set up
// Set LogonTypeResult to be the results of finding the contents of the LogonTypeNumber field in
// the table called winLogonType. If you can't find it, set LogonTypeResult to 0
let LogonTypeResult = lookup_value("winLogonType",LogonTypeNumber, 0);
// below here the funtion reads:
// Set LogonTypeErr to be the results of finding the contents of the
// $message.winlog_event_data_SubStatus (converted to a string with to_string()) field in
// the table called WinLogonErr. If you can't find it, set LogonTypeErr to 0
let LogonTypeErr = lookup_value("WinLogonErr" ,to_string($message.winlog_event_data_SubStatus), 0);
///From here build the body of the message
let build_mess_0 = concat("Failed Password Attempt - ", to_string($message.winlog_event_data_TargetUserName));
let build_mess_1 = concat(build_mess_0, " attempting to log in to ");
let build_mess_2 = concat(build_mess_1, to_string($message.winlog_event_SubjectDomainName));
let build_mess_3 = concat(build_mess_2, "-");
let build_mess_4 = concat(build_mess_3, to_string($message.winlog_host_name));
let build_mess_5 = concat(build_mess_4, ". Logon Type: ");
let build_mess_6 = concat(build_mess_5, to_string(LogonTypeResult));
let build_mess_7 = concat(build_mess_6, ". Attempt came from: ");
let build_mess_8 = concat(build_mess_7, to_string($message.winlog_event_data_WorkstationName));
let build_mess_9 = concat(build_mess_8, ". ERROR: ");
let build_mess_fin = concat(build_mess_9, to_string(LogonTypeErr));
set_field("e-mail_body", build_mess_fin);
route_to_stream("WinErrors");
end
3 Likes
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.