Hi Guys, I’m new to the forum. I hope I won’t do anything wrong.
I’ll explain my situation:
I was given the task of putting up a graylog server for the security team to check for behavior anomalies in the administrative accounts. I’m not in the IT department, so I’m struggling a little, but I managed to do it.
The server is installed in a VM with enough resources to make it function properly (it didn’t make me doubt that so far)
CentOS 7 graylog-server v3.1.0 elasticsearch-oss v6.6.2
What I need
We have a few hosts that send syslog messages to the Graylog server, something like a Domain Controller, a VPN, a couple of firewalls and some production server.
Unfotunately, it was the IT department that set up the input stream (just that) and it’s a single stream gathering every syslog message, so I will have to come out with some rule to divide by host because they are not going to change the input stream anytime soon.
I need, at this point of the task, a way to divide logs by whether they were generated during business hours or not.
Ideally, in the end I should be able to make an executive level monthly review with a couple of charts (just to stick to compliance rules, i guess)
I studied the documentation and did some research on google and on this forum (where the rule closest to my needs was in this topic) and i came with the following pipeline:
Stage 0 (All must be satisfied)
rule "has received time" when has_field("EventReceivedTime") then end rule "has target username" when has_field("TargetUserName") then end
I was testing this pipeline on a limited set of results
Stage 1 (Any one can be satisfied)
rule "business hour flag" when to_long(to_date($message.timestamp).hourOfDay) >= 7 && to_long(to_date($message.timestamp).hourOfDay) <= 19 && to_long(to_date($message.timestamp).dayOfWeek) >= 1 && to_long(to_date($message.timestamp).dayOfWeek) < 6 then set_field("business_hour", "true"); end rule "not business hour flag" when to_long(to_date($message.timestamp).hourOfDay) < 7 && to_long(to_date($message.timestamp).hourOfDay) > 19 && to_long(to_date($message.timestamp).dayOfWeek) < 1 && to_long(to_date($message.timestamp).dayOfWeek) > 5 then set_field("business_hour", "false"); end
Both rules were tested with and without quotes around the
Although i see some sort of throughput in both the stages of the pipeline, the throughput, which I expect it to be the same on the stages, is uneven. Moreover, and this is the real problem, in the search, none of the logs has the
I set, as I saw in the documentation, this in the message processor configuration:
1 AWS Instance Name Lookup active 2 GeoIP Resolver active 3 Message Filter Chain active 4 Pipeline Processor active
Any idea of the reason why this pipeline doesn’t set the new field?