Hi Guys, I’m new to the forum. I hope I won’t do anything wrong.
I’ll explain my situation:
I was given the task of putting up a graylog server for the security team to check for behavior anomalies in the administrative accounts. I’m not in the IT department, so I’m struggling a little, but I managed to do it.
The server
The server is installed in a VM with enough resources to make it function properly (it didn’t make me doubt that so far)
CentOS 7
graylog-server v3.1.0
elasticsearch-oss v6.6.2
What I need
We have a few hosts that send syslog messages to the Graylog server, something like a Domain Controller, a VPN, a couple of firewalls and some production server.
Unfotunately, it was the IT department that set up the input stream (just that) and it’s a single stream gathering every syslog message, so I will have to come out with some rule to divide by host because they are not going to change the input stream anytime soon.
I need, at this point of the task, a way to divide logs by whether they were generated during business hours or not.
Ideally, in the end I should be able to make an executive level monthly review with a couple of charts (just to stick to compliance rules, i guess)
The pipeline
I studied the documentation and did some research on google and on this forum (where the rule closest to my needs was in this topic) and i came with the following pipeline:
Stage 0 (All must be satisfied)
rule "has received time"
when
has_field("EventReceivedTime")
then
end
rule "has target username"
when
has_field("TargetUserName")
then
end
I was testing this pipeline on a limited set of results
Stage 1 (Any one can be satisfied)
rule "business hour flag"
when
to_long(to_date($message.timestamp).hourOfDay) >= 7 &&
to_long(to_date($message.timestamp).hourOfDay) <= 19 &&
to_long(to_date($message.timestamp).dayOfWeek) >= 1 &&
to_long(to_date($message.timestamp).dayOfWeek) < 6
then
set_field("business_hour", "true");
end
rule "not business hour flag"
when
to_long(to_date($message.timestamp).hourOfDay) < 7 &&
to_long(to_date($message.timestamp).hourOfDay) > 19 &&
to_long(to_date($message.timestamp).dayOfWeek) < 1 &&
to_long(to_date($message.timestamp).dayOfWeek) > 5
then
set_field("business_hour", "false");
end
Both rules were tested with and without quotes around the true/false values
The issue
Although i see some sort of throughput in both the stages of the pipeline, the throughput, which I expect it to be the same on the stages, is uneven. Moreover, and this is the real problem, in the search, none of the logs has the business_hour field
I set, as I saw in the documentation, this in the message processor configuration:
1 AWS Instance Name Lookup active
2 GeoIP Resolver active
3 Message Filter Chain active
4 Pipeline Processor active
Any idea of the reason why this pipeline doesn’t set the new field?