I’m attempting to solve a few inherent issues with Unraid, all revolving around logs, and pushing them into Graylog.
As a pre-req, I am new to Graylog, I have read as much of the docs as I can, however without knowing what does what, its a bit of a needle in a haystack. So obvious answers to experienced Graylog’ers such as ‘well just use X, its in the docs’, is not going to be obvious to me.
So kid gloves would be appreciated, I’m here to learn.
Problem Area 1: Unraid
Unraid stores its logs in RAM, so should the system reboot or lock up…you lose the logs.
There’s some creative ways about it, like having a cron job run to flush the logs to another device on a schedule, but I’d rather start telling all my devices and services to send logs to one location (Graylog), running on a self-contained system. In this case, a Raspberry Pi.
Context for issue
The first issue is that I’m finding that Unraid (and UniFi in all fairness) is sending the syslog message with incorrect timestamps (4 hours in the past), which when ingested into Graylog means that ‘recent queries’, or anything that relies on them, are next to useless.
The timezone is set correctly at both ends.
The “workaround” is to flip the input type from ‘Syslog/UDP’ to ‘RAW UDP’, which is not ideal, as you lose some of the automatic parsing being done.
But it does mean that the message timestamps in use are correct and usable, if if the timestamp in the text itself remains wrong.
Why do I know Graylog is behaving?
I’ve added the relevant extra parameters to my containers to push logs with tags into Graylog.
These all come into Graylog with everything correct.
My logic is that if Graylog had an inherrant config issue, it would affect that too…?
Is there a way that I can solve this Graylog side?
Is there a way to ‘overwrite’ the incoming timestamp perhaps?
Or automatically truncate the first 20 characters (that contain the time stamps) perhaps?
Note: I’d still want to ‘solve’ this on Graylog’s side irrespective of Unraid, as UniFi has the same issue.
Is this likely to be a rsyslog service config error in Unraid?
As if so, I would rather fix at source instead of workaround it (in the case of Unraid).
Could it be a format issue?
Problem Area 2: General Usage Questions
Q3: I’ve been splitting my incoming inputs into their own streams, then filtering into their own indicies to help seperate them from a data perspective. Is this good, bad or pointless to do?
Q4 Plugins and content packs - limited reading thus far, but surely i do not need to create custom docker images just to add a plugin? Is it not just a case of drop into a directory, restart the container and go?
Q5 Any other general tips?
Thank you in advance for your help with these probably basic questions.