New graylog3 user here. I have read all the documentation and
fundamentally understand all the parts of gl3 and how they work together
and have been able to setup monitoring and alerting on a few practical
test cases. The marketplace has been a great resource to see how
different applications can be integrated into gl3.
The part I’m having a hard time with is putting it all together for a
comprehensive enterprise deployment. Are there any complete public
examples, or architecture guides? How and when to create
new streams, indexes, how to designate and classify inputs, and how to
manage sidecar configurations across many operating system versions and
distributions.
For example, I have a Debian 9 machine running a Java application. I
want to monitor my custom application from a developer and security
standpoint. I also want to monitor system logs and auditd.
I have another CentOS server running Radius and want to monitor Radius,
system logs, and auditd.
In this example, I also have a large Windows domain with sysmon deployed
through GPO and enhanced Windows auditing also enabled through GPO.
There are also various firewalls, network devices, IoT etc.
How do I organize inputs? Lets say I like using beats (I do), do I have
1 beats input that takes messages from all those different sources, or
create separate purpose-driven inputs?
How do you organize streams? I understand logs can go to multiple
streams so I would want an authentication stream where I standardize
username fields where I can query a user across all log types for
example. Other than these context specific secondary streams, do people
generally just use All Messages, or is there a better approach?
Lastly, what’s the best practice for organizing sidecar configs? All of
my Linux servers need auditd and system monitored, but each one has
other unique applications (nginx, custom Java app, Radius, etc), so do I
need a unique sidecar config for each stack?
I know I’m asking a complicated question and communicating it poorly but
if anyone can identify where my confusion is and point me to some docs
or sample configs I’d appreciate it.