Inputs and configuration

[rooster (Dave Thistlewood)]

Hello all! For a brief background, I walked into this (literal) job position picking up where the previous employee left off mid-setting up a graylog server. I’ve gotten it up and running, found an nxlog agent he tried to get working and have it running; Life is wonderful.

We have an estimated 20 servers I’d like to be collecting data from, specifically event logs from Windows servers.

From what I’ve read doing research, it appears all this can be done with one input as I’m looking for the same data (event logs) from multiple servers, and all this would be done on one port (for sake of ease, we’ll say I’m using 12201, the default for gelf_TCP which is what was in place already (though not on that port).

The problem is, I’ve gone to data_collection_server_2, installed nxlog, copied the conf to the proper folder, restarted the service and data does not appear to be getting sent to graylog. It was easy to see when the 1st data_collection server made contact as there was input activity, but it is harder to notice additional servers. Am I just not waiting long enough for data to appear? I wait a bit, see nothing, then create a second input in graylog for (the same thing) just with the new windows server name as it’s name and a new port, edit the port on the data_collection_server_2, and shortly thereafter the new input is getting data.

From what I read, 1 input in graylog is good for many/all servers that are all using that same agent (or whatever, to be fair I don’t really know/understand all the different input collection things, just appears myt predecessor chose gelf_tcp/nxlog as it grabs windows events) and should allow all servers to dump their event logs on the graylog via the same port assigned to that input. Is this accurate?

In addition to said event logs, what else does the community recommend I gather (what CAN be gathered) from Windows servers that could/might be of use and how would I go about including it? Below is my existing conf file for reference…

–File added as an upload attachment because the structuring kept making weird things happen in the actual post lol–
–never mind, cannot attach non-image files?@?@! Guess I’ll paste it in here as a screenshot otherwise it’s going to look weird and messed up but is now nearly useless to edit/comment on

Thanks so much in advance, really appreciate it!

[faen]

I’ve always had a 1:1 input to source ratio. I never considered that you could have more than one source use the same input and would be interested to read what others think.

With regards to the Windows logs, I use Winlogbeat, which is installed on each Windows server that is sending logs to Graylog. On the Graylog end, I have a Beats input set up for each Windows server. Winlogbeat can be configured to run as a service, and there is a small YML config file that must be edited.

On my end, I use processing pipelines to clean up the data a bit and get rid of fields that I don’t think I need.

Dashboards are used to report on the resulting data sets.

Hope that is helpful!

[wdarcey]

I am just starting out with graylog but my approach so far has been the opposite: Minimal inputs (only one so far), splitting message traffic into streams where I need to process or retain messages differently.
I am managing, so far, to send diverse syslog formats to one input and distinguish/process them fairly easily.
With beats and their prepopulated metadata, I’d have thought the same could apply, maybe more so.
With firewalls and containerised applications, the fewer ports involved the better. IME doing it this way meant moving logic from extractors to rules/pipelines.

[rooster (Dave Thistlewood)]

wdarcey, do you use nxlog to get the events, and 1 port (12201 is the default I believe)? I see you mention beats - If so, what made you choose beats/winbeat over nxlog?

[gsmith]

Hey @wdarcey

I was using nxlog for along time, since then I have moved to Winlogbeat/Filebeat.
My Nxlog here is a simple configuration Ihave been using with Graylog.

 root@elastic-stack:/var/log# cat /etc/nxlog/nxlog.conf
########################################
# Global directives                    #
########################################
#User nxlog
#Group nxlog

include /etc/nxlog/nxlog.d/*.conf
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO

########################################
# Modules                              #
########################################
<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _gelf>
    Module      xm_gelf
</Extension>

<Extension _json>
    Module      xm_json
</Extension>

<Input messages>
    Module       im_file
    FILE         "/var/log/*.log"
    SavePos       TRUE
    ReadFromLast  TRUE
    PollInterval  1
    Exec  $Message = $raw_event;
</Input>
<Output out>
    Module      om_ssl
    Host        graylog.domain_name.com
    Port        51411
    OutputType  GELF_TCP
    CertFile        /var/lib/nxlog/cert/graylog3-certificate.pem
    CertKeyFile     /var/lib/nxlog/cert/graylog3-key.pem
    CAFile          /var/lib/nxlog/cert/cert3.pem
    KeyPass         secret
    AllowUntrusted  true
    Exec $short_message = $raw_event;
    Exec $Hostname = hostname_fqdn();
</Output>

########################################
# Routes                               #
########################################

<Route>
  Path messages => out
</Route>

Never had an issue except for certificates. and if you notice I do not comply with default ports

EDIT: I should have added the corresponds to my input.

[wdarcey]

It’s not a conscious choice. I’m not familiar with nxlog yet.
I have only ever used syslog (rsyslog and syslog-ng) and occassionally filebeat to ship logs to elastic. Now I’m exploring graylog, I am becoming aware of other methods to ship logs. I currently have no need for shipping windows logs and some of my devices won’t run anyrhing other than syslog, I concentrated my efforts on linux syslog forwarding.