Help is needed graylog

[Lexaworld]

I started using Graylog relatively recently and this software is simply amazing in its functionality, but I came across one problem. Due to the production necessity, we collect logs from computers running the Windows system, everything works stably and the logs are sent as they should, but the problem is that in only one working day, at least 111 gigabytes of information are sent, so the question is whether it is possible to configure somehow reduction in the amount of logs? It’s just that I did not expect to see such volumes. I ask you to be understanding, and if you can tell me in detail about the solution to this problem. Thanks in advance !

[Ponet (Jesse Hills)]

You should configure your sources to only send the log events that you are interested in.

The configuration for this may be different depending on what solution you are using to send the logs however, I use Nxlog and know it can be done with it.

See the documentation at the below links:

https://nxlog.co/documentation/nxlog-user-guide/eventlog-filtering.html

https://nxlog.co/documentation/nxlog-user-guide/im_msvistalog.html

[Lexaworld]

Thank you so much for the right direction, I will study these guidelines, I really hope that everything works out for me.

[Lexaworld]

I have a question - if you can answer it - I will be very grateful! I opened the event viewer, set up event filters properly as we need it, clicked on the xml tab, there I saw the generated code. I read the instructions again, still don’t understand where to insert this code for me - I repeated everything again, checked the manual request change box, clicked OK. I waited a while, went back to the graylog - the number of logs did not decrease, it immediately became clear to me that I was doing something wrong, I read the tool again several times, where to paste the received code - it’s not clear so far, it’s just awful - please help me sort it out - tell me what I’m doing wrong ?

[Ponet (Jesse Hills)]

The XML query needs to be within the <input> block of your nxlog.conf.

[Lexaworld]

Thank you so much for helping me figure it out, here is my config file from nx log, where do I need to paste the resulting code? thanks in advance .

Panic Soft
#NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data

Module xm_syslog Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 Module xm_exec Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

[Ponet (Jesse Hills)]

The configuration shown above is missing <input>, <output> and <route> blocks.

See the configuration in the thread below for an example and adapt as required:

Windows Server NXLog - Graylog

Hello, I’m sending syslog data from Window Server using NXlog to Graylog and we’re generating lots of logs well over 500 message per second. Is there a setting I’m missing in NXlog or Graylog to filter which messages are sent to Graylog? [image] NXlog conf ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder you…

[Lexaworld]

I apologize, I didn’t understand you correctly from the beginning, here is the configuration file of which I provided below - apparently you were talking about it? If so, tell me where I need to insert the code I wrote about earlier - to correct the collection and sending of logs. I apologize in advance, it’s just the first time I’ve come across this and thank you for participating.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules

CacheDir %ROOT%\data

Pidfile %ROOT%\data\nxlog.pid

SpoolDir %ROOT%\data

LogFile %ROOT%\data\nxlog.log

LogLevel INFO

Module  xm_fileop

<Schedule>

    When    @daily

    Exec    file_cycle('%ROOT%\data\nxlog.log', 7);

 </Schedule>

Module xm_gelf

Avoid truncation of the short_message field to 64 characters.

ShortMessageLength 65536

    Module im_msvistalog

    PollInterval 1

    SavePos True

    ReadFromLast True

   

    #Channel System

    #<QueryXML>

    #  <QueryList>

    #   <Query Id='1'>

    #    <Select Path='Security'>*[System/Level=4]</Select>

    #    </Query>

    #  </QueryList>

    #</QueryXML>

            Module im_file

            File 'C:\Windows\MyLogDir\\*.log'

            PollInterval 1

            SavePos              True

            ReadFromLast True

            Recursive False

            RenameCheck False

            Exec $FileName = file_name(); # Send file name with each message

            Module om_tcp

            Host ########
            Port #########

            OutputType  GELF_TCP

            <Exec>

              # These fields are needed for Graylog

              $gl2_source_collector = '${sidecar.nodeId}';

              $collector_node_id = '${sidecar.nodeName}';

            </Exec>

Path eventlog => gelf

Path file => gelf

[Lexaworld]

Thank you so much for the tips! by trial and error, all the same, I achieved what I wanted, now we need to wait a while and see how this will affect the volume of incoming information. Tomorrow the will of dinner, unsubscribe about the results. Thanks again!