hello
My linux server send too many log to my graylog server (2M logs in 1minutes) and because of that my graylog server is laggy
i transfer my logs off my linux servers with syslog-ng
Syslog-ng conf
source s_src {
file("/var/log/messages");
file("/var/log/debug");
file("/var/log/error");
};
destination d_net {
tcp(“GraylogIP” port(3516) log_fifo_size(1000));
};
log {
source(s_src);
destination(d_net);
};
this configuration of syslog-ng is the same on both
my input on graylog is syslog tcp : bind_addr 0.0.0.0 port 3516
my real problem is my linux server send too many logs to my graylog server or its normal and then its my graylog configuration
is not enough good to treat all my logs this is only on my linux and not on my windows server to provide my logs to my graylog server
i used nxlog with this configuration:
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf\nxlog.d
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
Module xm_gelf
Module im_msvistalog
Module im_mseventlog
Module om_tcp
Host graylogIP
Port 3514
OutputType GELF_TCP
<Route 1>
Path win => out
Im on Graylog 4
elasticsearch : “version” : {
“number” : “7.10.2”,
“build_flavor” : “oss”,
“build_type” : “deb”,
“build_hash” : “747e1cc71def077253878a59143c1f785afa92b9”,
“build_date” : “2021-01-13T00:42:12.435326Z”,
“build_snapshot” : false,
“lucene_version” : “8.7.0”,
“minimum_wire_compatibility_version” : “6.8.0”,
“minimum_index_compatibility_version” : “6.0.0-beta1”
mongodb 4.0.28
i have try to limit the incoming flow from my linux server but it didn’t work
i can increase it a little but i dont have enough to responde to your prerequisites did you have some tips or tricks
i can do