Hello dear,
My company is using Scalyr for monitoring logs. Now, i am making the POC using Graylog2.
In Scalyr, there is a scalyr agent that manage logs before send to Graylog.
For example,
sampling_rate: 0, it means the log will not be sent to Scalyr.
So, i am curious that feature in Graylog2. Do you know what feature called as Graylog agent or something which can be filtering logs before send it to Graylog ? If the access.log has 1.5 GB (500 MB error 5xx, and 1 GB error 4xx), and i want to send error 5xx only (500mb), it will be save more disk usage
the type of shipper you choose gives you different options to deal with that situations. When you go with filebeat you can use the exclude function. nxlog should have the same options given and syslog would be also possible.
Graylog has the Collector Sidecar what is a kind of supervisor to collectors but you are not bound to use that. The free Software world give you lots of options but not the golden way to follow.
Where are youre log messages located? What host System? You can use whatever logshipper is best for your situation.
If the source is a Linux System and a file on disk, take filebeat as simple go binary that is the most powerful way to go.
@jan Thank for enlightening me. You are right, jan.
I found the documentation about these Exclude and Include lines.
I was not sure, i thought it should be processed by Graylog before writing into ES. But, it depends of the shipper/collector.
If the scalyr has sampling_rate : 0.1 (10/100). it means, shipper or scallyr will ship 10% of total.
Do you know what feature in Filebeat or Graylog2 ?
Thank you very much for your time
