I am completely new in the Graylog world ! and i have few question about log shipping.
I planned to use Graylog to centralize some logs from different .NET applications. They are currently logging trace into plain text files(one per day YYYY-MM-DD.log)
If i use a .NET library to send GELF log, what if if the network is down (not the Graylog server) but some external event that make the communication between the application and the Graylog server impossible. i suppose that the logs are lost ?
So i imagine to keep to log into local files and use filebeat log shipper or nxlog. Does it handle multiple files (one per day) ?
Or is there any local GELF forwarder that my application can send GELF logs and which can spool in case of error ?
This depends on the actual transport for your logs. If the library you’re using doesn’t buffer logs locally in case of a network failure, your logs will be lost. Otherwise it depends on the size of the local buffer.
Alternatively, you could use a broker like RabbitMQ or Apache Kafka to add a little more resilience to your logging setup.
Yes, Filebeat and probably also NXLOG are able to handle file rotations.
Thank you very much for your answer.
I guess that my .NET library only send and forget the logs.
As i want to handle network long time issue (for example, a bad firewall configuration, remote internet connectivity which can take hours to be detected and solved), i think that even a local buffer in the library will not be enough (particularly if my application stops, the log buffer is lost)
The RabbitMQ or Apache Kafka is a good solution but i have to install it to another server (and if there is a firewall issue ou internet connectivity issue), it would be the same.
Based on your response, i think the best for me is to “buffer” locally in plain log files and then forward them with NXLOG or Filebeat.
