I have a web app that generates its own logs. At the moment ,I am just writing them to a file on the local disk, but I want to put them in graylog. I have found out that there are several ways to do this, and I am looking for advice on what is the best way:
way 1: Send directly to graylog via udp. Downside of this is that some logs get lost due to non-reliability of udp.
way 2: Send directly to graylog via tcp. Downside of this is that if graylog goes offline, I will lose logs and it will block the web app.
way 3: Write the logs to disk, and then get another program (an agent) to send the logs to graylog. No downsides but more complicated. Will lose logs if disk is full, but this will not happen.
I think way 3 is the best, but I am not sure. What do other people do?
why tcp would block your app? if it’s done good (logging library, exception handling) it shouldn’t block, it should just disconnect and continue while trying to connect to graylog via tcp
Personally, I would use a logging agent /w TCP/TLS if possible.
The reason why some agent/s have a caching directory like Nxlog. Not sure about Winlogbeat, etc…
Yes, we use nXLOG at my work, so I am planning on using that. I have a question though. What format should I write my logs to disk as? Should I write my logs as GELF? So one GELF object per line?
I read an article that if you use TCP for logging and the destination is down, then the client will wait an unspecified number of seconds trying to connect, which will block your app. I actually tested this, and I didn’t notice this, so I am not sure if it’s true.
I need to research this. At any rate, if I use a non blocking connection, that could work.
Anyway, my preference is for an agent-based solution, as it’s more resilient to network errors.
