We have 1 graylog instance running in our company. We are currently sending IIS and Windows Event Logs logs to graylog.
But I want to start sending our custom applications logs’ to graylog. The way I plan on doing it is using TCP to send a GELF log to graylog. I like the idea of the _[additional field] (GELF — Graylog 4.0.0 documentation) because it opens up the possibility for powerful analytics of our application.
But I have some doubts about it.
- My understanding is that Elasticsearch will create an index for each additional field. What if the number of additional fields being indexed grows large? Will that cause graylog to stop working properly?
- What if there is a name clash between two additional fields?
To remedy the two problems above, should I try to minimise the number of additional fields? I could just shove them all into the full_messgae field, rather than each being its own field?
I should also mention that the way I have designed the logging in our application is that a programmer can log whatever key value pair he wants to as part of a log message. But this encourages the 2 problems mentioned above. Perhaps this is a bad idea? I am curious as to what other people do…