We’ve been running oss graylog for a while and have decided on getting the enterprise license to our new log host.
We have only converted few hosts to use the new log host and today I noticed the clusters daily limits have been exceeded too many times and the processing is disabled.
This is a bit odd as our old log host processes around 1.7Gb daily and it includes some fluff we’re getting rid of. So our estimate would have been that once our hosts are migrated we would be looking at little less than 1gb of daily logs.
Looking at the new clusters System>Enterprise Outgoing traffic it lists zero traffic and then for last 6 days its been doing around 14Gb per day! We had 5 hosts configured to send syslog/eventlog to it so I’m not sure how it gets that amount of traffic.
I’d assume its because our elasticsearch service was missconfigured and wouldn’t run properly. We had issue in our elasticsearch config which was fixed just before the daily outgoing traffic spike occured.
So I guess all the logs got buffered and the graylog started indexing them once elasticsearch was fixed.
Is there a way to control the rate of indexing from graylog to elasticsearch? So in case of these sort of issues it could “catch up” at rate of 3gb/day for example?
How to proceed now that we have violated the license terms? Do we need to purchase new license with increased limits or downgrade to OSS license? Or as we’re still setting up, wipe and re-install with new enterprise license?