License violation due to elasticsearch error?

[timom (Timo Myyrä)]

Hi,

We’ve been running oss graylog for a while and have decided on getting the enterprise license to our new log host.
We have only converted few hosts to use the new log host and today I noticed the clusters daily limits have been exceeded too many times and the processing is disabled.
This is a bit odd as our old log host processes around 1.7Gb daily and it includes some fluff we’re getting rid of. So our estimate would have been that once our hosts are migrated we would be looking at little less than 1gb of daily logs.

Looking at the new clusters System>Enterprise Outgoing traffic it lists zero traffic and then for last 6 days its been doing around 14Gb per day! We had 5 hosts configured to send syslog/eventlog to it so I’m not sure how it gets that amount of traffic.

I’d assume its because our elasticsearch service was missconfigured and wouldn’t run properly. We had issue in our elasticsearch config which was fixed just before the daily outgoing traffic spike occured.
So I guess all the logs got buffered and the graylog started indexing them once elasticsearch was fixed.

Is there a way to control the rate of indexing from graylog to elasticsearch? So in case of these sort of issues it could “catch up” at rate of 3gb/day for example?

How to proceed now that we have violated the license terms? Do we need to purchase new license with increased limits or downgrade to OSS license? Or as we’re still setting up, wipe and re-install with new enterprise license?

[jan (Jan Doberstein)]

he @timom

I noticed the clusters daily limits have been exceeded too many times and the processing is disabled

Graylog does not stop ingest or processing when the license is not valid anymore. Only the enterprise features are disabled - but that should not stop processing at all.

I can only guess how your traffic looks like - but the rule is you can spike 5 times over the limit in 30. If that is under 5 in 30 days the license will be valid again. So you have a sliding window that is looked at for your license.

In Graylog is no way to buffer/lower the ingest from Graylog to Elasticsearch - as you want that to be fast as possible.

You could either wait until the sliding window is within your license limits again and all will work again or you contact sales … http://graylog.org/contact-sales

[timom (Timo Myyrä)]

Ah, well we don’t have the other servers migrated yet so we can easily wait for month to get enterprise features back. Seems the easiest option for us.

Still need to figure out how to prevent these from happening in the future. Better monitoring and fixing elasticsearch issues seems to be the key so it won’t generate huge “backlog” of log events.