Hello,
How often Graylog create new indices? As of right now, we do have 3 indices.
I know that it’s important to have indices. But, is it possible to set up Graylog not to create new indices? I want it to stop creating new indices. My goal here is to forward the Elasticsearch log to SQL server. Using only 1 table makes it easier to do that.
Thanks for your help.
you setup your indices in the UI System > Indices that is the place to configure when the rollover should happen. ( http://docs.graylog.org/en/2.4/pages/configuration/index_model.html#index-rotation )
My advice would be to not make to large indices because this is the only option to drop data and handling big indices is more painfull than having small one.
https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster
Small shards result in small segments, which increases overhead. Aim to keep the average shard size between a few GB and a few tens of GB. For use-cases with time-based data, it is common to see shards between 20GB and 40GB in size.
Elasticsearch is better for store and search logs. Why would like to forward it to SQL?
Graylog won’t delete logs from elasticsearch, so if you want only one index you need to keep all logs for forever.
You also can search in ES without the index name, so I think it couldn’t be a limit of your goal.
Thanks for the input. I needed to forward it to SQL so I can create a good report using Power BI. Power BI supports Elasticsearch but not with the refresh. We want it to continuously refresh so that latest data will be available.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
