I ran the Elasticsearch Migration Helper and it came back reporting that about half of my indexes have too many fields. Apparently ES v5.0 has a limit of 1000 fields, and my indexes use on average 1000-1300… ?
I’m learning as I go… My goal is to get us to Graylog v3 with Elasticsearch v6, preferably with all our indexes still accessible.
Anyone else run into this? Any suggestions?
change the setting for that limit:
https://www.graylog.org/post/what-to-do-when-you-have-1000-fields
Thanks Jan, it seems like this is something to avoid if possible. I already wasn’t happy with the performance of ES/GL when doing large searches, I imagine it will only make it worse if I change the field limit to 1350 to accommodate all my old indexes.
How do I prevent this in the future though? The link you provided mentioned being able to send different inputs into different indexes, but it didn’t say how to do it… Can you point me in the right direction?
you actually create multiple multiple index sets ( http://docs.graylog.org/en/3.0/pages/configuration/index_model.html ) and route different streams ( http://docs.graylog.org/en/3.0/pages/streams.html ) into different indices.
It will not harm you, raising the max fields to 1350 … I guess the number 1000 is out of the blue.
The article above shows how to adjust an existing index. Are there any instructions for setting future indices to a larger field limit?
should I really google that for you?
Well, no I wouldn’t expect that. I thought maybe you or someone would be able to point me generally in the right direction. I have searched for an answer, and perhaps it’s my lack of Elasticsearch experience, but I either haven’t found a clear answer or I’ve looked past one out of ignorance.
I’ll search some more and add the answer here once I find it.
sorry that I might sound rude - I forgot that you might not know HOW to set that…
You should use index templates/mappings
http://docs.graylog.org/en/3.0/pages/configuration/elasticsearch.html#custom-index-mappings
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/indices-templates.html
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
