Hi, my configuration is:
Linux Ubuntu 18.04
Graylog 2.4.4+4659dbe, codename Wildwuchs
Elasticsearch “number” : “5.6.9”
mongodb version v3.4.14
only 1 node
at this point I’ve graylog_9 index generated and it shows 4 shards green.
Changing the number of replicas from 0 to 2 and viceversa
my problem is that everything was fine until 1 month ago but graylog stop processing messages in inputs and i’ve tried almost everything in this site: https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/#reason-1-shard-allocation-is-purposefully-delayed
I’ve already restart elasticsearch service with systemctl, I don’t want to loose index information that still in the memory or journal so I haven’t restarted graylog and mongodb services, only restarted elasticsearch, also elasticsearch and graylog logs don’t show any problem about it.
Also I tried some parameters adjustments to generate slowlog in the index graylog_9 at node level:
curl -XPUT “localhost:9200/_settings?pretty” -H ‘Content-Type: application/json’ -d’{“index.search.slowlog.threshold.query.warn”: “5s”, “index.search.slowlog.threshold.query.info”: “5s”, “index.search.slowlog.threshold.query.debug”: “2s”, “index.search.slowlog.threshold.query.trace”: “500ms”, “index.search.slowlog.threshold.fetch.warn”: “1s”, “index.search.slowlog.threshold.fetch.info”: “800ms”, “index.search.slowlog.threshold.fetch.debug”: “500ms”, “index.search.slowlog.threshold.fetch.trace”: “200ms”, “index.search.slowlog.level”: “debug”, “index.indexing.slowlog.threshold.index.warn”: “5s”}’
curl -XPUT “localhost:9200/_settings?pretty” -H ‘Content-Type: application/json’ -d’{“index.indexing.slowlog.threshold.index.warn”: “10s”, “index.indexing.slowlog.threshold.index.info”: “5s”, “index.indexing.slowlog.threshold.index.debug”: “2s”, “index.indexing.slowlog.threshold.index.trace”: “500ms”}’
curl -XPUT -d ‘{“index.search.slowlog.threshold.query.warn” : “50ms”,“index.search.slowlog.threshold.fetch.warn”: “50ms”,“index.indexing.slowlog.threshold.index.warn”: “50ms”}’ http://127.0.0.1:9200/graylog_9/_settings | jq
and still the same it’s not processing messages to the index, I hope you can help me to solve this, because at some point my node will fail, thanks.