# Could not retrieve Elasticsearch cluster health. Fetching Elasticsearch cluster health failed: There was an error fetching a resource: Internal Server Error. Additional information: Couldn't read Elasticsearch cluster health **URL:** \ **Category:** Graylog Central (peer support)\ **Tags:** docker\ **Created:** [January 19, 2023, 10:53am UTC](https://community.graylog.org/t/could-not-retrieve-elasticsearch-cluster-health-fetching-elasticsearch-cluster-health-failed-there-was-an-error-fetching-a-resource-internal-server-error-additional-information-couldnt-read-elasticsearch-cluster-health/27357 "2023-01-19T10:53:33Z")\ **Posts on this page:** 1\ **Showing post:** 4
**Author:** ![macko003](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/macko003/32/3175_2.png) [@macko003](https://community.graylog.org/u/macko003)\ **Post date:** [January 23, 2023, 5:56pm UTC](https://community.graylog.org/t/could-not-retrieve-elasticsearch-cluster-health-fetching-elasticsearch-cluster-health-failed-there-was-an-error-fetching-a-resource-internal-server-error-additional-information-couldnt-read-elasticsearch-cluster-health/27357/4 "2023-01-23T17:56:35Z")
Thanks the help. Ok, I have cloned the system, to do a playground. Empty graylog,opensearch folders. The original mongo data. The error is the same. I recreated again with full empty folders/data. Everything is fine in the system, it shows a green elastic status, and I can recreate active index. So the problem is somewhere in the mongo’s data. So the docker logs: mongo: No error just some like that ```auto {"t":{"$date":"2023-01-23T17:44:31.092+01:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1674492271:92567][1:0x7fde96783700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 1031585, snapshot max: 1031585 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 8741050"}} {"t":{"$date":"2023-01-23T17:45:31.156+01:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1674492331:156534][1:0x7fde96783700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 1031751, snapshot max: 1031751 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 8741050"}} {"t":{"$date":"2023-01-23T17:46:31.212+01:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1674492391:211988][1:0x7fde96783700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 1031923, snapshot max: 1031923 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 8741050"}} ``` opensearch Just info, no errors ```auto [2023-01-23T17:39:55,531][INFO][o.o.j.s.JobSweeper] [graylog-opensearch] Running full sweep [2023-01-23T17:44:55,532][INFO][o.o.j.s.JobSweeper] [graylog-opensearch] Running full sweep [2023-01-23T17:46:29,302][INFO][o.o.c.m.MetadataMappingService] [graylog-opensearch] [graylog_70/l-3UTHBSS8SHOGDMSsCD0A] update_mapping [_doc] [2023-01-23T17:46:29,353][INFO][o.o.a.u.d.DestinationMigrationCoordinator] [graylog-opensearch] Detected cluster change event for destination migration ``` Graylog: Full with errors… ☹ (I treid to pick one-one errors, not the full log) ```auto 2023-01-19 12:02:37,611 WARN : org.graylog2.indexer.indices.Indices - Couldn't create index gl-failures_0. Error: No index template provider found for type 'failures' java.lang.IllegalStateException: No index template provider found for type 'failures' 2023-01-19 12:02:37,615 ERROR: org.graylog2.periodical.IndexRotationThread - Couldn't point deflector to a new index java.lang.RuntimeException: Could not create new target index . 2023-01-19 12:02:39,843 ERROR: org.graylog.events.processor.EventProcessorEngine - Caught an unhandled exception while executing event processor - Make sure to modify the event processor to throw only EventProcessorExecutionException so we get more context! org.graylog2.indexer.IndexNotFoundException: Unable to perform scroll search[graylog_884] Index not found for query: graylog_884. Try recalculating your index ranges. 2023-01-19 12:02:39,896 WARN : org.graylog.plugins.map.geoip.MaxMindIpResolver - Error creating DatabaseReader for 'MaxMindIpAsnResolver' with config file '' 2023-01-19 12:02:39,900 WARN : org.graylog.plugins.map.geoip.MaxMindIpResolver - Error creating DatabaseReader for 'MaxMindIpAsnResolver' with config file '' 2023-01-19 12:02:41,217 ERROR: org.graylog2.indexer.messages.Messages - Caught exception during bulk indexing: ElasticsearchException{message=ElasticsearchException[An error occurred:]; nested: IOException[Unable to parse response body for Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1, host=http://graylog-opensearch:9200, response=HTTP/1.1 200 OK}]; nested: NullPointerException;, errorDetails=[]}, retrying (attempt #1). 2023-01-19 12:02:41,244 ERROR: org.graylog2.indexer.messages.Messages - Caught exception during bulk indexing: ElasticsearchException{message=ElasticsearchException[An error occurred:]; nested: IOException[Unable to parse response body for Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1, host=http://graylog-opensearch:9200, response=HTTP/1.1 200 OK}]; nested: NullPointerException;, errorDetails=[]}, retrying (attempt #2). 2023-01-19 12:02:41,557 ERROR: org.graylog.plugins.threatintel.whois.ip.WhoisIpLookup - Could not lookup WHOIS information for [192.168.1.2] at [ARIN]. 2023-01-19 12:02:41,561 ERROR: org.graylog.plugins.threatintel.whois.ip.WhoisIpLookup - Could not lookup WHOIS information for [192.168.0.15] at [ARIN]. ``` What I tried: - delete the deflectors, It created again, and It write data to elasticsearch. I also can “rotate active write index” - Disable GeoIP resolver, and Threat Int plugins. - Recalculate index range, becaule it doesn’t find the graylog\_884 (I deleted it, so it is normal), but the same error… - create the 884 index. - It solved the search error, I see the data now under the search.AND it solved the cluster state error also 🙂 - create the gl-failures\_0 - recalculate all index sets’ range, it was successfull When I tried to rotate the index set, I get: ```auto 2023-01-23 18:26:15,293 INFO : org.graylog2.rest.resources.system.DeflectorResource - Cycling deflector for index set <61fc1f731bf7eb4ce04d9a7d>. Reason: REST request. 2023-01-23 18:26:15,298 INFO : org.graylog2.indexer.MongoIndexSet - Cycling from to . 2023-01-23 18:26:15,299 INFO : org.graylog2.indexer.MongoIndexSet - Creating target index . 2023-01-23 18:26:15,306 WARN : org.graylog2.indexer.indices.Indices - Couldn't create index gl-failures_1. Error: No index template provider found for type 'failures' java.lang.IllegalStateException: No index template provider found for type 'failures' ``` This is the only error left. I did a new research, and I found this [https://community.graylog.org/t/migration-from-elasticsearch-to-opensearch-gone-wrong/27237](https://community.graylog.org/t/migration-from-elasticsearch-to-opensearch-gone-wrong/27237) @gsmith mentioned a security plugin. I haven’t installed it. BUT the old graylog was an enterprise one (demo, we don’t need the feature). It could be the problem. I can’t delete the Index Set, because an stream connectred to it. Can I delete the stream and the Index set? How? ![kép](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/b/ba63e5999040872a7c4bd2267e91345426802fc4.png) ![kép](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/a/ad6fbcd4ddd1c11852dea38e8c13d5da2ef622aa.png) (I tried to check how can I remove the enterprise feature, but I find only “remove the plugin and restat”) --- _[View the full topic](https://community.graylog.org/t/could-not-retrieve-elasticsearch-cluster-health-fetching-elasticsearch-cluster-health-failed-there-was-an-error-fetching-a-resource-internal-server-error-additional-information-couldnt-read-elasticsearch-cluster-health/27357)._