# Index error after one ES node crashed **URL:** \ **Category:** Graylog Central (peer support)\ **Created:** [October 5, 2021, 4:15am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374 "2021-10-05T04:15:42Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![darrinh](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@darrinh](https://community.graylog.org/u/darrinh)\ **Post date:** [October 5, 2021, 4:15am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/1 "2021-10-05T04:15:42Z")
When clicking on the Search link in graylog, it is now giving the following error: > **While retrieving data for this widget, the following error(s) occurred:** > > - Elasticsearch exception [type=index\_not\_found\_exception, reason=no such index]. This occurred after one of the ES nodes crashed. It was restarted and Graylog now reports: > Elasticsearch cluster is green. Shards: 136 active, 0 initializing, 0 relocating, 0 unassigned, There is a backlog of messages currently that its working through, however the process buffer and the output buffer are both at 100%. There are 3 x Graylog nodes and 3 X ES nodes. Any ideas on what could be the issue? many thanks Darrin ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [October 5, 2021, 4:22am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/2 "2021-10-05T04:22:28Z")
Hello, > [@darrinh](#): > > This occurred after one of the ES nodes crashed. When the ES nodes crash I’m sure that your journal was filling up quick. If the process buffer and the output buffer are both at 100% give Elasticsearch a few minutes to ingest logs from your journal. The amount of time depends on how many messages are left in the journal. EDIT @darrinh What do you see when executing this command? `curl -XGET http://localhost:9200/_cat/indices` What do you see in graylog, Elasticsearch Logs? ---
**Author:** ![darrinh](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@darrinh](https://community.graylog.org/u/darrinh)\ **Post date:** [October 5, 2021, 4:42am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/3 "2021-10-05T04:42:09Z")
Hi there, that query returns the following: > green open graylog\_47 YfHKg0B7QdmybDhtbH4-Bw 4 0 20009885 0 15.7gb 15.7gb > green open graylog\_46 JlOPoAt5RIWubu-cgUqvQg 4 0 20034290 0 15.3gb 15.3gb > green open graylog\_49 NsswIRCrRJOH3B6w6-H2dQ 4 0 20010584 0 14.4gb 14.4gb > green open graylog\_48 9hjMD3vFSgewL-\_PSdzpFA 4 0 20000459 0 15.1gb 15.1gb > green open gl-system-events\_2 BlVfXN\_RTNGcU2DK-bQG5g 4 0 0 0 832b 832b > green open graylog\_50 FVVo22JRQXCBHbfxvsNDiA 4 0 20019164 0 15.3gb 15.3gb > green open gl-system-events\_3 TKR0Iw7aR06I21xYv9Sx6Q 4 0 0 0 832b 832b > green open gl-system-events\_0 0DUHH3kASTKABL7l8szC9w 4 0 0 0 832b 832b > green open graylog\_52 OouRanfrR5-dSRqT5BSszg 4 0 20003684 0 15.5gb 15.5gb > green open graylog\_51 lpSFTKmDQxuIxVmV9zqLjw 4 0 20010395 0 15.2gb 15.2gb > green open gl-system-events\_1 GgiD8fXgRfqR39nnzsQW3A 4 0 0 0 832b 832b > green open gl-system-events\_6 ezkgzHQQSkOX\_mTno1UJug 4 0 0 0 832b 832b > green open graylog\_54 mr\_qLlIERXW905wqNruvAw 4 0 20016804 0 14.5gb 14.5gb > green open graylog\_53 gy3wn5pMReugjjYCw0fFqQ 4 0 20002959 0 14.8gb 14.8gb > green open gl-system-events\_4 Wy4hPLUoQWGORUR20VO6pA 4 0 0 0 832b 832b > green open graylog\_56 5HUYnTqBSFGxjR-YI1AxWg 4 0 20012099 0 15.7gb 15.7gb > green open gl-system-events\_5 tez\_k\_QvTcymoemi3Z5wog 4 0 0 0 832b 832b > green open graylog\_55 MXHD0iM5QAWRFcsGzldYyA 4 0 20015437 0 15.4gb 15.4gb > green open graylog\_58 ov1BRy6-S4Kcs93zYZAN\_g 4 0 20001513 0 14.9gb 14.9gb > green open graylog\_57 \_OUGQ2LTQ2Ckn4tIx3neZg 4 0 20004554 0 15.7gb 15.7gb > green open graylog\_59 Jh057X9KRKerY0tAL27Raw 4 0 20022569 0 14.3gb 14.3gb > green open gl-events\_6 xGu9O0EVQbm\_ZX\_08ebdDg 4 0 0 0 832b 832b > green open gl-events\_1 GMuZeMPGRZqMPxpq-2H2iw 4 0 242 0 131.9kb 131.9kb > green open gl-events\_0 CieUCs6fS\_SoXcGadNUTrg 4 0 24 0 55.9kb 55.9kb > green open gl-events\_5 SLG4ONFmQCKERuOLPE7Bow 4 0 959 0 297.6kb 297.6kb > green open gl-events\_4 ef3nW638Q8yC9hhyeJbQ8w 4 0 8209 0 1.7mb 1.7mb > green open gl-events\_3 X3HsZY81TbOWG0LYMf7oHQ 4 0 10660 0 2.3mb 2.3mb > green open gl-events\_2 OT6SPV9fRcqkUO1sYWdCOQ 4 0 10657 0 2.3mb 2.3mb > green open graylog\_61 mGeWUTlUSaGC8oz4RhriaA 4 0 20010363 0 15.7gb 15.7gb > green open graylog\_60 TP0KzwLrRqaCZXEsldtFeQ 4 0 20015648 0 15.5gb 15.5gb > green open graylog\_63 AR7Hq7nJRsu0CPVxTrfaEQ 4 0 62955669 135 64.4gb 64.4gb > green open graylog\_62 qyZSteyHRaGPkStMokoY\_g 4 0 135143037 64 104.8gb 104.8gb > green open graylog\_45 rjqnwl3STkyXbTw\_3k2Z2g 4 0 20009220 2469 15.5gb 15.5gb > green open graylog\_44 c45ShOTYTAG2FctN2DlUNg 4 0 20012229 0 13.7gb 13.7gb The process and output buffers have gotten back to 0% utilisation now, still getting that same error from the search. many thanks darrin ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [October 5, 2021, 4:43am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/4 "2021-10-05T04:43:56Z")
Do you see anything in ES logs they may point to what index that Graylog cant find. See if there is anything they may pertain to this issue. ---
**Author:** ![darrinh](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@darrinh](https://community.graylog.org/u/darrinh)\ **Post date:** [October 5, 2021, 4:50am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/5 "2021-10-05T04:50:19Z")
I can’t see anything obvious, graylog shows this in the logs: 2021-10-05T15:49:19.588+11:00 ERROR [PivotAggregationSearch] Aggregation search query returned an error: Elasticsearch exception [type=index\_not\_found\_exception, reason=no such index]. If i set the search time to 5 days ago, I can see data, but any time after that gives the no index error. ---
**Author:** ![darrinh](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@darrinh](https://community.graylog.org/u/darrinh)\ **Post date:** [October 5, 2021, 4:57am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/6 "2021-10-05T04:57:59Z")
ok, I did `System > indices > [index name] > maintenance >` on all of the indices and the search function has been restored. thanks for your assistance. cheers darrin ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [October 5, 2021, 4:59am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/7 "2021-10-05T04:59:25Z")
😆 you beat me by seconds, I was going to ask you about that Glad it was resolved. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [October 19, 2021, 4:59am UTC](https://community.graylog.org/t/index-error-after-one-es-node-crashed/21374/8 "2021-10-19T04:59:34Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.