grayog web interface was receiving 502 bad gateway issues. When I checked I found My ELasticache in red state.
{"cluster_name":"graylog","status":"red","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":4,"unassigned_shards":24,"delayed_unassigned_shards":0,"number_of_pending_tasks":7,"number_of_in_flight_fetch":1,"task_max_waiting_in_queue_millis":11,"active_shards_percent_as_number":0.0|
Output of sudo curl -XGET localhost:9200/_cat/shards
:
graylog_6 3 p UNASSIGNED
graylog_6 2 p UNASSIGNED
graylog_6 1 p INITIALIZING 127.0.0.1 Chronos
graylog_6 0 p INITIALIZING 127.0.0.1 Chronos
graylog_1 1 p UNASSIGNED
graylog_1 2 p UNASSIGNED
graylog_1 3 p UNASSIGNED
graylog_1 0 p UNASSIGNED
graylog_0 3 p UNASSIGNED
graylog_0 2 p UNASSIGNED
graylog_0 1 p UNASSIGNED
graylog_0 0 p UNASSIGNED
graylog_5 3 p UNASSIGNED
graylog_5 2 p INITIALIZING 127.0.0.1 Chronos
graylog_5 1 p INITIALIZING 127.0.0.1 Chronos
graylog_5 0 p UNASSIGNED
graylog_4 1 p UNASSIGNED
graylog_4 2 p UNASSIGNED
graylog_4 3 p UNASSIGNED
graylog_4 0 p UNASSIGNED
graylog_3 3 p UNASSIGNED
graylog_3 2 p UNASSIGNED
graylog_3 1 p UNASSIGNED
graylog_3 0 p UNASSIGNED
graylog_2 3 p UNASSIGNED
graylog_2 2 p UNASSIGNED
graylog_2 1 p UNASSIGNED
graylog_2 0 p UNASSIGNED
curl http://localhost:9200/_stats/index,store?pretty
{
"_shards" : {
"total" : 28,
"successful" : 1,
"failed" : 0
},
"_all" : {
"primaries" : {
"store" : {
"size_in_bytes" : 354842957,
"throttle_time_in_millis" : 0
}
},
"total" : {
"store" : {
"size_in_bytes" : 354842957,
"throttle_time_in_millis" : 0
}
}
},
"indices" : {
"graylog_6" : {
"primaries" : {
"store" : {
"size_in_bytes" : 354842957,
"throttle_time_in_millis" : 0
}
},
"total" : {
"store" : {
"size_in_bytes" : 354842957,
"throttle_time_in_millis" : 0
}
}
}
}
}
curl 'localhost:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
red open graylog_6 4 0
red open graylog_1 4 0
red open graylog_0 4 0
red open graylog_5 4 0
red open graylog_4 4 0
red open graylog_3 4 0
red open graylog_2 4 0
40M ./graylog_0
6.1G ./graylog_1
40G ./graylog_2
21G ./graylog_3
220K ./graylog_4
2.3M ./graylog_5
3.3G ./graylog_6
70G total
What should I do.How to fix disk out of space.The I have only 75GB in total.How can I make server up.Do i need to delete all the indexes.What would happen if the indexes are deleted. Should I add more diskspace.I am very beginner with graylog, Is it safe to delete some indexes