My server is running in ESXi, the host lost communication with graylog over the weekend. After a bit of troubleshooting why I am not seeing any new logs I see I have graylog_deflector for my current shard. what I think i need to do is run the below but I am not sure how to do that, can someone advise on running the below so I can move graylog_deflector files to graylog_105. I am running graylog on Ubuntu 18.4 and I followed the graylog setup on this site.
right now search is not returning anything from the last few days. And I have an err message that says Deflector exists as an index and is not an alias. (triggered 16 hours ago)
The deflector is meant to be an alias but exists as an index. Multiple failures of infrastructure can lead to this. Your messages are still indexed but searches and all maintenance tasks will fail or produce incorrect results. It is strongly recommend that you act as soon as possible.
A few days ago the ESXi host graylog runs on went nuts and lost connection to several servers. I had to run fsck on /ubuntu–vg-root to get the OS to boot. I thought all was good but Monday I checked and search was not working.
I am not able to recalculate date range from the webgui, I get an error. So I am trying to recalculate via ssh running POST http://localhost:9000/api/system/indices/range/graylog_deflector/rebuild and I am getting asked a question that I do not know the answer to
Please enter content (application/x-www-form-urlencoded) to be POSTed:
And my index alias (i think thats what I am looking at) are graylog_### . The one that is failed says this graylog_104 Index does not contain any messages. (764.0B / 0 messages)
The bad index
graylog_deflector Time range of index is unknown, because index range is not available. Please recalculate index ranges manually. (12.8GB / 8,616,142 messages)
What is your elasticsearch cluster status?
If it’s red, don’t do anything on GL web intraface, it won’t work.
First fix elastic, and make the status green.
My elasticsearch status is status is green
curl http://localhost:9200/_cat/indices returns several indicies. green open graylog_deflector yoPb2xQvRP-DmkTCTgwFIw has data, "green open graylog_104 WG9qG27cScCoCbpZjFmNNg " does not have data, can’t i somehow copy data from yoPb2xQvRP-DmkTCTgwFIw to WG9qG27cScCoCbpZjFmNNg and reindex? Or how do I alias yoPb2xQvRP-DmkTCTgwFIw to graylog_105. Graylog_105 is not a current index alias.
I saw that but what I do not how to Reindex the data https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-reindex.html
I am not sure how to run it. Do I make a sh file and set it as executable?
you interact with the Elasticsearch API via REST. So you could use curl to run the command.
That is the reason you find a
copy as curl at the commands bottom.
My Directory gave me the go ahead to delete graylog_deflector. All is working fine now, thanks for you help.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.