Hello everyone.
I stood up a 3 server graylog cluster and a 4 server elasticsearch cluster back in November. Things have been functioning fine this whole time taking in around 20 gigs a day.
Now recently the messages are not processing anymore and Graylog has 16 million logs in the journal unprocessed.
I can see that the elasticsearch servers are using almost no CPU, so I know they are not working hard, but I can not for the life of me figure out why. The logs are not returning any obvious problems. The cluster is green, there is no complaints about the shards being unassigned or corrupted. Yet it is just not working.
Please I hope someone has some advice on the best way to handle this.