I ran some pending Ubuntu updates this morning on my Graylog and Elasticsearch cluster Servers after rebooting I could not get Graylog to connect to the Elsdb cluster. I didn’t pay close attention to what the updates were so I can’t definitively list them, however Im pretty sure one was a Elastic DB update. (could have been for log4j)
-
OS Information: Ubuntu Linux 20.04.3 (for both Graylog and Elastic DB Servers)
-
Package Version: Graylog v. 3.3.16+f766a24, codename Sloth Rocket , Elastic v. 6.8
It looks like what happened is after the updates were applied it reset my custom /etc/elasticsearch/jvm.options file to defaults.
It also added a new line #94 -Dlog4j2.formatMsgNoLookups=true
I had to reset the memory values to the custom values I set, for me it was lines 22 and 23.
The new jvm.options file had
Line #22: -Xms1g
Line #23: -Xmx1g
My jvm.options file (These are based of the amount of ram on my server, yours will most likely differ.)
Line #22: -Xms32g
Line #23: -Xmx32g
After rebooting both servers everything started working normally. Hopefully this will help someone else.