After running pending Ubuntu updates Graylog couldn't connect to ESDB cluster

I ran some pending Ubuntu updates this morning on my Graylog and Elasticsearch cluster Servers after rebooting I could not get Graylog to connect to the Elsdb cluster. I didn’t pay close attention to what the updates were so I can’t definitively list them, however Im pretty sure one was a Elastic DB update. (could have been for log4j)

  • OS Information: Ubuntu Linux 20.04.3 (for both Graylog and Elastic DB Servers)

  • Package Version: Graylog v. 3.3.16+f766a24, codename Sloth Rocket , Elastic v. 6.8

It looks like what happened is after the updates were applied it reset my custom /etc/elasticsearch/jvm.options file to defaults.

It also added a new line #94 -Dlog4j2.formatMsgNoLookups=true

I had to reset the memory values to the custom values I set, for me it was lines 22 and 23.
The new jvm.options file had
Line #22: -Xms1g
Line #23: -Xmx1g

My jvm.options file (These are based of the amount of ram on my server, yours will most likely differ.)
Line #22: -Xms32g
Line #23: -Xmx32g

After rebooting both servers everything started working normally. Hopefully this will help someone else.

1 Like