[2.4.0, 2.4.1] Graylog hangs at startup

(Giacomo "Mr Wolf" Furlan) #1


I’ve been experiencing this issue before, and I “resolved” returning to a previous state of the data (mongo/elasticsearch/graylog).

This is happening using the official Docker image. Analysing the Graylog’s log, it hangs without any warning after “INFO : org.graylog2.bindings.providers.RulesEngineProvider - No static rules file loaded.”… No inputs are initialised, no web api, so I don’t really know where to look now.

May you please help me?
This is the full log: https://pastebin.com/PTLjFW4j

(Jochen) #2

Please provide the complete configuration of Graylog (also docker-compose.yml if you’re using Docker Compose).

(Giacomo "Mr Wolf" Furlan) #3

Hello jochen,

here is the docker-compose configuration:


As per the configuration file, is the one which comes with the official docker image (image id: f93945a7d67c).

Thank you very much :slight_smile:

I can enter and interact with the mongo database, and the elasticsearch’s health API reports the status green.

(Jochen) #4

What are the hardware specs of the machine running these Docker containers?

(Giacomo "Mr Wolf" Furlan) #5

It’s an Amazon c5.2xlarge instance (8 vCPU, 16 GB of RAM, running with a 300GB SSD). Running a new instance of the graylog stack via docker-compose it works, but obviously I’d loose the previous data and, worst thing, it’d have the same problem again in the future.

(Jochen) #6

Try starting Graylog in debug mode by setting the environment variable GRAYLOG_SERVER_OPTS to --debug.

(Giacomo "Mr Wolf" Furlan) #7

It keeps repeating the following log (again and again):

graylog_1        | 2018-01-23 14:40:18,581 DEBUG: org.mongodb.driver.protocol.command - Sending command {find : BsonString{value='roles'}} to database graylog on connection [connectionId{localValue:2, serverValue:4}] to server mongo:27017
graylog_1        | 2018-01-23 14:40:18,582 DEBUG: org.mongodb.driver.protocol.command - Command execution completed

(Jochen) #8

Please post the complete log and not just some arbitrary lines.

(Giacomo "Mr Wolf" Furlan) #9

Here it is. After those lines it will continue spamming the two lines above.


Sorry, it was capped to ~6000 lines

(Giacomo "Mr Wolf" Furlan) #10

As an additional information, I can confirm that the broken docker-composed environment has the very same (mongo) graylog.roles collection as a newly created one… if of any interest in debugging…

(Jochen) #11

That’s a red herring. The “roles” collection is read each second by default.

(Giacomo "Mr Wolf" Furlan) #12

Do you see anything particular in the given log?

(Giacomo "Mr Wolf" Furlan) #13

Playing around with the mongo collections, I wiped the pipeline ones and graylog started working again… strange, also because I don’t really know why it wasn’t logged anywhere…

(Jochen) #14

Do you happen to have a backup of the pipeline-related collections (or better: the complete MongoDB database) so we could try to reproduce the problem?

(Giacomo "Mr Wolf" Furlan) #15

I could try recovering it. But I think I can recover the pipeline now:

rule "Drop unsupported browsers errors"
    AND lookup_value("unsupported-browsers", $message.user_agent_browser_name_and_version, "0") == "1"

Where “unsupported-browsers” is a cached CSV lookup table

(system) closed #16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.