It’s a free space issue … i’ve resized the graylog volume and all works fine!
Hi, today my installation have some problem … graylog become unreachable suddenly,
this is the log of garylog:
Blockquote
2018-07-19T18:07:34.148+02:00 INFO [cluster] No server chosen by WritableServerSelector from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]}. Waiting for 30000 ms before timing out
2018-07-19T18:07:34.176+02:00 WARN [Errors] The following warnings have been detected: WARNING: Unknown HK2 failure detected:
MultiException stack 1 of 1
com.google.inject.ProvisionException: Unable to provision, see the following errors:
- Error injecting constructor, com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
at org.graylog2.users.UserServiceImpl.(UserServiceImpl.java:67)
while locating org.graylog2.users.UserServiceImpl
while locating org.graylog2.shared.users.UserService
1 error
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028)
at org.jvnet.hk2.guice.bridge.internal.GuiceServiceHk2Bean.create(GuiceServiceHk2Bean.java:114)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:479)
and this is the journalctl otput after try to restart mongo
Blockquote
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:199)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:199)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:180)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at com.google.inject.Guice.createInjector(Guice.java:99)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at org.graylog2.shared.bindings.GuiceInjectorHolder.createInjector(GuiceInjectorHolder.java:34)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at org.graylog2.bootstrap.CmdLineTool.setupInjector(CmdLineTool.java:379)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:193)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at org.graylog2.bootstrap.Main.main(Main.java:44)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: Caused by: java.io.IOException: No space left on device
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at java.io.FileOutputStream.writeBytes(Native Method)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at java.io.FileOutputStream.write(FileOutputStream.java:326)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: at org.apache.logging.log4j.core.appender.OutputStreamManager.writeToDestination(OutputStreamManager.java:262)
Jul 19 18:16:22 localhost.localdomain graylog-server[970]: … 122 more
Jul 19 18:17:55 localhost.localdomain polkitd[644]: Registered Authentication Agent for unix-process:1509:58402 (system bus name :1.20 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object
Jul 19 18:17:55 localhost.localdomain systemd[1]: Starting MongoDB Database Server…
– Subject: Unit mongod.service has begun start-up
– Defined-By: systemd
– Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
–
– Unit mongod.service has begun starting up.
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ resumed (module ‘builtin:omfile’) [v8.24.0 try You searched for error 2359 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain rsyslogd[972]: action ‘action 0’ suspended, next retry is Thu Jul 19 18:18:25 2018 [v8.24.0 try You searched for error 2007 - rsyslog ]
Jul 19 18:17:55 localhost.localdomain mongod[1524]: 2018-07-19T18:17:55.890+0200 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'no
Jul 19 18:17:55 localhost.localdomain mongod[1524]: about to fork child process, waiting until server is ready for connections.
Jul 19 18:17:55 localhost.localdomain mongod[1524]: forked process: 1527
Jul 19 18:17:55 localhost.localdomain mongod[1524]: ERROR: child process failed, exited with error number 1
Jul 19 18:17:55 localhost.localdomain mongod[1524]: To see additional information in this output, start without the “–fork” option.
Jul 19 18:17:55 localhost.localdomain systemd[1]: mongod.service: control process exited, code=exited status=1
Jul 19 18:17:55 localhost.localdomain systemd[1]: Failed to start MongoDB Database Server.
– Subject: Unit mongod.service has failed
– Defined-By: systemd
– Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
–
– Unit mongod.service has failed.
–
– The result is failed.
Jul 19 18:17:55 localhost.localdomain systemd[1]: Unit mongod.service entered failed state.
Jul 19 18:17:55 localhost.localdomain systemd[1]: mongod.service failed.
Jul 19 18:17:55 localhost.localdomain polkitd[644]: Unregistered Authentication Agent for unix-process:1509:58402 (system bus name :1.20, object path /org/freedesktop/PolicyKit1/Authenticati
if i restart entire machine all work fine. any idea?