I have 3 nodes in a cluster that was running on Leap 15.1 everything was working. I was tasked to update the servers. I updated to 15.2. Graylog is currently 3.2.4-1 and MongoDB is 6.4. I see Graylog is running on the servers but the web interface is not working. On one of the server node web interface page I get a blank page and on the other two nodes I get “Unable to connect” error message. I am new to Graylog and Linux so I am not sure where I should look to start troubleshooting. Any help would be grateful.
Hello && Welcome
I need to ask a couple questions.
- Do you have a load balancer for your cluster? if so, is the service running?
- Have you check if all the other services on each node like MongoDb and elasticsearch if there functioning?
- Did you check Elasticsearch/ MongoDb/ Graylog log files to see if there were any warnings or errors that pertain to this issue?
- Are you using HTTPS to connect to the Web UI?
- After updating your nodes did you have to reboot?
Normally in a cluster there should be one URL to connect to the web interface (maybe the master node)
Are you able to show your Graylog configuration file?
1 Like
Using HTTPS
Reboot all the nodes several times
I checked the logs for all three and in server.log I see the following errors
2021-07-06T09:28:13.611-05:00 WARN [Errors] The following warnings have been detected: WARNING: Unknown HK2 failure detected:
MultiException stack 1 of 1
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) Error injecting constructor, com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at org.graylog2.users.UserServiceImpl.<init>(UserServiceImpl.java:75)
while locating org.graylog2.users.UserServiceImpl
while locating org.graylog2.shared.users.UserService
1 error
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:226)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1053)
at org.jvnet.hk2.guice.bridge.internal.GuiceServiceHk2Bean.create(GuiceServiceHk2Bean.java:114)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:479)
at org.jvnet.hk2.guice.bridge.internal.GuiceScopeContext.findOrCreate(GuiceScopeContext.java:75)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2022)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:682)
at org.jvnet.hk2.internal.ThreeThirtyResolver.resolve(ThreeThirtyResolver.java:78)
at org.jvnet.hk2.internal.ClazzCreator.resolve(ClazzCreator.java:212)
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:235)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:358)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:487)
at org.glassfish.jersey.process.internal.RequestScope.findOrCreate(RequestScope.java:162)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2022)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:774)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:737)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:707)
at org.glassfish.jersey.internal.inject.Injections.getOrCreate(Injections.java:172)
at org.glassfish.jersey.server.model.MethodHandler$ClassBasedMethodHandler.getInstance(MethodHandler.java:284)
at org.glassfish.jersey.server.internal.routing.PushMethodHandlerRouter.apply(PushMethodHandlerRouter.java:74)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:109)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:92)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:61)
at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:318)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384)
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at com.mongodb.connection.SocketStream.read(SocketStream.java:87)
at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:547)
at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:418)
at com.mongodb.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:290)
at com.mongodb.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
at com.mongodb.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:98)
at com.mongodb.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:441)
at com.mongodb.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:80)
at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:189)
at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:264)
at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:126)
at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:118)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:226)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:217)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:154)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:147)
at com.mongodb.operation.CreateIndexesOperation$1.call(CreateIndexesOperation.java:173)
at com.mongodb.operation.CreateIndexesOperation$1.call(CreateIndexesOperation.java:168)
at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:462)
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:424)
at com.mongodb.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:168)
at com.mongodb.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:69)
at com.mongodb.Mongo$3.execute(Mongo.java:837)
at com.mongodb.Mongo$3.execute(Mongo.java:818)
at com.mongodb.DBCollection.createIndex(DBCollection.java:1623)
at com.mongodb.DBCollection.createIndex(DBCollection.java:1608)
at com.mongodb.DBCollection.createIndex(DBCollection.java:1566)
at org.graylog2.users.UserServiceImpl.<init>(UserServiceImpl.java:84)
at org.graylog2.users.UserServiceImpl$$FastClassByGuice$$45abaa11.newInstance(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050)
... 40 more
WARNING: Unknown HK2 failure detected:
MultiException stack 1 of 3
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) Error injecting constructor, com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at org.graylog2.users.UserServiceImpl.<init>(UserServiceImpl.java:75)
while locating org.graylog2.users.UserServiceImpl
while locating org.graylog2.shared.users.UserService
1 error
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:226)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1053)
at org.jvnet.hk2.guice.bridge.internal.GuiceServiceHk2Bean.create(GuiceServiceHk2Bean.java:114)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:479)
at org.jvnet.hk2.guice.bridge.internal.GuiceScopeContext.findOrCreate(GuiceScopeContext.java:75)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2022)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:682)
at org.jvnet.hk2.internal.ThreeThirtyResolver.resolve(ThreeThirtyResolver.java:78)
at org.jvnet.hk2.internal.ClazzCreator.resolve(ClazzCreator.java:212)
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:235)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:358)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:487)
at org.glassfish.jersey.process.internal.RequestScope.findOrCreate(RequestScope.java:162)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2022)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:774)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:737)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:707)
at org.glassfish.jersey.internal.inject.Injections.getOrCreate(Injections.java:172)
at org.glassfish.jersey.server.model.MethodHandler$ClassBasedMethodHandler.getInstance(MethodHandler.java:284)
at org.glassfish.jersey.server.internal.routing.PushMethodHandlerRouter.apply(PushMethodHandlerRouter.java:74)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:109)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:92)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:61)
at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:318)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384)
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at com.mongodb.connection.SocketStream.read(SocketStream.java:87)
at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:547)
at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:418)
at com.mongodb.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:290)
at com.mongodb.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
at com.mongodb.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:98)
at com.mongodb.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:441)
at com.mongodb.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:80)
at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:189)
at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:264)
at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:126)
at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:118)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:226)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:217)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:154)
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:147)
at com.mongodb.operation.CreateIndexesOperation$1.call(CreateIndexesOperation.java:173)
at com.mongodb.operation.CreateIndexesOperation$1.call(CreateIndexesOperation.java:168)
at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:462)
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:424)
at com.mongodb.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:168)
at com.mongodb.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:69)
at com.mongodb.Mongo$3.execute(Mongo.java:837)
at com.mongodb.Mongo$3.execute(Mongo.java:818)
at com.mongodb.DBCollection.createIndex(DBCollection.java:1623)
at com.mongodb.DBCollection.createIndex(DBCollection.java:1608)
at com.mongodb.DBCollection.createIndex(DBCollection.java:1566)
at org.graylog2.users.UserServiceImpl.<init>(UserServiceImpl.java:84)
at org.graylog2.users.UserServiceImpl$$FastClassByGuice$$45abaa11.newInstance(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050)
... 40 more
MultiException stack 2 of 3
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of org.graylog2.shared.rest.resources.system.LoadBalancerStatusResource errors were found
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:247)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:358)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:487)
at org.glassfish.jersey.process.internal.RequestScope.findOrCreate(RequestScope.java:162)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2022)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:774)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:737)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:707)
at org.glassfish.jersey.internal.inject.Injections.getOrCreate(Injections.java:172)
at org.glassfish.jersey.server.model.MethodHandler$ClassBasedMethodHandler.getInstance(MethodHandler.java:284)
at org.glassfish.jersey.server.internal.routing.PushMethodHandlerRouter.apply(PushMethodHandlerRouter.java:74)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:109)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:92)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:61)
at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:318)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384)
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
MultiException stack 3 of 3
java.lang.IllegalStateException: Unable to perform operation: resolve on org.graylog2.shared.rest.resources.system.LoadBalancerStatusResource
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:387)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:487)
at org.glassfish.jersey.process.internal.RequestScope.findOrCreate(RequestScope.java:162)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2022)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:774)
at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetService(ServiceLocatorImpl.java:737)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:707)
at org.glassfish.jersey.internal.inject.Injections.getOrCreate(Injections.java:172)
at org.glassfish.jersey.server.model.MethodHandler$ClassBasedMethodHandler.getInstance(MethodHandler.java:284)
at org.glassfish.jersey.server.internal.routing.PushMethodHandlerRouter.apply(PushMethodHandlerRouter.java:74)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:109)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage._apply(RoutingStage.java:112)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:92)
at org.glassfish.jersey.server.internal.routing.RoutingStage.apply(RoutingStage.java:61)
at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:318)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384)
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-07-06T09:28:30.813-05:00 ERROR [EventProcessorEngine] Caught an unhandled exception while executing event processor <aggregation-v1/ERROR and CRITICAL/5e78f879bf72855e50bf7b47> - Make sure to modify the event processor to throw only EventProcessorExecutionException so we get more context!
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) Error injecting constructor, java.lang.RuntimeException: Couldn't get node ID
at org.graylog.events.event.EventProcessorEventFactory.<init>(EventProcessorEventFactory.java:34)
while locating org.graylog.events.event.EventProcessorEventFactory
1 error
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:226) ~[graylog.jar:?]
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1053) ~[graylog.jar:?]
at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?]
at org.graylog.events.processor.EventProcessorExecutionJob.execute(EventProcessorExecutionJob.java:111) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.executeJob(JobExecutionEngine.java:166) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.lambda$handleTrigger$2(JobExecutionEngine.java:144) ~[graylog.jar:?]
at com.codahale.metrics.Timer.time(Timer.java:137) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.handleTrigger(JobExecutionEngine.java:144) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.lambda$execute$0(JobExecutionEngine.java:119) ~[graylog.jar:?]
at org.graylog.scheduler.worker.JobWorkerPool.lambda$execute$0(JobWorkerPool.java:110) ~[graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: java.lang.RuntimeException: Couldn't get node ID
at org.graylog.events.event.EventProcessorEventFactory.<init>(EventProcessorEventFactory.java:40) ~[graylog.jar:?]
at org.graylog.events.event.EventProcessorEventFactory$$FastClassByGuice$$cf23907a.newInstance(<generated>) ~[graylog.jar:?]
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306) ~[graylog.jar:?]
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050) ~[graylog.jar:?]
... 13 more
Caused by: org.graylog2.cluster.NodeNotFoundException: Unable to find node 7b556aac-8328-49b1-a3f9-237bbdc7afc1
at org.graylog2.cluster.NodeServiceImpl.byNodeId(NodeServiceImpl.java:76) ~[graylog.jar:?]
at org.graylog2.cluster.NodeServiceImpl.byNodeId(NodeServiceImpl.java:84) ~[graylog.jar:?]
at org.graylog.events.event.EventProcessorEventFactory.<init>(EventProcessorEventFactory.java:38) ~[graylog.jar:?]
at org.graylog.events.event.EventProcessorEventFactory$$FastClassByGuice$$cf23907a.newInstance(<generated>) ~[graylog.jar:?]
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306) ~[graylog.jar:?]
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050) ~[graylog.jar:?]
... 13 more
2021-07-06T09:28:30.820-05:00 ERROR [EventProcessorExecutionJob] Event processor <aggregation-v1/5f724e32bf728504d0d91d49> failed to execute: Couldn't create events for: EventDefinitionDto{id=5f724e32bf728504d0d91d49, title=ERROR and CRITICAL, description=Migrated message count alert condition, priority=2, alert=false, config=AggregationEventProcessorConfig{type=aggregation-v1, query=level:"2", queryParameters=[], streams=[5f342b9ad5723b06f337d1ba], groupBy=[], series=[AggregationSeries{id=e303ac8f-cdf7-4018-9088-796e2ce68f4a, function=COUNT, field=Optional.empty}], conditions=Optional[AggregationConditions{expression=Optional[Greater{expr=>, left=NumberReference{expr=number-ref, ref=e303ac8f-cdf7-4018-9088-796e2ce68f4a}, right=NumberValue{expr=number, value=0.0}}]}], searchWithinMs=60000, executeEveryMs=60000}, fieldSpec={}, keySpec=[], notificationSettings=EventNotificationSettings{gracePeriodMs=0, backlogSize=0}, notifications=[], storage=[Config{type=persist-to-streams-v1, streams=[000000000000000000000002]}]} (retry in 5000 ms)
org.graylog.events.processor.EventProcessorException: Couldn't create events for: EventDefinitionDto{id=5f724e32bf728504d0d91d49, title=ERROR and CRITICAL, description=Migrated message count alert condition, priority=2, alert=false, config=AggregationEventProcessorConfig{type=aggregation-v1, query=level:"2", queryParameters=[], streams=[5f342b9ad5723b06f337d1ba], groupBy=[], series=[AggregationSeries{id=e303ac8f-cdf7-4018-9088-796e2ce68f4a, function=COUNT, field=Optional.empty}], conditions=Optional[AggregationConditions{expression=Optional[Greater{expr=>, left=NumberReference{expr=number-ref, ref=e303ac8f-cdf7-4018-9088-796e2ce68f4a}, right=NumberValue{expr=number, value=0.0}}]}], searchWithinMs=60000, executeEveryMs=60000}, fieldSpec={}, keySpec=[], notificationSettings=EventNotificationSettings{gracePeriodMs=0, backlogSize=0}, notifications=[], storage=[Config{type=persist-to-streams-v1, streams=[000000000000000000000002]}]}
at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:106) ~[graylog.jar:?]
at org.graylog.events.processor.EventProcessorExecutionJob.execute(EventProcessorExecutionJob.java:111) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.executeJob(JobExecutionEngine.java:166) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.lambda$handleTrigger$2(JobExecutionEngine.java:144) ~[graylog.jar:?]
at com.codahale.metrics.Timer.time(Timer.java:137) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.handleTrigger(JobExecutionEngine.java:144) ~[graylog.jar:?]
at org.graylog.scheduler.JobExecutionEngine.lambda$execute$0(JobExecutionEngine.java:119) ~[graylog.jar:?]
at org.graylog.scheduler.worker.JobWorkerPool.lambda$execute$0(JobWorkerPool.java:110) ~[graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) Error injecting constructor, java.lang.RuntimeException: Couldn't get node ID
at org.graylog.events.event.EventProcessorEventFactory.<init>(EventProcessorEventFactory.java:34)
while locating org.graylog.events.event.EventProcessorEventFactory
1 error
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:226) ~[graylog.jar:?]
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1053) ~[graylog.jar:?]
at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?]
... 12 more
Caused by: java.lang.RuntimeException: Couldn't get node ID
at org.graylog.events.event.EventProcessorEventFactory.<init>(EventProcessorEventFactory.java:40) ~[graylog.jar:?]
at org.graylog.events.event.EventProcessorEventFactory$$FastClassByGuice$$cf23907a.newInstance(<generated>) ~[graylog.jar:?]
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306) ~[graylog.jar:?]
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050) ~[graylog.jar:?]
at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?]
... 12 more
Caused by: org.graylog2.cluster.NodeNotFoundException: Unable to find node 7b556aac-8328-49b1-a3f9-237bbdc7afc1
at org.graylog2.cluster.NodeServiceImpl.byNodeId(NodeServiceImpl.java:76) ~[graylog.jar:?]
at org.graylog2.cluster.NodeServiceImpl.byNodeId(NodeServiceImpl.java:84) ~[graylog.jar:?]
at org.graylog.events.event.EventProcessorEventFactory.<init>(EventProcessorEventFactory.java:38) ~[graylog.jar:?]
at org.graylog.events.event.EventProcessorEventFactory$$FastClassByGuice$$cf23907a.newInstance(<generated>) ~[graylog.jar:?]
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91) ~[graylog.jar:?]
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306) ~[graylog.jar:?]
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050) ~[graylog.jar:?]
at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?]
... 12 more
It looks like your graylog instance isn’t able to connect to Mongodb. Can you confirm that Mongodb is up and running and you’re able to make a TCP connection to it over port 27017?
1 Like
It is up and running. Can you tell me how to confirm the last part? New to linux. I was able to get the web instance up but I see this
So I am wanted to know how to fix this issue.
Hello
Looks like you have 1211 unassigned shards.
You can execute a couple of commands that will help you. The examples below is configured for “Localhost” if yours is different just replace “localhost” with your configuration.
Check Shards This will show you which ones are unassigned.
curl -XGET http://localhost:9200/_cat/shards
ES Shard Info (shows the reason)
curl -XGET http://localhost:9200/_cluster/allocation/explain?pretty
Here is some information for you.
Hope that helps
Thanks for the link. This still doesn’t help me. Is there support I can call so they can walk me through it?
We do offer paid support. If you’d like, I can have one of our folks reach out to you.
1 Like
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.