Upgrade Error from 3.1.4 to 3.2.0.b3

Graylog Central (peer support)
1

I had upgraded my Graylog from 3.1.4 to 3.2.0.b3 can’t see Graylog view management OR Graylog Dashboard management like: 192.168.0.1:9000/views ou 192.168.0.1:9000/dashboards

I got some errors on API Browser like:

http://192.168.0.1:9000/api/views?page=1&per_page=5&sort=title

Response Body:

{
  "type": "ApiError",
  "message": "IOException encountered while reading from a byte array input stream"
}

Response Code:

500

Response Headers:

{"Connection":"close","Content-Length":"100","Content-Type":"application/json","Date":"Tue, 21 Jan 2020 13:39:27 GMT","X-Graylog-Node-Id":"5f7e2f30-363e-4977-b6fc-047a3c960a2d"}

Error log like:

2020-01-21 10:26:37,239 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource

java.lang.RuntimeException: IOException encountered while reading from a byte array input stream

at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:67) ~[graylog.jar:?]

at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:49) ~[graylog.jar:?]

at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:29) ~[graylog.jar:?]

at com.mongodb.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52) ~[graylog.jar:?]

at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60) ~[graylog.jar:?]

at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]

at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:41) ~[graylog.jar:?]

at org.bson.codecs.configuration.LazyCodec.decode(LazyCodec.java:47) ~[graylog.jar:?]

at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:101) ~[graylog.jar:?]

at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63) ~[graylog.jar:?]

at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]

at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:41) ~[graylog.jar:?]

at com.mongodb.connection.ReplyMessage.<init>(ReplyMessage.java:51) ~[graylog.jar:?]

at com.mongodb.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:301) ~[graylog.jar:?]

at com.mongodb.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255) ~[graylog.jar:?]

at com.mongodb.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:98) ~[graylog.jar:?]

at com.mongodb.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:441) ~[graylog.jar:?]

at com.mongodb.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:80) ~[graylog.jar:?]

at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:189) ~[graylog.jar:?]

at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:264) ~[graylog.jar:?]

at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:126) ~[graylog.jar:?]

at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:118) ~[graylog.jar:?]

at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:226) ~[graylog.jar:?]

at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:217) ~[graylog.jar:?]

at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:120) ~[graylog.jar:?]

at com.mongodb.operation.FindOperation$1.call(FindOperation.java:717) ~[graylog.jar:?]

at com.mongodb.operation.FindOperation$1.call(FindOperation.java:711) ~[graylog.jar:?]

at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:471) ~[graylog.jar:?]

at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:415) ~[graylog.jar:?]

at com.mongodb.operation.FindOperation.execute(FindOperation.java:711) ~[graylog.jar:?]

at com.mongodb.operation.FindOperation.execute(FindOperation.java:83) ~[graylog.jar:?]

at com.mongodb.Mongo$3.execute(Mongo.java:826) ~[graylog.jar:?]

at com.mongodb.Mongo$3.execute(Mongo.java:813) ~[graylog.jar:?]

at com.mongodb.DBCursor.initializeCursor(DBCursor.java:877) ~[graylog.jar:?]

at com.mongodb.DBCursor.hasNext(DBCursor.java:144) ~[graylog.jar:?]

at org.mongojack.DBCursor.hasNext(DBCursor.java:330) ~[graylog.jar:?]

at java.util.Iterator.forEachRemaining(Iterator.java:115) ~[?:1.8.0_181]

at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[?:1.8.0_181]

at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_181]

at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_181]

at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_181]

at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_181]

at java.util.stream.LongPipeline.reduce(LongPipeline.java:438) ~[?:1.8.0_181]

at java.util.stream.LongPipeline.sum(LongPipeline.java:396) ~[?:1.8.0_181]

at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:526) ~[?:1.8.0_181]

at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSort(PaginatedDbService.java:158) ~[graylog.jar:?]

at org.graylog.plugins.views.search.views.ViewService.searchPaginated(ViewService.java:64) ~[graylog.jar:?]

at org.graylog.plugins.views.search.views.ViewService.searchPaginatedByType(ViewService.java:86) ~[graylog.jar:?]

at org.graylog.plugins.views.search.rest.DashboardsResource.views(DashboardsResource.java:88) ~[graylog.jar:?]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_181]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]

at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]

at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]

at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]

at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]

at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]

at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]

at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]

at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]

at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]

at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]

at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]

at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]

at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]

at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "dashboard_state" (class org.graylog.plugins.views.search.views.$AutoValue_ViewDTO$Builder), not marked as ignorable (11 known properties: "state", "search_id", "owner", "summary", "title", "type", "description", "requires", "properties", "_id", "created_at"])

 at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@2fb1d2c0; pos: 1283] (through reference chain: org.graylog.plugins.views.search.views.$AutoValue_ViewDTO$Builder["dashboard_state"])

at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:823) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1153) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1589) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1567) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:274) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:193) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3985) ~[graylog.jar:?]

at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2343) ~[graylog.jar:?]

at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]

at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]

I can access views or dashboards from direct link, like:

http://192.168.0.1/dashboards/asdasd12312117e34e62sdasd1
http://192.168.0.1/views/123das12312117e3sdasd14e62

But I can’t list all dashboards or views.

I tryed to go back to Graylog 3.1.4 and http://192.168.0.1/dashboards now is working, but http://192.168.0.1/views doesn’t working and I can’t list my views.

2

Try the upgrade path cited in the documentation:

$ wget https://packages.graylog2.org/repo/packages/graylog-3.1-repository_latest.deb
$ sudo dpkg -i graylog-3.1-repository_latest.deb
$ sudo apt-get update
$ sudo apt-get install graylog-server
3

Hello Joseph!

Thank you for your answer, but I’m not using Linux, I’m using FreeBSD.

4

he @chalfling

downgrade is nothing that the Graylog Database supports - you have the risk of loosing data.

The 3.2 beta is still a beta … so it might be that you run into issues. In addition to that some REST data endpoints will/have changed compared to 3.1.

As you are back to 3.1 now debugging is not possible anymore.

5

Hello @jan

I went back to 3.1 because all my dashboard goes missing. I think the concept of dashboard and view merged on 3.2.

I can go forward again to 3.2 beta, but I want to know how to upgrade all my olds Dashboard into the new view concept or can use both (old dashboards/views).

Can I run some command to fix Graylog Database to list views in my 3.1 or list my Dashboards on 3.2.

Thanks!

6

he @chalfling

we provide an upgrade path, that include the transfer of existing dashboards and views into the new search of 3.2 that should have run without problems - if not that is a bug that we should try to fix. But we would need some details, include log file from the upgrade and ideal the mongodb before and after the upgrade.

We do not provide any script to downgrade the database from 3.2 to 3.1 - as this is an unsupported operation. Sorry.

7

Where is can I get this upgrade path? Thank you!

8

he @chalfling

the upgrade and migration should run automatically on first start of the new Graylog server. That would be indicated in the logs. I’m not sure if you can trigger that a second time when you already did that once.

Do you have the server.log from the upgrade?

9

I Have:

2020-01-21 09:46:45,320 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: AWS plugins 3.2.0-beta.3 [org.graylog.aws.AWSPlugin]
2020-01-21 09:46:45,328 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Integrations 3.1.2 [org.graylog.integrations.IntegrationsPlugin]
2020-01-21 09:46:45,330 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Collector 3.2.0-beta.3 [org.graylog.plugins.collector.CollectorPlugin]
2020-01-21 09:46:45,331 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Stream Lookup pipeline function 2.0.0 [org.graylog.plugins.slookup.StreamLookupFunctionPlugin]
2020-01-21 09:46:45,333 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Threat Intelligence Plugin 3.2.0-beta.3 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
2020-01-21 09:46:46,054 INFO : org.graylog2.bootstrap.CmdLineTool - Running with JVM arguments: -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dapp=graylog -Dlog4j.configurationFile=/usr/local/etc/graylog/server/log4j2.xml -Xms8g -Xmx8g -XX:-OmitStackTraceInFastThrow -XX:+AggressiveOpts -XX:+CMSClassUnloadingEnabled -XX:+CMSConcurrentMTEnabled -XX:+CMSParallelRemarkEnabled -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -XX:+ResizeTLAB -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseFastAccessorMethods -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=75 -XX:NewRatio=1
2020-01-21 09:46:46,688 INFO : org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator null
2020-01-21 09:46:51,490 INFO : org.graylog2.shared.buffers.InputBufferImpl - Message journal is enabled.
2020-01-21 09:46:51,555 INFO : org.graylog2.plugin.system.NodeId - Node ID: 5f7e2f33-363h-4980-b6ff-047a3c960a2g
2020-01-21 09:46:52,001 INFO : kafka.log.LogManager - Loading logs.
2020-01-21 09:46:52,224 INFO : kafka.log.LogManager - Logs loading complete.
2020-01-21 09:46:52,231 INFO : org.graylog2.shared.journal.KafkaJournal - Initialized Kafka based journal at /var/db/graylog/journal
2020-01-21 09:46:52,558 INFO : org.mongodb.driver.cluster - Cluster created with settings {hosts=[10.138.138.220:27017, 127.0.0.1:27017], mode=MULTIPLE, requiredClusterType=REPLICA_SET, serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000, requiredReplicaSetName='replica0'}
2020-01-21 09:46:52,558 INFO : org.mongodb.driver.cluster - Adding discovered server 10.138.138.220:27017 to client view of cluster
2020-01-21 09:46:52,630 INFO : org.mongodb.driver.cluster - Adding discovered server 127.0.0.1:27017 to client view of cluster
2020-01-21 09:46:52,672 INFO : org.mongodb.driver.cluster - No server chosen by com.mongodb.Mongo$4@7cfb8e98 from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=10.138.138.220:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=127.0.0.1:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2020-01-21 09:46:52,712 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:2, serverValue:66}] to 127.0.0.1:27017
2020-01-21 09:46:52,726 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=127.0.0.1:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 10]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=4463137, setName='replica0', canonicalAddress=10.138.138.211:27017, hosts=[10.138.138.211:27017, 10.138.138.220:27017], passives=[], arbiters=[10.138.138.212:27017], primary='10.138.138.211:27017', tagSet=TagSet{[]}, electionId=7fffffff000000000000002a, setVersion=188512, lastWriteDate=Tue Jan 21 09:46:52 BRT 2020, lastUpdateTimeNanos=132974302898616}
2020-01-21 09:46:52,728 INFO : org.mongodb.driver.cluster - Adding discovered server 10.138.138.211:27017 to client view of cluster
2020-01-21 09:46:52,730 INFO : org.mongodb.driver.cluster - Adding discovered server 10.138.138.212:27017 to client view of cluster
2020-01-21 09:46:52,732 INFO : org.mongodb.driver.cluster - Server 127.0.0.1:27017 is no longer a member of the replica set.  Removing from client view of cluster.
2020-01-21 09:46:52,733 INFO : org.mongodb.driver.cluster - Canonical address 10.138.138.211:27017 does not match server address.  Removing 127.0.0.1:27017 from client view of cluster
2020-01-21 09:46:52,736 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:3, serverValue:67}] to 10.138.138.211:27017
2020-01-21 09:46:52,738 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:4, serverValue:12}] to 10.138.138.212:27017
2020-01-21 09:46:52,743 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=10.138.138.211:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 10]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=5847042, setName='replica0', canonicalAddress=10.138.138.211:27017, hosts=[10.138.138.211:27017, 10.138.138.220:27017], passives=[], arbiters=[10.138.138.212:27017], primary='10.138.138.211:27017', tagSet=TagSet{[]}, electionId=7fffffff000000000000002a, setVersion=188512, lastWriteDate=Tue Jan 21 09:46:52 BRT 2020, lastUpdateTimeNanos=132974326346987}
2020-01-21 09:46:52,744 INFO : org.mongodb.driver.cluster - Setting max election id to 7fffffff000000000000002a from replica set primary 10.138.138.211:27017
2020-01-21 09:46:52,744 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=10.138.138.212:27017, type=REPLICA_SET_ARBITER, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 10]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=null, roundTripTimeNanos=5446363, setName='replica0', canonicalAddress=10.138.138.212:27017, hosts=[10.138.138.211:27017, 10.138.138.220:27017], passives=[], arbiters=[10.138.138.212:27017], primary='10.138.138.211:27017', tagSet=TagSet{[]}, electionId=null, setVersion=188512, lastWriteDate=Tue Jan 21 09:46:07 BRT 2020, lastUpdateTimeNanos=132974327470035}
2020-01-21 09:46:52,745 INFO : org.mongodb.driver.cluster - Setting max set version to 188512 from replica set primary 10.138.138.211:27017
2020-01-21 09:46:52,745 INFO : org.mongodb.driver.cluster - Discovered replica set primary 10.138.138.211:27017
2020-01-21 09:46:52,791 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:5, serverValue:68}] to 10.138.138.211:27017
2020-01-21 09:46:53,277 INFO : org.graylog2.shared.buffers.InputBufferImpl - Initialized InputBufferImpl with ring size <32768> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2020-01-21 09:46:53,774 INFO : io.searchbox.client.AbstractJestClient - Setting server pool to a list of 3 servers: [http://10.138.138.211:9200,http://10.138.138.212:9200,http://10.138.138.213:9200]
2020-01-21 09:46:53,775 INFO : io.searchbox.client.JestClientFactory - Using multi thread/connection supporting pooling connection manager
2020-01-21 09:46:53,895 INFO : io.searchbox.client.JestClientFactory - Using custom ObjectMapper instance
2020-01-21 09:46:53,895 INFO : io.searchbox.client.JestClientFactory - Node Discovery enabled...
2020-01-21 09:46:53,927 INFO : io.searchbox.client.JestClientFactory - Idle connection reaping disabled...
2020-01-21 09:46:54,844 INFO : org.graylog2.shared.buffers.ProcessBuffer - Initialized ProcessBuffer with ring size <8192> and wait strategy <BlockingWaitStrategy>.
2020-01-21 09:46:55,943 INFO : org.graylog2.buffers.OutputBuffer - Initialized OutputBuffer with ring size <8192> and wait strategy <BlockingWaitStrategy>.
2020-01-21 09:46:56,110 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:6, serverValue:69}] to 10.138.138.211:27017
2020-01-21 09:46:58,768 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server 3.2.0-beta.3+1189691 starting up
2020-01-21 09:46:58,768 INFO : org.graylog2.bootstrap.ServerBootstrap - JRE: Oracle Corporation 1.8.0_181 on FreeBSD 11.3-STABLE
2020-01-21 09:46:58,768 INFO : org.graylog2.bootstrap.ServerBootstrap - Deployment: unknown
2020-01-21 09:46:58,769 INFO : org.graylog2.bootstrap.ServerBootstrap - OS: FreeBSD
2020-01-21 09:46:58,769 INFO : org.graylog2.bootstrap.ServerBootstrap - Arch: amd64
2020-01-21 09:46:58,814 INFO : org.graylog2.shared.initializers.PeriodicalsService - Starting 30 periodicals ...
2020-01-21 09:46:58,937 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling every [1s].
2020-01-21 09:46:58,958 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:7, serverValue:70}] to 10.138.138.211:27017
2020-01-21 09:46:58,963 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:8, serverValue:71}] to 10.138.138.211:27017
2020-01-21 09:46:58,979 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration] periodical, running forever.
2020-01-21 09:46:58,983 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.AlertScannerThread] periodical. Not configured to run on this node.
2020-01-21 09:46:58,983 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s].
2020-01-21 09:46:58,988 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], polling every [20s].
2020-01-21 09:46:58,988 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical. Not configured to run on this node.
2020-01-21 09:46:58,989 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever.
2020-01-21 09:46:58,989 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2020-01-21 09:46:58,991 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2020-01-21 09:46:58,991 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s].
2020-01-21 09:46:58,993 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2020-01-21 09:46:58,994 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling every [1800s].
2020-01-21 09:46:58,995 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], polling every [1s].
2020-01-21 09:46:58,996 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
2020-01-21 09:46:58,995 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:9, serverValue:72}] to 10.138.138.211:27017
2020-01-21 09:46:58,996 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], polling every [86400s].
2020-01-21 09:46:58,996 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:10, serverValue:73}] to 10.138.138.211:27017
2020-01-21 09:46:58,997 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running forever.
2020-01-21 09:46:58,999 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, running forever.
2020-01-21 09:46:58,999 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], polling every [3600s].
2020-01-21 09:46:59,001 INFO : org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration - Legacy default stream has no connections, no migration needed.
2020-01-21 09:46:59,055 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not configured to run on this node.
2020-01-21 09:46:59,055 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, running forever.
2020-01-21 09:46:59,073 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.LdapGroupMappingMigration] periodical. Not configured to run on this node.
2020-01-21 09:46:59,088 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running forever.
2020-01-21 09:46:59,089 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.TrafficCounterCalculator] periodical in [0s], polling every [1s].
2020-01-21 09:46:59,092 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical] periodical in [0s], polling every [3600s].
2020-01-21 09:46:59,092 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.scheduler.periodicals.ScheduleTriggerCleanUp] periodical in [120s], polling every [86400s].
2020-01-21 09:46:59,093 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredSidecarsThread] periodical in [0s], polling every [600s].
2020-01-21 09:46:59,095 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredConfigurationUploads] periodical in [0s], polling every [600s].
2020-01-21 09:46:59,097 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.views.search.db.SearchesCleanUpJob] periodical in [3600s], polling every [28800s].
2020-01-21 09:46:59,098 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.events.periodicals.EventNotificationStatusCleanUp] periodical in [120s], polling every [86400s].
2020-01-21 09:46:59,100 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread] periodical in [0s], polling every [3600s].

2020-01-21 09:47:00,000 INFO : org.graylog2.shared.initializers.JerseyService - Enabling CORS for HTTP endpoint
2020-01-21 09:47:00,020 ERROR: org.graylog2.periodical.ConfigurationManagementPeriodical - Error while running migration <V20190705071400_AddEventIndexSetsMigration{2019-07-05T07:14:00Z}>
2020-01-21 09:47:00,167 WARN : org.graylog.events.legacy.LegacyAlertConditionMigrator - Couldn't migrate unknown legacy alert condition type: com.alexjck.plugins.conditional.alerts.ConditionalCountAlertCondition
2020-01-21 09:47:00,167 WARN : org.graylog.events.legacy.LegacyAlertConditionMigrator - Couldn't migrate unknown legacy alert condition type: com.alexjck.plugins.conditional.alerts.ConditionalCountAlertCondition
2020-01-21 09:47:00,167 WARN : org.graylog.events.legacy.LegacyAlertConditionMigrator - Couldn't migrate unknown legacy alert condition type: com.alexjck.plugins.conditional.alerts.ConditionalCountAlertCondition
2020-01-21 09:47:00,187 WARN : org.graylog.events.legacy.LegacyAlertConditionMigrator - Couldn't migrate unknown legacy alert condition type: com.alexjck.plugins.conditional.alerts.ConditionalCountAlertCondition
2020-01-21 09:47:00,374 ERROR: org.graylog2.periodical.ConfigurationManagementPeriodical - Error while running migration <V20191125144500_MigrateDashboardsToViews{2019-11-25T14:45:00Z}>
2020-01-21 09:47:04,387 INFO : org.graylog2.lookup.LookupDataAdapterRefreshService - Adding job for <tor-exit-node/5bb65b7ba6e817ce85810592/@201bb75f> [interval=3600000ms]
2020-01-21 09:47:12,671 INFO : org.mongodb.driver.cluster - Exception in monitor thread while connecting to server 10.138.138.220:27017
2020-01-21 09:47:36,696 WARN : org.graylog2.audit.jersey.AuditEventModelProcessor - REST endpoint does not use a registered audit type:   POST api/plugins/org.graylog.integrations/aws/inputs (type: "integrations:kinesis_input:create")
2020-01-21 09:47:38,268 INFO : org.glassfish.grizzly.http.server.NetworkListener - Started listener bound to [0.0.0.0:9000]
2020-01-21 09:47:38,271 INFO : org.glassfish.grizzly.http.server.HttpServer - [HttpServer] Started.
2020-01-21 09:47:38,271 INFO : org.graylog2.shared.initializers.JerseyService - Started REST API at <0.0.0.0:9000>
2020-01-21 09:47:38,272 INFO : org.graylog2.shared.initializers.ServiceManagerListener - Services are healthy
2020-01-21 09:47:38,272 INFO : org.graylog2.shared.initializers.InputSetupService - Triggering launching persisted inputs, node transitioned from Uninitialized?[LB:DEAD] to Running?[LB:ALIVE]
2020-01-21 09:47:38,287 INFO : org.graylog2.bootstrap.ServerBootstrap - Services started, startup times in ms: {BufferSynchronizerService [RUNNING]=5, OutputSetupService [RUNNING]=8, JobSchedulerService [RUNNING]=8, EtagService [RUNNING]=10, ConfigurationEtagService [RUNNING]=10, UrlWhitelistService [RUNNING]=14, StreamCacheService [RUNNING]=94, GracefulShutdownService [RUNNING]=123, KafkaJournal [RUNNING]=131, InputSetupService [RUNNING]=135, JournalReader [RUNNING]=136, MongoDBProcessingStatusRecorderService [RUNNING]=238, PeriodicalsService [RUNNING]=293, LookupTableService [RUNNING]=5467, JerseyService [RUNNING]=39458}
2020-01-21 09:47:38,321 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server up and running.
2020-01-21 09:47:46,125 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:46,130 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:46,135 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:46,140 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:49,516 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:14, serverValue:74}] to 10.138.138.211:27017
2020-01-21 09:47:50,319 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:47:52,011 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,156 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,158 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,160 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,923 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,924 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,925 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:52,928 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:53,502 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:54,079 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:54,480 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:47:54,872 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:16, serverValue:75}] to 10.138.138.211:27017
2020-01-21 09:47:57,193 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:01,771 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:01,777 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:01,779 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:01,782 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:05,029 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:05,030 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:05,031 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:05,033 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:05,740 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:05,848 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:07,429 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:07,431 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:07,432 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:07,434 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:07,612 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:13,722 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,727 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,728 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,730 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,819 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:13,888 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,890 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,892 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:13,894 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:30,096 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:30,099 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:30,100 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:30,103 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:30,186 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:34,156 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,158 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,159 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,161 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,782 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,783 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,784 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,785 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:34,907 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:34,935 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:48:35,263 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:35,265 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:35,266 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:35,269 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:47,696 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:47,698 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:47,699 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:47,701 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:57,847 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:57,849 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:57,851 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:48:57,854 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:00,842 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:00,844 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:00,845 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:00,848 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:13,943 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:49:13,944 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:49:18,641 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,643 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,645 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,691 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,732 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,734 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,735 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:18,737 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:19,323 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:49:20,933 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:49:20,934 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:49:39,012 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,020 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,021 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,023 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,563 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,565 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,567 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,569 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:39,710 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
2020-01-21 09:49:43,017 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:27, serverValue:1}] to 10.138.138.220:27017
2020-01-21 09:49:43,019 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=10.138.138.220:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 10]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=704209, setName='replica0', canonicalAddress=10.138.138.220:27017, hosts=[10.138.138.211:27017, 10.138.138.220:27017], passives=[], arbiters=[10.138.138.212:27017], primary='10.138.138.211:27017', tagSet=TagSet{[]}, electionId=null, setVersion=188512, lastWriteDate=Tue Jan 21 09:46:07 BRT 2020, lastUpdateTimeNanos=133144601752032}
2020-01-21 09:49:50,659 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:53, serverValue:4}] to 10.138.138.220:27017
2020-01-21 09:49:50,722 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=10.138.138.220:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 10]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=24679323, setName='replica0', canonicalAddress=10.138.138.220:27017, hosts=[10.138.138.211:27017, 10.138.138.220:27017], passives=[], arbiters=[10.138.138.212:27017], primary='10.138.138.211:27017', tagSet=TagSet{[]}, electionId=null, setVersion=188512, lastWriteDate=Tue Jan 21 09:49:50 BRT 2020, lastUpdateTimeNanos=133152305260648}
2020-01-21 09:49:58,050 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,052 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,053 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,054 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,253 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,255 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,257 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:58,259 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:59,886 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:59,900 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:59,912 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
2020-01-21 09:49:59,914 ERROR: org.graylog2.streams.StreamServiceImpl - Skipping alert condition.
10

I can spot direct that you have not upgraded the integrations plugin to the latest beta too (you run that on 3.1.2) but all Graylog provided Plugins need to be on the latest version.

maybe that will already solve the problems.

11

he @chalfling

is the following something that describes nearly your problem?

12

I’ll try to upgrade the integrations plugin too. Thank you!

13

This is the exactly same problem.

14

I tried to upgrade to new version 3.2.0-beta4 with integrations plugin 3.1.2.

I had the same problem again. Look full log from start to shutdown:
https://pastebin.com/YVsTHE4s

Thank you!

15

he @chalfling

your MongoDB is now haunted. We have no way to clean that up in any way and I hope for you that you have made a backup before you upgraded to the beta version.

Currently I can not think about a way to solve the Database problems you are in. Because you have a non clear state of the database.

Sorry, I do not have a solution for you

16

There is any way to export/import dashboards/views in another installation?

17

yes - the content packs are made for that.

18

Hello @jan !

I tried a fresh install. My steps:

  1. Install and Start MongoDB (rs.initiate)
  2. Install and Start Graylog 3.1.4 (with different index set like: graylog3) and Start Graylog Server
  3. Create a Content Pack with my Streams and Dashboards (from my production cluster in 3.1.4)
  4. Install the Content Pack in the fresh installation.
  5. Go to http://URL:9000/dashboards and see all my dashboards
  6. Stop Graylog Server fresh installation and remove package
  7. Install Graylog 3.2.0-beta.4 package and Start Graylog Server
  8. Go to http://URL:9000/dashboards and see ONLY “Sources” Dashboard

See my fresh install logs:
https://pastebin.com/5q3q4reG

19

Hey @chalfling,

good find! I can’t seem to reproduce it by myself though. I have an idea, but would you mind give me the content pack to faster reproduce the problem?

Thanks for your support!

20

Hey @chalfling,

we have made some improvements in 3.2.0-rc.1 that should address your issue. Feel free to try it! If you do, we would be happy if you would report back your outcome.

Kr,
D.

1 Like