Hi,
I’ve upgrade my graylog server to 3.0.
I’ve this events in the graylogs logs:
2020-05-18T16:17:08.452+02:00 ERROR [ConfigurationManagementPeriodical] Error while running migration <V20180924111644_AddDefaultGrokPatterns{2018-09-24T11:16:44Z}>
org.graylog2.contentpacks.exceptions.ContentPackException: Failed to install content pack <a3ce55ad-bdf3-7a50-305c-1e5bf3de6eca/1>
at org.graylog2.contentpacks.ContentPackService.installContentPack(ContentPackService.java:158) ~[graylog.jar:?]
at org.graylog2.contentpacks.ContentPackService.installContentPack(ContentPackService.java:99) ~[graylog.jar:?]
at org.graylog2.migrations.V20180924111644_AddDefaultGrokPatterns.upgrade(V20180924111644_AddDefaultGrokPatterns.java:76) ~[graylog.jar:?]
at org.graylog2.periodical.ConfigurationManagementPeriodical.doRun(ConfigurationManagementPeriodical.java:43) [graylog.jar:?]
at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:77) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: org.graylog2.contentpacks.exceptions.DivergingEntityConfigurationException: Expected Grok pattern for name "HOUR": <(?:2[0123]|[01]?[0-9])>; actual Grok pattern: <(?:[0-2][0-9])>
at org.graylog2.contentpacks.facades.GrokPatternFacade.compareGrokPatterns(GrokPatternFacade.java:138) ~[graylog.jar:?]
at org.graylog2.contentpacks.facades.GrokPatternFacade.lambda$findExisting$0(GrokPatternFacade.java:131) ~[graylog.jar:?]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_232]
at org.graylog2.contentpacks.facades.GrokPatternFacade.findExisting(GrokPatternFacade.java:131) ~[graylog.jar:?]
at org.graylog2.contentpacks.facades.GrokPatternFacade.findExisting(GrokPatternFacade.java:119) ~[graylog.jar:?]
at org.graylog2.contentpacks.ContentPackService.installContentPack(ContentPackService.java:131) ~[graylog.jar:?]
... 5 more
[...]
2020-05-18T16:17:31.470+02:00 WARN [AbstractTcpTransport] receiveBufferSize (SO_RCVBUF) for input SyslogTCPInput{title=LINUX, type=org.graylog2.inputs.syslog.tcp.SyslogTCPInput, nodeId=null} (channel [id: 0x15d4248d, L:/0.0.0.0:61025]) should be 1048576 but is 425984.
[...]
2020-05-18T16:18:03.124+02:00 ERROR [ServerRuntime$Responder] An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed
at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: java.io.IOException: Connection is closed
at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:445) ~[graylog.jar:?]
at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:677) ~[graylog.jar:?]
at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:83) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2039) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1354) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1301) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:49) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:689) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1429) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:951) ~[graylog.jar:?]
at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[graylog.jar:?]
at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[graylog.jar:?]
at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[graylog.jar:?]
... 20 more
Caused by: java.io.IOException: Peer reset connection
at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_232]
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_232]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_232]
at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_232]
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) ~[?:1.8.0_232]
at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:149) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeSimpleBuffer(TCPNIOUtils.java:133) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:126) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:106) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:260) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:169) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:71) ~[graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:126) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:191) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112) ~[graylog.jar:?]
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:890) ~[graylog.jar:?]
at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:858) ~[graylog.jar:?]
at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1059) ~[graylog.jar:?]
at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:709) ~[graylog.jar:?]
at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:83) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:233) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2039) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeNumber(UTF8JsonGenerator.java:853) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.NumberSerializers$IntegerSerializer.serialize(NumberSerializers.java:137) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:689) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1429) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:951) ~[graylog.jar:?]
at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[graylog.jar:?]
at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) ~[graylog.jar:?]
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) ~[graylog.jar:?]
at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ~[graylog.jar:?]
... 20 more
After a second restart, I have no more errors. Weird ??
Also my received buffer is set to 1048576 but events say they are set to 425984???
On my second node I’ve:
2020-05-18T16:29:39.757+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.net.SocketTimeoutException: timeout
at okio.Okio$4.newTimeoutException(Okio.java:232) ~[graylog.jar:?]
at okio.AsyncTimeout.exit(AsyncTimeout.java:285) ~[graylog.jar:?]
at okio.AsyncTimeout$2.read(AsyncTimeout.java:241) ~[graylog.jar:?]
at okio.RealBufferedSource.indexOf(RealBufferedSource.java:355) ~[graylog.jar:?]
[...]
2020-05-18T16:29:44.851+02:00 WARN [NodePingThread] Did not find meta info of this node. Re-registering.
Someone could help me ?
Thanks
