Error with multiple group by fields in aggregation search in version 5

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
A functional aggregation from Graylog 4.2.3/elasticsearch 7.0.2 doesn’t work on Graylog 5.0.2/opensearch 2.4.1
At my newly installed Graylog 5.0.2, 3 node cluster with a 3+5 (master/data) node opensearch 2.4.1 cluster, I have created an aggregation on a dashboard with the following settings:
Group by fields:
streams, row
timestamp, interval 1 day, column
Matrics:
Sum(gl2_accounted_message_size

Searching 7 days back I will get a find result. There are only 9 streams, less than 30 millions records and 40GB of data.

I added a second group by field:
source, column

The order of group by fields looks like this:
Row, streams
Row, source
Column, timestamp, interval 1 day

Error 1:
Search time range for 1 day gives a result, but with errors comparing to the old setup. The result is not grouped together. In the old setup the result are grouped together as defined in the search (streams,source) and you can sort by count. In the new version 5, the result are NOT grouped together any more.

Error 2:
If I change the search time range to more days I do not receive a result. In the browser’s debug console window I get this error:

FetchError: There was an error fetching a resource: Gateway Time-out. Additional information: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
    r SharedEntity.ts:93
    d FetchError.ts:29
    p FetchError.ts:39
    e FetchProvider.ts:42
    s FetchProvider.ts:2
    _invoke FetchProvider.ts:2
    b FetchProvider.ts:2
    O FetchProvider.ts:2
    a FetchProvider.ts:38
    promise callback*O FetchProvider.ts:2
    a FetchProvider.ts:38
    t FetchProvider.ts:38
    t FetchProvider.ts:38
    I FetchProvider.ts:38
    errorHandler FetchProvider.ts:124
    promise callback*value CancellablePromise.ts:34
    value FetchProvider.ts:196
    M FetchProvider.ts:222
    M FetchProvider.ts:224
    d SearchJobs.ts:43
    trackJob SearchStore.ts:163
    _executePromise SearchStore.ts:244
    execute SearchStore.ts:170
    n vendor.1e79e27aeb952fbe940b.js:2
    emit vendor.1e79e27aeb952fbe940b.js:2
    trigger vendor.1e79e27aeb952fbe940b.js:2
    triggerAsync vendor.1e79e27aeb952fbe940b.js:2
    <anonymous> task.js:61
    C task.js:35
    S task.js:46
SearchStore.ts:47:10
    C SearchStore.ts:47
    (Async: promise callback)
    value CancellablePromise.ts:30
    _executePromise SearchStore.ts:245
    execute SearchStore.ts:170
    n vendor.1e79e27aeb952fbe940b.js:2
    emit vendor.1e79e27aeb952fbe940b.js:2
    trigger vendor.1e79e27aeb952fbe940b.js:2
    triggerAsync vendor.1e79e27aeb952fbe940b.js:2
    <anonym> task.js:61
    C task.js:35
    S task.js:46

I the graylog logs I get this error message:

ERROR: org.glassfish.jersey.server.ServerRuntime$Responder - An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed
        at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:642) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:373) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:363) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:258) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
        at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.io.IOException: Connection is closed
        at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?]
        at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?]
        at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2171) ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeBytes(UTF8JsonGenerator.java:1260) ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:284) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:726) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:808) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:764) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:808) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:764) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1518) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1007) ~[graylog.jar:?]
        at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:617) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?]
        at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?]
        at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?]
        ... 20 more
Caused by: java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
        at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[?:?]
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) ~[?:?]
        at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
        at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
        at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeSimpleBuffer(TCPNIOUtils.java:109) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:102) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?]
        at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?]
        at org.glassfish.grizzly.ssl.SSLBaseFilter$SSLTransportFilterWrapper.handleWrite(SSLBaseFilter.java:1125) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?]
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?]
        at org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:370) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?]
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?]
        at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?]
        at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?]
        at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?]
        at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:204) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2171) ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:515) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedStringListSerializer.serializeContents(IndexedStringListSerializer.java:103) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedStringListSerializer.serialize(IndexedStringListSerializer.java:77) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedStringListSerializer.serialize(IndexedStringListSerializer.java:22) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:808) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:764) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:808) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:764) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1518) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1007) ~[graylog.jar:?]
        at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:617) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?]
        at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?]
        at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?]
        at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?]
        ... 20 more

2. Describe your environment:

  • OS Information:
    Ubuntu server 22.04 with docker.
    3 nodes with Graylog 5.0.2 and mongo 6.0.3
    3 master nodes and 5 data nodes with opensearch 2.4.1

  • Package Version:

  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?
Tried several debugging where there could be an time-out issue.

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

Hello @jmkofoed

This error

An I/O error has occurred while writing a response message entity to the container output stream.

An FetchError: There was an error fetching a resource: Gateway Time-out.

Please check the firewalls, SElinux and proxies to see if you have not set up a connection timeout. This would be the most likely culprit.

Couple Question,

What do you see in Opensearch logs? and is it posible to see a stacktrace before this error occurs? meaning when you started Graylog service did you notice any other errors/warnings? Same with you Opensearch clusters. Since your using Opensearch by chance did you disable the security plugin?