IOEXception when checking alerts

daniel2 · January 11, 2024, 2:19pm

Graylog version: Graylog 5.2.2+8eab621

I get a red field indicating an error when trying to check alerts.
Error in Browser:
Could not retrieve event definitions Fetching event definitions failed with status: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: IOException encountered while reading from a byte array input stream

Corresponding stacktrace in Log:

2024-01-11T14:46:46.976+01:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.lang.RuntimeException: IOException encountered while reading from a byte array input stream
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:67) ~[graylog.jar:?]
        at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:49) ~[graylog.jar:?]
        at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:29) ~[graylog.jar:?]
        at com.mongodb.internal.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52) ~[graylog.jar:?]
        at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42) ~[graylog.jar:?]
        at org.bson.internal.LazyCodec.decode(LazyCodec.java:53) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:104) ~[graylog.jar:?]
        at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42) ~[graylog.jar:?]
        at com.mongodb.internal.connection.ReplyMessage.<init>(ReplyMessage.java:50) ~[graylog.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.getCommandResult(InternalStreamConnection.java:538) ~[graylog.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:423) ~[graylog.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:340) ~[graylog.jar:?]
        at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:116) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:643) ~[graylog.jar:?]
        at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:206) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:119) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:85) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:75) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServer$OperationCountTrackingConnection.command(DefaultServer.java:293) ~[graylog.jar:?]
        at com.mongodb.internal.operation.CommandOperationHelper.createReadCommandAndExecute(CommandOperationHelper.java:233) ~[graylog.jar:?]
        at com.mongodb.internal.operation.FindOperation.lambda$execute$1(FindOperation.java:332) ~[graylog.jar:?]
        at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$0(OperationHelper.java:356) ~[graylog.jar:?]
        at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:381) ~[graylog.jar:?]
        at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$1(OperationHelper.java:355) ~[graylog.jar:?]
        at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:381) ~[graylog.jar:?]
        at com.mongodb.internal.operation.OperationHelper.withSourceAndConnection(OperationHelper.java:354) ~[graylog.jar:?]
        at com.mongodb.internal.operation.FindOperation.lambda$execute$2(FindOperation.java:329) ~[graylog.jar:?]
        at com.mongodb.internal.async.function.RetryingSyncSupplier.get(RetryingSyncSupplier.java:67) ~[graylog.jar:?]
        at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:340) ~[graylog.jar:?]
        at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:79) ~[graylog.jar:?]
        at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:191) ~[graylog.jar:?]
        at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:168) ~[graylog.jar:?]
        at com.mongodb.DBCursor.initializeCursor(DBCursor.java:810) ~[graylog.jar:?]
        at com.mongodb.DBCursor.hasNext(DBCursor.java:163) ~[graylog.jar:?]
        at org.mongojack.DBCursor.hasNext(DBCursor.java:255) ~[graylog.jar:?]
        at java.util.Iterator.forEachRemaining(Unknown Source) ~[?:?]
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
        at java.util.stream.ReduceOps$5.evaluateSequential(Unknown Source) ~[?:?]
        at java.util.stream.ReduceOps$5.evaluateSequential(Unknown Source) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
        at java.util.stream.ReferencePipeline.count(Unknown Source) ~[?:?]
        at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSortWithGrandTotal(PaginatedDbService.java:200) ~[graylog.jar:?]
        at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSort(PaginatedDbService.java:181) ~[graylog.jar:?]
        at org.graylog.events.processor.DBEventDefinitionService.searchPaginated(DBEventDefinitionService.java:65) ~[graylog.jar:?]
        at org.graylog.events.rest.EventDefinitionsResource.list(EventDefinitionsResource.java:222) ~[graylog.jar:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:134) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:177) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[graylog.ja>
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:81) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
        at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Null field
 at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@8b4d45; pos: 316] (through reference chain: org.graylog.events.processor.$AutoValue_EventDefinitionDto$Builde>
        at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:276) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:630) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:618) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:173) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:167) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4801) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3043) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
        ... 79 more
Caused by: java.lang.NullPointerException: Null field
        at org.graylog.plugins.views.search.searchtypes.pivot.series.AutoValue_Sum$Builder.field(AutoValue_Sum.java:118) ~[graylog.jar:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:170) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:167) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4801) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3043) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
        ... 79 more

drewmiranda-gl · January 11, 2024, 5:17pm

Is this something that just recently started to happen in your cluster or has this been a problem for some time?

The error is pointing me in the direction of there being an error in one of the event definitions.

Also, do you happen to have any custom graylog plugins installed?

If there is an event definition with formatting issues, its possible to review these via graylog’s MongoDB database. You can export the events collection using the following command:

mongoexport --collection=event_definitions --db=graylog --out=event_definitions.json

If you feel comfortable sharing the output, we may be able to spot the issue.

daniel2 · January 12, 2024, 3:12pm

Thanks.
But I don’t know what would qualify as custom plugin in this context.

Graylog Plugins:
Threat Intelligence Lookup
AWS
Geo-Location Processor

Sharing those should not be an issue without names and description.
Regex for the sanitized fields: “[A-Za-z ]*”
So I think we can be fairly certain that no strange character in there caused the problem.

there is very little as it only was for testing so far, but names and descriptions where already close to production.

{“_id”:{“$oid”:“628b89b102f3e719b540683d”},“title”:“clear”,“description”:“clear”,“priority”:3,“alert”:true,“config”:{“type”:“aggregation-v1”,“query”:“”,“query_parameters”:,“streams”:[“619ce5b986ec512199078eaf”],“group_by”:,“series”:[{“id”:“sum-”,“field”:null,“type”:“sum”}],“conditions”:{“expression”:{“expr”:“>”,“left”:{“expr”:“number-ref”,“ref”:“sum-”},“right”:{“expr”:“number”,“value”:10.0}}},“search_within_ms”:7200000,“execute_every_ms”:600000},“field_spec”:{},“key_spec”:,“notification_settings”:{“grace_period_ms”:57600000,“backlog_size”:50},“notifications”:[{“notification_id”:“628b862a02f3e719b5406467”,“notification_parameters”:null}],“storage”:[{“type”:“persist-to-streams-v1”,“streams”:[“000000000000000000000002”]}]}

{“_id”:{“$oid”:“64803e891855ce22530a1454”},“_scope”:“SYSTEM_NOTIFICATION_EVENT”,“title”:“clear”,“description”:“clear”,“updated_at”:{“$date”:“2023-06-07T08:23:37.942Z”},“priority”:1,“alert”:false,“config”:{“type”:“system-notifications-v1”},“field_spec”:{},“key_spec”:,“notification_settings”:{“grace_period_ms”:0,“backlog_size”:0},“notifications”:,“storage”:[{“type”:“persist-to-streams-v1”,“streams”:[“000000000000000000000003”]}],“state”:“ENABLED”}

drewmiranda-gl · January 12, 2024, 9:40pm

Are those the only 2 event definitions you had?

The one with oid 628b89b102f3e719b540683d caught my eye here:

If you feel comfortable using the mongo command line you can try deleting the record. Note that deleting any data always carries risks and these instructions are provided as is. I tested them with Ubuntu Server 22.04 LTS and Mongo 6.0.12

ssh to the server where mongodb is running
run one of these commands, this will open a mongo shell and automatically select the graylog database:
- mongo 5 or earlier: mongo graylog
- mongo 6: mongosh graylog
the terminal should show graylog> at this point. If it does not, you can run the following to select the graylog database:
- use graylog
Verify you get a result when searching for the record by its OID:
- db.getCollection("event_definitions").find({"_id":ObjectId('628b89b102f3e719b540683d')})
- Should return something like
  
  image1566×282 57.6 KB
Delete the record by its OID:
- db.event_definitions.deleteOne( {"_id":ObjectId('628b89b102f3e719b540683d')} )
- Should return something like
  
  image1414×76 40.9 KB

Its possible this is realted to Event Definitions: Aggregation with `card()` or `max()` functions without fields prevents access to Events page after upgrade to `5.2.0`. · Issue #17367 · Graylog2/graylog2-server · GitHub , but i’m not 100% certain.

patrickmann · January 15, 2024, 9:08am

Did you recently upgrade to 5.2? I agree with @drewmiranda-gl that this looks like

github.com/Graylog2/graylog2-server

Event Definitions: Aggregation with `card()` or `max()` functions without fields prevents access to Events page after upgrade to `5.2.0`.

opened 01:45PM - 17 Nov 23 UTC

thll

bug triaged

## Expected Behavior It should not be possible to save an event definition …that uses an aggregation with the `card()` function but doesn't have a field for the function set. ## Current Behavior The following event definition is considered valid: ![image](https://github.com/Graylog2/graylog2-server/assets/886541/57f87ab9-bcd0-46bb-8606-27bcca763907) But after saving it, the following errors are printed to the server log: <details> <summary>stacktrace</summary> <pre> 2023-11-17 13:28:37,493 ERROR: org.graylog.events.processor.EventProcessorEngine - Caught an unhandled exception while executing event processor <aggregation-v1/Test/655769de14af3362f3e455a7> - Make sure to modify the event processor to throw only EventProcessorException so we get more context! java.lang.IllegalArgumentException: Function <card> requires a field at org.graylog.events.processor.aggregation.AggregationFunction.toSeriesSpec(AggregationFunction.java:77) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.lambda$getAggregationQuery$6(PivotAggregationSearch.java:427) ~[graylog.jar:?] at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?] at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.getAggregationQuery(PivotAggregationSearch.java:428) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.getSearchJob(PivotAggregationSearch.java:360) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.doSearch(PivotAggregationSearch.java:126) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.aggregatedSearch(AggregationEventProcessor.java:248) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.createEvents(AggregationEventProcessor.java:126) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:91) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorExecutionJob.execute(EventProcessorExecutionJob.java:115) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.executeJob(JobExecutionEngine.java:186) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$handleTrigger$2(JobExecutionEngine.java:159) ~[graylog.jar:?] at com.codahale.metrics.Timer.time(Timer.java:151) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.handleTrigger(JobExecutionEngine.java:159) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$execute$0(JobExecutionEngine.java:128) ~[graylog.jar:?] at org.graylog.scheduler.worker.JobWorkerPool.lambda$execute$0(JobWorkerPool.java:121) ~[graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Unknown Source) [?:?] 2023-11-17 13:28:37,495 ERROR: org.graylog.events.processor.EventProcessorExecutionJob - Event processor <aggregation-v1/655769de14af3362f3e455a7> failed to execute: Couldn't create events for: EventDefinitionDto{scope=DEFAULT, id=655769de14af3362f3e455a7, title=Test, description=, updatedAt=2023-11-17T13:25:50.313Z, priority=2, alert=false, config=AggregationEventProcessorConfig{type=aggregation-v1, query=, queryParameters=[], streams=[], groupBy=[], series=[AggregationSeries{id=card-, function=CARD, field=Optional.empty}], conditions=Optional[AggregationConditions{expression=Optional[Greater{expr=>, left=NumberReference{expr=number-ref, ref=card-}, right=NumberValue{expr=number, value=0.0}}]}], searchWithinMs=300000, executeEveryMs=300000}, fieldSpec={}, keySpec=[], notificationSettings=EventNotificationSettings{gracePeriodMs=300000, backlogSize=0}, notifications=[], storage=[Config{type=persist-to-streams-v1, streams=[000000000000000000000002]}], schedulerCtx=null} (retry in 5000 ms) org.graylog.events.processor.EventProcessorException: Couldn't create events for: EventDefinitionDto{scope=DEFAULT, id=655769de14af3362f3e455a7, title=Test, description=, updatedAt=2023-11-17T13:25:50.313Z, priority=2, alert=false, config=AggregationEventProcessorConfig{type=aggregation-v1, query=, queryParameters=[], streams=[], groupBy=[], series=[AggregationSeries{id=card-, function=CARD, field=Optional.empty}], conditions=Optional[AggregationConditions{expression=Optional[Greater{expr=>, left=NumberReference{expr=number-ref, ref=card-}, right=NumberValue{expr=number, value=0.0}}]}], searchWithinMs=300000, executeEveryMs=300000}, fieldSpec={}, keySpec=[], notificationSettings=EventNotificationSettings{gracePeriodMs=300000, backlogSize=0}, notifications=[], storage=[Config{type=persist-to-streams-v1, streams=[000000000000000000000002]}], schedulerCtx=null} at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:105) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorExecutionJob.execute(EventProcessorExecutionJob.java:115) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.executeJob(JobExecutionEngine.java:186) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$handleTrigger$2(JobExecutionEngine.java:159) ~[graylog.jar:?] at com.codahale.metrics.Timer.time(Timer.java:151) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.handleTrigger(JobExecutionEngine.java:159) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$execute$0(JobExecutionEngine.java:128) ~[graylog.jar:?] at org.graylog.scheduler.worker.JobWorkerPool.lambda$execute$0(JobWorkerPool.java:121) ~[graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Unknown Source) [?:?] Caused by: java.lang.IllegalArgumentException: Function <card> requires a field at org.graylog.events.processor.aggregation.AggregationFunction.toSeriesSpec(AggregationFunction.java:77) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.lambda$getAggregationQuery$6(PivotAggregationSearch.java:427) ~[graylog.jar:?] at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?] at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.getAggregationQuery(PivotAggregationSearch.java:428) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.getSearchJob(PivotAggregationSearch.java:360) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.PivotAggregationSearch.doSearch(PivotAggregationSearch.java:126) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.aggregatedSearch(AggregationEventProcessor.java:248) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.createEvents(AggregationEventProcessor.java:126) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:91) ~[graylog.jar:?] ... 12 more </pre> </details> What's even worse is that if a cluster with such a non-working event definition is upgraded to `5.2.0`, the events page doesn't render anymore and the following error is thrown: <details> <summary>Stacktrace `5.2.0`</summary> <pre> 2023-11-17 13:33:05,773 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource java.lang.RuntimeException: IOException encountered while reading from a byte array input stream at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:67) ~[graylog.jar:?] at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:49) ~[graylog.jar:?] at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:29) ~[graylog.jar:?] at com.mongodb.internal.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52) ~[graylog.jar:?] at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60) ~[graylog.jar:?] at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87) ~[graylog.jar:?] at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42) ~[graylog.jar:?] at org.bson.internal.LazyCodec.decode(LazyCodec.java:53) ~[graylog.jar:?] at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:104) ~[graylog.jar:?] at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63) ~[graylog.jar:?] at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87) ~[graylog.jar:?] at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42) ~[graylog.jar:?] at com.mongodb.internal.connection.ReplyMessage.<init>(ReplyMessage.java:50) ~[graylog.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.getCommandResult(InternalStreamConnection.java:538) ~[graylog.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:423) ~[graylog.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:340) ~[graylog.jar:?] at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:116) ~[graylog.jar:?] at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:643) ~[graylog.jar:?] at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71) ~[graylog.jar:?] at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:206) ~[graylog.jar:?] at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:119) ~[graylog.jar:?] at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:85) ~[graylog.jar:?] at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:75) ~[graylog.jar:?] at com.mongodb.internal.connection.DefaultServer$OperationCountTrackingConnection.command(DefaultServer.java:293) ~[graylog.jar:?] at com.mongodb.internal.operation.CommandOperationHelper.createReadCommandAndExecute(CommandOperationHelper.java:233) ~[graylog.jar:?] at com.mongodb.internal.operation.FindOperation.lambda$execute$1(FindOperation.java:332) ~[graylog.jar:?] at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$0(OperationHelper.java:356) ~[graylog.jar:?] at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:381) ~[graylog.jar:?] at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$1(OperationHelper.java:355) ~[graylog.jar:?] at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:381) ~[graylog.jar:?] at com.mongodb.internal.operation.OperationHelper.withSourceAndConnection(OperationHelper.java:354) ~[graylog.jar:?] at com.mongodb.internal.operation.FindOperation.lambda$execute$2(FindOperation.java:329) ~[graylog.jar:?] at com.mongodb.internal.async.function.RetryingSyncSupplier.get(RetryingSyncSupplier.java:67) ~[graylog.jar:?] at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:340) ~[graylog.jar:?] at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:79) ~[graylog.jar:?] at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:191) ~[graylog.jar:?] at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:168) ~[graylog.jar:?] at com.mongodb.DBCursor.initializeCursor(DBCursor.java:810) ~[graylog.jar:?] at com.mongodb.DBCursor.hasNext(DBCursor.java:163) ~[graylog.jar:?] at org.mongojack.DBCursor.hasNext(DBCursor.java:255) ~[graylog.jar:?] at java.util.Iterator.forEachRemaining(Unknown Source) ~[?:?] at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?] at java.util.stream.ReduceOps$5.evaluateSequential(Unknown Source) ~[?:?] at java.util.stream.ReduceOps$5.evaluateSequential(Unknown Source) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?] at java.util.stream.ReferencePipeline.count(Unknown Source) ~[?:?] at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSortWithGrandTotal(PaginatedDbService.java:200) ~[graylog.jar:?] at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSort(PaginatedDbService.java:181) ~[graylog.jar:?] at org.graylog.events.processor.DBEventDefinitionService.searchPaginated(DBEventDefinitionService.java:65) ~[graylog.jar:?] at org.graylog.events.rest.EventDefinitionsResource.getPage(EventDefinitionsResource.java:189) ~[graylog.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:134) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:177) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:81) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?] at java.lang.Thread.run(Unknown Source) [?:?] Caused by: com.fasterxml.jackson.databind.JsonMappingException: Null field at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@221e4a5a; pos: 260] (through reference chain: org.graylog.events.processor.$AutoValue_EventDefinitionDto$Builder["config"]->org.graylog.events.processor.aggregation.AutoValue_AggregationEventProcessorConfig$Builder["series"]->java.util.ArrayList[0]->org.graylog.plugins.views.search.searchtypes.pivot.series.AutoValue_Cardinality$Builder["field"]) at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:630) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:618) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:173) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:167) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4801) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3043) ~[graylog.jar:?] at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?] at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?] ... 79 more Caused by: java.lang.NullPointerException: Null field at org.graylog.plugins.views.search.searchtypes.pivot.series.AutoValue_Cardinality$Builder.field(AutoValue_Cardinality.java:118) ~[graylog.jar:?] at jdk.internal.reflect.GeneratedMethodAccessor391.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:170) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserializeFromObject(BuilderBasedDeserializer.java:338) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:240) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170) ~[graylog.jar:?] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:167) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217) ~[graylog.jar:?] at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4801) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3043) ~[graylog.jar:?] at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?] at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?] ... 79 more </pre> </details> ## Possible Solution Luckily, `5.2` doesn't allow saving the event definition though because it fails with an error, however we should * handle broken even definitions inherited from `< 5.2` versions by adding a migration that removes the offending `card` function and disables the event definition * validate the `card()` function and show that a field is missing ## Steps to Reproduce (for bugs) 1. On `5.1.5`: save an event definition with an aggregation and a `card()` function without a field 2. Upgrade to `5.2.0` 3. Try to open the event definitions page ## Context [HS-2076449047] ## Your Environment * Graylog Version: `5.1.5`, `5.2.0` * Java Version: * OpenSearch Version: * MongoDB Version: * Operating System: * Browser version:

Easiest solution is to delete the invalid event definition and recreate it.

system · January 29, 2024, 9:08am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Fetching event definitions failed. Internal Server Error Graylog Central (peer support) components	7	2819	March 14, 2023
Users and Teams / Users Overview "IOException encountered while reading from a byte array input stream" Graylog Central (peer support) sidecar	8	1366	September 29, 2021
Event Definitions cannot loading Graylog Central (peer support)	10	1983	April 19, 2020
Could not retireve event definition - IOException encountered - Unrecognized field "updated_at" Graylog Central (peer support) mongodb	3	573	June 22, 2023
Welcome page-IOException encountered while reading from a byte array input stream Graylog Central (peer support)	9	370	December 18, 2023

IOEXception when checking alerts

Related topics