Graylog alertconditions error

[mev9669 (M D)]

Hi,

I used the REST API to create alert conditions. But I entered a wrong field name in one of the conditions. Now When I tried to load the alert conditions in the UI, I get an an error in the log file.

ERROR [AlertConditionFactory] Could not load alert condition ‘testin index alert’ , invalid configuration detected.
ERROR [StreamServiceImpl] Skipping alert condition.
org.graylog2.plugin.configuration.ConfigurationException: Mandatory configuration field dateformat is missing or has the wrong data type
at org.graylog2.plugin.configuration.ConfigurationRequest.check(ConfigurationRequest.java:117) ~[graylog.jar:?]
at org.graylog2.alerts.AlertConditionFactory.createAlertCondition(AlertConditionFactory.java:63) ~[graylog.jar:?]
at org.graylog2.alerts.AlertServiceImpl.fromPersisted(AlertServiceImpl.java:170) ~[graylog.jar:?]
at org.graylog2.streams.StreamServiceImpl.getAlertConditions(StreamServiceImpl.java:285) ~[graylog.jar:?]
at org.graylog2.rest.resources.streams.alerts.AlertConditionsResource.lambda$all$2(AlertConditionsResource.java:69) ~[graylog.jar:?]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) [?:1.8.0_91]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) [?:1.8.0_91]
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) [?:1.8.0_91]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) [?:1.8.0_91]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) [?:1.8.0_91]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) [?:1.8.0_91]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:1.8.0_91]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) [?:1.8.0_91]
at org.graylog2.rest.resources.streams.alerts.AlertConditionsResource.all(AlertConditionsResource.java:77) [graylog.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_91]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_91]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

How can i delete this alert condition from the db as it is not available in the Web UI?

[jochen]

You can delete the alert from the “alerts” collection in MongoDB or use the Graylog REST API (DELETE /streams/{streamId}/alerts/conditions/{conditionId}).

[mev9669 (M D)]

I have multiple streams and I cannot find what stream they are on.

I checked the “alerts” collection in mongodb ant the collection contains the actual alerts rather than the alert conditions.

Sample

{ “_id” : ObjectId(“582c4920bc7c6b1a6c8195bc”), “triggered_at” : ISODate(“2016-11-16T11:55:12.292Z”), “description” : “Stream had 264 messages in the last 45 minutes with trigger condition more than 10 messages. (Current grace time: 0 minutes)”, “condition_parameters” : { “grace” : 0, “threshold_type” : “more”, “threshold” : 10, “time” : 45, “backlog” : 1 }, “stream_id” : “582c34efbc7c6b1a6c817ee0”, “condition_id” : “cd08d207-9d01-4abd-b363-fb6dabe1032c” }
{ “_id” : ObjectId(“582c495cbc7c6b1a6c8195fe”), “triggered_at” : ISODate(“2016-11-16T11:56:12.286Z”), “description” : “Stream had 4 messages in the last 1 minutes with trigger condition more than 2 messages. (Current grace time: 0 minutes)”, “condition_parameters” : { “grace” : 0, “threshold_type” : “more”, “threshold” : 2, “time” : 1, “backlog” : 1 }, “stream_id” : “582c34efbc7c6b1a6c817ee0”, “condition_id” : “e7a273dc-e6f3-4c38-b19a-893eca4552c2” }

[jochen]

@mev9669

I checked the “alerts” collection in mongodb ant the collection contains the actual alerts rather than the alert conditions.

You’re right. The alert conditions are stored as embedded documents in the stream definitions in the “streams” collection in MongoDB.

[mev9669 (M D)]

Thanks for your help.

Got it resolved.

Connected to MongoDB shell and ran command db.streams.find()
Copied the result to notepad++.
Matched the alert condition id in the collection.
Got the relevant stream id of the alert condition.
Used Graylog API to delete the alert conditions.

[jochen]

Could you please post the exact request you’ve been using to create the invalid alert conditions?

This shouldn’t be possible and has to be fixed. Ideally, you could create a bug report at https://github.com/Graylog2/graylog2-server/issues describing the issue.

[mev9669 (M D)]

The plugins are custom alert condition plugins.
I think I managed to create them because of the free text allowed in the REST API body post request in v2.2. I think this has been fixed in v2.3 and above. I will report the issue if I manage to reproduce this again.

Thanks