Stream error ERROR: org.graylog2.alerts.AlertScanner - Skipping alert check that threw an exception


(MC) #1

Hi,

I created a new stream with an alert. When StuckThreadMaxTime is seen in the logs send out an email.When enabled,I see the following error on the logs. I believe is due to special characters and regex, but i can’t seem to find where. Any help with it would be appreciated.

2017-02-23 09:40:36,439 ERROR: org.graylog2.alerts.AlertScanner - Skipping alert check that threw an exception.
2017-02-23_17:40:36.50854 org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
2017-02-23_17:40:36.50899       at org.elasticsearch.action.search.AbstractSearchAsyncAction.onFirstPhaseResult(AbstractSearchAsyncAction.java:206) ~[graylog.jar:?]
2017-02-23_17:40:36.51032       at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:152) ~[graylog.jar:?]
2017-02-23_17:40:36.51072       at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:46) ~[graylog.jar:?]
2017-02-23_17:40:36.51241       at org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:184) ~[graylog.jar:?]
2017-02-23_17:40:36.51288       at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:174) ~[graylog.jar:?]
2017-02-23_17:40:36.51371       at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:122) ~[graylog.jar:?]
2017-02-23_17:40:36.51527       at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[graylog.jar:?]
2017-02-23_17:40:36.51680       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[graylog.jar:?]
2017-02-23_17:40:36.51731       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[graylog.jar:?]
2017-02-23_17:40:36.51778       at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[graylog.jar:?]
2017-02-23_17:40:36.51900       at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[graylog.jar:?]
2017-02-23_17:40:36.52510       at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[graylog.jar:?]
2017-02-23_17:40:36.52552       at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[graylog.jar:?]
2017-02-23_17:40:36.52660       at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[graylog.jar:?]
2017-02-23_17:40:36.52704       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[graylog.jar:?]
2017-02-23_17:40:36.52821       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[graylog.jar:?]
2017-02-23_17:40:36.52861       at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[graylog.jar:?]
2017-02-23_17:40:36.52921       at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[graylog.jar:?]
2017-02-23_17:40:36.53186       at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[graylog.jar:?]
2017-02-23_17:40:36.53339       at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[graylog.jar:?]
2017-02-23_17:40:36.53399       at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[graylog.jar:?]
2017-02-23_17:40:36.53662       at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[graylog.jar:?]
2017-02-23_17:40:36.53699       at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[graylog.jar:?]
2017-02-23_17:40:36.53757       at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[graylog.jar:?]
2017-02-23_17:40:36.53861       at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[graylog.jar:?]
2017-02-23_17:40:36.54060       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
2017-02-23_17:40:36.54095       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
2017-02-23_17:40:36.54149       at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
2017-02-23_17:40:36.54385 Caused by: org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: token_mgr_error: Lexical error at line 1, column 105.  Encountered: <EOF> after : ""
2017-02-23_17:40:36.54568       at org.apache.lucene.queryparser.classic.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1130) ~[graylog.jar:?]
2017-02-23_17:40:36.54613       at org.apache.lucene.queryparser.classic.QueryParser.jj_scan_token(QueryParser.java:589) ~[graylog.jar:?]
2017-02-23_17:40:36.54688       at org.apache.lucene.queryparser.classic.QueryParser.jj_3R_2(QueryParser.java:472) ~[graylog.jar:?]
2017-02-23_17:40:36.54897       at org.apache.lucene.queryparser.classic.QueryParser.jj_3_1(QueryParser.java:479) ~[graylog.jar:?]
2017-02-23_17:40:36.54955       at org.apache.lucene.queryparser.classic.QueryParser.jj_2_1(QueryParser.java:465) ~[graylog.jar:?]
2017-02-23_17:40:36.55277       at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:216) ~[graylog.jar:?]
2017-02-23_17:40:36.55431       at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:202) ~[graylog.jar:?]
2017-02-23_17:40:36.55470       at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:160) ~[graylog.jar:?]
2017-02-23_17:40:36.55586       at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:117) ~[graylog.jar:?]
2017-02-23_17:40:36.55633       at org.apache.lucene.queryparser.classic.MapperQueryParser.parse(MapperQueryParser.java:880) ~[graylog.jar:?]
2017-02-23_17:40:36.55759       at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:227) ~[graylog.jar:?]
2017-02-23_17:40:36.55792       at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:250) ~[graylog.jar:?]
2017-02-23_17:40:36.55946       at org.elasticsearch.index.query.BoolQueryParser.parse(BoolQueryParser.java:76) ~[graylog.jar:?]
2017-02-23_17:40:36.55988       at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:250) ~[graylog.jar:?]
2017-02-23_17:40:36.56081       at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:324) ~[graylog.jar:?]
2017-02-23_17:40:36.56246       at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:224) ~[graylog.jar:?]
2017-02-23_17:40:36.56306       at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:219) ~[graylog.jar:?]
2017-02-23_17:40:36.56403       at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33) ~[graylog.jar:?]
2017-02-23_17:40:36.56736       at org.elasticsearch.search.SearchService.parseSource(SearchService.java:856) ~[graylog.jar:?]
2017-02-23_17:40:36.56769       at org.elasticsearch.search.SearchService.createContext(SearchService.java:667) ~[graylog.jar:?]
2017-02-23_17:40:36.56838       at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:633) ~[graylog.jar:?]
2017-02-23_17:40:36.56998       at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:377) ~[graylog.jar:?]
2017-02-23_17:40:36.57068       at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368) ~[graylog.jar:?]
2017-02-23_17:40:36.57180       at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365) ~[graylog.jar:?]
2017-02-23_17:40:36.57252       at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[graylog.jar:?]
2017-02-23_17:40:36.57918       at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77) ~[graylog.jar:?]
2017-02-23_17:40:36.57975       at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:293) ~[graylog.jar:?]
2017-02-23_17:40:36.58162       at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[graylog.jar:?]
2017-02-23_17:40:36.58512       ... 3 more

(Jochen) #2

Hi Marsel,

Please post the Stream rules and the Alert condition you’re using.