Hey All,
We recently upgraded our graylog installation from version 3.0.2 to 3.1.2 and found that we can no longer sort by any column other then Timestamp and source when running a search.
After finding this we rolled our production graylog back to a snapshot (version 3.0.2) and we installed a fresh instance of graylog 3.1.2 on a new install of ubuntu 18.04. We found it to be doing the same thing. The Lab has just a single windows DC logging to graylog using winlogbeat and the sidecar.
For an example of the issue when we search and sort by [event_id] we get:
** Could not execute search**
There was an error executing your search. Please check your Graylog server logs for more information.
Error Message:
Unable to perform search query No mapping found for [event_id] in order to sort onNo mapping found for [event_id] in order to sort onNo mapping found for [event_id] in order to sort onNo mapping found for [event_id] in order to sort on
Details:
No mapping found for [event_id] in order to sort on
Judging by the error its a mapping issue in elasticsearch, but after searching for a few days and finding no solution I thought I might see if anyone could provide some guidance as to how to get this working again.
Thanks,
Elasticsearch Log below: /var/log/elasticsearch/graylog.log
[2019-10-17T04:05:56,923][INFO ][o.e.c.m.MetaDataMappingService] [MTkRQN-] [graylog_2/yQiF08xqT327Qrn9F0UE6g] update_mapping [message]
[2019-10-17T04:05:56,954][INFO ][o.e.c.m.MetaDataMappingService] [MTkRQN-] [graylog_2/yQiF08xqT327Qrn9F0UE6g] update_mapping [message]
[2019-10-17T18:29:58,303][DEBUG][o.e.a.s.TransportSearchAction] [MTkRQN-] [gl-events_0][0], node[MTkRQN-VSzqE7VHo6qRPTw], [P], s[STARTED], a[id=vaKH3HFeSp6Mr6nmkOph2Q]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indic$
org.elasticsearch.transport.RemoteTransportException: [MTkRQN-][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.index.query.QueryShardException: No mapping found for [computer_name] in order to sort on
at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:321) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:153) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:808) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.createContext(SearchService.java:637) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:596) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:387) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.access$100(SearchService.java:126) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:359) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:355) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1107) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.8.3.jar:6.8.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
[2019-10-17T18:29:58,303][DEBUG][o.e.a.s.TransportSearchAction] [MTkRQN-] [gl-events_0][1], node[MTkRQN-VSzqE7VHo6qRPTw], [P], s[STARTED], a[id=1dx9jXAaSEOYcoRDscfxoQ]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indic$
org.elasticsearch.transport.RemoteTransportException: [MTkRQN-][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.index.query.QueryShardException: No mapping found for [computer_name] in order to sort on
at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:321) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:153) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:808) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.createContext(SearchService.java:637) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:596) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:387) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.access$100(SearchService.java:126) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:359) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:355) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1107) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.8.3.jar:6.8.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
[2019-10-17T18:29:58,305][DEBUG][o.e.a.s.TransportSearchAction] [MTkRQN-] [gl-events_0][2], node[MTkRQN-VSzqE7VHo6qRPTw], [P], s[STARTED], a[id=hULZ9-LKTYix_VHd-2w5jA]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indic$
org.elasticsearch.transport.RemoteTransportException: [MTkRQN-][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.index.query.QueryShardException: No mapping found for [computer_name] in order to sort on
at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:321) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:153) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:808) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.createContext(SearchService.java:637) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:596) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:387) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService.access$100(SearchService.java:126) ~[elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:359) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:355) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1107) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.8.3.jar:6.8.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.8.3.jar:6.8.3]