Sources Page - Could not load sources data - after Upgrade Graylog and Elasticsearch


(Gottschling) #1

Hi,

after upgrading Graylog to 2.3 and Elasticsearch from 2.3 to 5.5 the sources page did not display the sources.
Elasticsearch log shows following error:

[2017-08-25T16:09:54,801][DEBUG][o.e.a.s.TransportSearchAction] [elasticsearch-n15-01] All shards failed for phase: [query]
org.elasticsearch.ElasticsearchException$1: Fielddata is disabled on text fields by default. Set fielddata=true on [source] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
        at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:618) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:125) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:240) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:88) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:47) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.action.search.InitialSearchPhase$1.onFailure(InitialSearchPhase.java:156) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1067) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.transport.TcpTransport.lambda$handleException$16(TcpTransport.java:1467) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.transport.TcpTransport.handleException(TcpTransport.java:1465) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.transport.TcpTransport.handlerResponseError(TcpTransport.java:1457) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1401) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) [transport-netty4-5.5.2.jar:5.5.2]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.11.Final.jar:4.1.11.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

How can I enable field data or change ‘source’ to keyword type?

Thanks,

Andreas


(Jochen) #2

fielddata = true is already used for the source field in the index template created by Graylog:

Make sure that this index template is being used and that your indices (created after the upgrade to Elasticsearch 5.5.x) have the correct mapping:


(Gottschling) #3

HI,
the template shows the correct mapping:

          "source" : {
            "fielddata" : true,
            "analyzer" : "analyzer_keyword",
            "type" : "text"
          },
 

I had to be patient until Graylog/Elasticsearch created a new index with the correct mapping at least.Could it be possible, that the field type of ‘source’ has been changed from string to text but the fielddata wasn’t set during the upgrade of the open index?

  "graylog_34" : {
    "mappings" : {
      "message" : {
        "dynamic_templates" : [
          {
            "internal_fields" : {
              "match" : "gl2_*",
              "mapping" : {
                "index" : "not_analyzed",
                "type" : "string"
...
          "source" : {
            "type" : "string",
            "analyzer" : "analyzer_keyword"
          },
...
  "graylog_35" : {
    "mappings" : {
      "message" : {
        "dynamic_templates" : [
          {
            "internal_fields" : {
              "match" : "gl2_*",
              "mapping" : {
                "index" : "not_analyzed",
                "type" : "string"
...
          "source" : {
            "type" : "text",
            "analyzer" : "analyzer_keyword"
          },
...
  "graylog_36" : {
    "mappings" : {
      "message" : {
        "dynamic_templates" : [
          {
            "internal_fields" : {
              "match" : "gl2_*",
              "mapping" : {
                "index" : "not_analyzed",
                "type" : "string"
...
          "source" : {
            "type" : "text",
            "analyzer" : "analyzer_keyword"
          },
...
  "graylog_37" : {
    "mappings" : {
      "message" : {
        "dynamic_templates" : [
          {
            "internal_fields" : {
              "match" : "gl2_*",
              "mapping" : {
                "type" : "keyword"
...
          "source" : {
            "type" : "text",
            "analyzer" : "analyzer_keyword",
            "fielddata" : true
          },

Regards,

Andreas


(Jochen) #4

Yes, the field type changes from string to keyword or text (depending on some mapping attributes) when using Elasticsearch 5.x.
Also see https://www.elastic.co/guide/en/elasticsearch/reference/5.5/breaking_50_mapping_changes.html#_literal_string_literal_fields_replaced_by_literal_text_literal_literal_keyword_literal_fields

Existing indices will not be upgraded automatically (at least not by Graylog).

If you think this is a bug, please open an issue at https://github.com/Graylog2/graylog2-server/issues


(system) #5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.