Some field show NaN and errors

Uporaba · September 6, 2019, 12:06pm

Some data is not displayed. How can one determine why? How can i fix this?

I still have some problems. why do they arise?

i Tryed it Some statistics still NaN? but is didnt work (
But if i use curl -X GET 'http://localhost:9200/graylog_deflector/_mapping?pretty it is displayed:

{
  "graylog_113" : {
    "mappings" : {
      "message" : {
        "dynamic_templates" : [
          {
            "internal_fields" : {
              "match" : "gl2_*",
              "mapping" : {
                "type" : "keyword"
              }
            }
          },
          {
            "store_generic" : {
              "match_mapping_type" : "string",
              "mapping" : {
                "type" : "keyword"
              }
            }
          }
        ],
        "properties" : {
          "Output" : {
            "type" : "keyword"
          },
          "RC" : {
            "type" : "keyword"
          },
          "Results" : {
            "type" : "keyword"
          },
          "Source" : {
            "type" : "keyword"
          },
          "access_token" : {
            "type" : "keyword"
          },
          "agreementnumber" : {
            "type" : "keyword"
          },
          "api_errorcode" : {
            "type" : "keyword"
          },
          "apiname" : {
            "type" : "keyword"
          },
          "apistatus" : {
            "type" : "keyword"
          },
          "args" : {
            "type" : "keyword"
          },
          "body_bytes_sent" : {
            "type" : "long"
          },
          "city" : {
            "type" : "keyword"
          },
          "city_domain" : {
            "type" : "keyword"
          },
          "citydomain" : {
            "type" : "keyword"
          },
          "client_mac" : {
            "type" : "keyword"
          },
          "container_id" : {
            "type" : "keyword"
          },
          "facility" : {
            "type" : "keyword"
          },
          "from_nginx" : {
            "type" : "keyword"
          },
          "full_message" : {
            "type" : "text",
            "analyzer" : "standard"
          },
          "gl2_remote_ip" : {
            "type" : "keyword"
          },
          "gl2_remote_port" : {
            "type" : "keyword"
          },
          "gl2_source_input" : {
            "type" : "keyword"
          },
          "gl2_source_node" : {
            "type" : "keyword"
          },
          "host" : {
            "type" : "keyword"
          },
          "http_referrer" : {
            "type" : "keyword"
          },
          "http_status" : {
            "type" : "keyword"
          },
          "http_user_agent" : {
            "type" : "keyword"
          },
          "http_x_forwarded_for" : {
            "type" : "keyword"
          },
          "json" : {
            "type" : "keyword"
          },
          "lb-fqdn" : {
            "type" : "keyword"
          },
          "level" : {
            "type" : "long"
          },
          "message" : {
            "type" : "text",
            "analyzer" : "standard"
          },
          "nas_ip" : {
            "type" : "keyword"
          },
          "nginx_access" : {
            "type" : "keyword"
          },
          "nginx_error" : {
            "type" : "keyword"
          },
          "oauthclient" : {
            "type" : "keyword"
          },
          "pkg" : {
            "type" : "keyword"
          },
          "platform" : {
            "type" : "keyword"
          },
          "port_index" : {
            "type" : "keyword"
          },
          "remote_addr" : {
            "type" : "keyword"
          },
          "request" : {
            "type" : "keyword"
          },
          "request_id" : {
            "type" : "keyword"
          },
          "request_method" : {
            "type" : "keyword"
          },
          "request_time" : {
            "type" : "float"
          },
          "response_status" : {
            "type" : "long"
          },
          "response_time" : {
            "type" : "keyword"
          },
          "response_time_nums" : {
            "type" : "float"
          },
          "server" : {
            "type" : "keyword"
          },
          "source" : {
            "type" : "text",
            "analyzer" : "analyzer_keyword",
            "fielddata" : true
          },
          "streams" : {
            "type" : "keyword"
          },
          "switch_ip" : {
            "type" : "keyword"
          },
          "switch_mac" : {
            "type" : "keyword"
          },
          "switch_port" : {
            "type" : "keyword"
          },
          "system" : {
            "type" : "keyword"
          },
          "time_to_live" : {
            "type" : "keyword"
          },
          "timestamp" : {
            "type" : "date",
            "format" : "yyyy-MM-dd HH:mm:ss.SSS"
          },
          "upstream_addr" : {
            "type" : "keyword"
          },
          "upstream_cache_status" : {
            "type" : "keyword"
          },
          "upstream_connect_time" : {
            "type" : "keyword"
          },
          "upstream_header_time" : {
            "type" : "keyword"
          },
          "upstream_response_time" : {
            "type" : "keyword"
          },
          "url_without_param" : {
            "type" : "keyword"
          },
          "user_agent" : {
            "type" : "keyword"
          },
          "value" : {
            "type" : "keyword"
          },
          "x-project" : {
            "type" : "keyword"
          },
          "x_forwarded_for" : {
            "type" : "keyword"
          },
          "x_real_ip" : {
            "type" : "keyword"
          },
          "x_real_ipv6" : {
            "type" : "keyword"
          },
          "x_request_id" : {
            "type" : "keyword"
          }
        }
      }
    }
  }
}
{
  "graylog_113" : {
    "mappings" : {
      "message" : {
        "dynamic_templates" : [
          {
            "internal_fields" : {
              "match" : "gl2_*",
              "mapping" : {
                "type" : "keyword"
              }
            }
          },
          {
            "store_generic" : {
              "match_mapping_type" : "string",
              "mapping" : {
                "type" : "keyword"
              }
            }
          }
        ],
        "properties" : {
          "Output" : {
            "type" : "keyword"
          },
          "RC" : {
            "type" : "keyword"
          },
          "Results" : {
            "type" : "keyword"
          },
          "Source" : {
            "type" : "keyword"
          },
          "access_token" : {
            "type" : "keyword"
          },
          "agreementnumber" : {
            "type" : "keyword"
          },
          "api_errorcode" : {
            "type" : "keyword"
          },
          "apiname" : {
            "type" : "keyword"
          },
          "apistatus" : {
            "type" : "keyword"
          },
          "args" : {
            "type" : "keyword"
          },
          "body_bytes_sent" : {
            "type" : "long"
          },
          "city" : {
            "type" : "keyword"
          },
          "city_domain" : {
            "type" : "keyword"
          },
          "citydomain" : {
            "type" : "keyword"
          },
          "client_mac" : {
            "type" : "keyword"
          },
          "container_id" : {
            "type" : "keyword"
          },
          "facility" : {
            "type" : "keyword"
          },
          "from_nginx" : {
            "type" : "keyword"
          },
          "full_message" : {
            "type" : "text",
            "analyzer" : "standard"
          },
          "gl2_remote_ip" : {
            "type" : "keyword"
          },
          "gl2_remote_port" : {
            "type" : "keyword"
          },
          "gl2_source_input" : {
            "type" : "keyword"
          },
          "gl2_source_node" : {
            "type" : "keyword"
          },
          "host" : {
            "type" : "keyword"
          },
          "http_referrer" : {
            "type" : "keyword"
          },
          "http_status" : {
            "type" : "keyword"
          },
          "http_user_agent" : {
            "type" : "keyword"
          },
          "http_x_forwarded_for" : {
            "type" : "keyword"
          },
          "json" : {
            "type" : "keyword"
          },
          "lb-fqdn" : {
            "type" : "keyword"
          },
          "level" : {
            "type" : "long"
          },
          "message" : {
            "type" : "text",
            "analyzer" : "standard"
          },
          "nas_ip" : {
            "type" : "keyword"
          },
          "nginx_access" : {
            "type" : "keyword"
          },
          "nginx_error" : {
            "type" : "keyword"
          },
          "oauthclient" : {
            "type" : "keyword"
          },
          "pkg" : {
            "type" : "keyword"
          },
          "platform" : {
            "type" : "keyword"
          },
          "port_index" : {
            "type" : "keyword"
          },
          "remote_addr" : {
            "type" : "keyword"
          },
          "request" : {
            "type" : "keyword"
          },
          "request_id" : {
            "type" : "keyword"
          },
          "request_method" : {
            "type" : "keyword"
          },
          "request_time" : {
            "type" : "float"
          },
          "response_status" : {
            "type" : "long"
          },
          "response_time" : {
            "type" : "keyword"
          },
          "response_time_nums" : {
            "type" : "float"
          },
          "server" : {
            "type" : "keyword"
          },
          "source" : {
            "type" : "text",
            "analyzer" : "analyzer_keyword",
            "fielddata" : true
          },
          "streams" : {
            "type" : "keyword"
          },
          "switch_ip" : {
            "type" : "keyword"
          },
          "switch_mac" : {
            "type" : "keyword"
          },
          "switch_port" : {
            "type" : "keyword"
          },
          "system" : {
            "type" : "keyword"
          },
          "time_to_live" : {
            "type" : "keyword"
          },
          "timestamp" : {
            "type" : "date",
            "format" : "yyyy-MM-dd HH:mm:ss.SSS"
          },
          "upstream_addr" : {
            "type" : "keyword"
          },
          "upstream_cache_status" : {
            "type" : "keyword"
          },
          "upstream_connect_time" : {
            "type" : "keyword"
          },
          "upstream_header_time" : {
            "type" : "keyword"
          },
          "upstream_response_time" : {
            "type" : "keyword"
          },
          "url_without_param" : {
            "type" : "keyword"
          },
          "user_agent" : {
            "type" : "keyword"
          },
          "value" : {
            "type" : "keyword"
          },
          "x-project" : {
            "type" : "keyword"
          },
          "x_forwarded_for" : {
            "type" : "keyword"
          },
          "x_real_ip" : {
            "type" : "keyword"
          },
          "x_real_ipv6" : {
            "type" : "keyword"
          },
          "x_request_id" : {
            "type" : "keyword"
          }
        }
      }
    }
  }
}

and response_time is a float data "response_time_nums" : { "type" : "float" but somehow they are not displayed.

jan · September 6, 2019, 12:45pm

you should create a custom mapping, that this is always stored as number:

http://docs.graylog.org/en/3.1/pages/configuration/elasticsearch.html#custom-index-mappings

so all future datapoints are saved correctly.

Only if you have a range selected where the data comes out of the correct datapoints the statistic function will return anything.

Jan

Uporaba · September 9, 2019, 11:27am

I’m a little confused about what I need to do. Can you give me an example based on my data?

Uporaba · September 9, 2019, 11:34am

@jan

I didn’t change the settings to the ones you offered me. But now for some reason the data appeared. But if I put search last 8 hours the data does not appear:

in the interval from 5 minutes to 2 hours the data is.

Uporaba · September 9, 2019, 11:43am

when I enter curl -X GET 'http://localhost:9200/graylog_deflector/_mapping?pretty ' shows me that the request_time
field is of type float is this correct or not?

jan · September 9, 2019, 12:38pm

that happens because on index creation the first ingested value was a number … this will work if the fist ingested value is a number but will break if it becomes a string again - for whatever reason.

With a custom mapping you force a specific field type.

Uporaba · September 10, 2019, 9:29am

@jan why then after a search for 8 hours all breaking?

Uporaba · September 10, 2019, 9:29am

although if I put the search for 2 hours then everything is ok

Uporaba · September 10, 2019, 9:30am

@jan what diagnostic need to be carried out to find a problem?

jan · September 10, 2019, 9:31am

I guess you search in multiple indices and the type of that field is different in those indices.

Uporaba · September 10, 2019, 9:32am

@jan how then to fix this error on other indices?

Uporaba · September 10, 2019, 9:35am

@jan I checked. other indexes have the same type of float for this field

Uporaba · September 10, 2019, 11:00am

@jan I figured out what was the matter. How can I change the data type in old indexes?

jan · September 10, 2019, 11:22am

only by re-index the data.

That is not a task you can do within Graylog.

system · September 24, 2019, 11:22am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
NaN value for statistics Graylog Central (peer support)	2	743	March 10, 2020
Getting a NaN for mean value Graylog Central (peer support)	9	1350	January 5, 2019
Under this API, what can be used as "Filter" filter conditions Graylog Central (peer support)	2	531	September 2, 2018
Graylog missing field in json embedded object Graylog Central (peer support) pipeline-rules , debuggingpl	8	922	March 4, 2020
Error on service graylog-web , to start Graylog Central (peer support)	3	608	November 9, 2018

Some field show NaN and errors

Related topics