Field Statistics, float numbers and Elastic "keyword" type

Hello, everyone!

After writing a Grok to extract values from a MariaDB Slow Query log, I noticed that the Field Statics was not validating one of the fields (mariadb_query_time) as a number. In the Grok they were filtered as “%{BASE10NUM:mariadb_query_time;float}”. Then, I changed it to “%{NUMBER:mariadb_query_time;float}”. Saved the Grok and waited for new messages to come.

When checking how these values were been stored, it was possible to see that it is correctly stored as a number for the new occurrences, for instance:

  "mariadb_slow_clientuser" : "graylog_db",
  **"mariadb_query_time"** : 21.133185,
  "mariadb_qc_hit" : "No",
  "mariadb_schema_name" : "nation",          
  "mariadb_lock_time" : 1.09E-4,

But when checking the mapping mariadb_query_time is shown as keyword:

     "mariadb_query" : {
        "type" : "keyword"
      },
      **"mariadb_query_time" : {**
      **  "type" : "keyword"**
     ** },**
      "mariadb_rows_affected" : {
        "type" : "keyword"
      },

I checked other threads with similar questions, but I’m in doubt if custom-index-mappings is what I need and how to apply it.

So, if anyone can help me I’d appreciate.

Thanks!

Updating:

I tried using the custom index mapping feature, but to no avail. Then, after some searches on Elastic’s documentation I tried the following:

$ curl -X PUT “localhost:9200/graylog_0/_mapping/_doc?pretty” -H ‘Content-Type: application/json’ -d’
{
“properties”: {
“mariadb_query_time”: {
“type”: “double”
}
}
}

{
“error” : {
“root_cause” : [
{
“type” : “illegal_argument_exception”,
"reason" : "mapper [mariadb_query_time] of different type, current_type [keyword], merged_type [double]"
}
],
“type” : “illegal_argument_exception”,
“reason” : “mapper [mariadb_query_time] of different type, current_type [keyword], merged_type [double]”
},
“status” : 400
}

I thought this would update my current value. I don’t have that many messages and re-indexing won’t be that painful, if that is the case.

Any help is appreciated.

Thanks!

he @rcapp

on first ingest elasticsearch guess the field type - if that type had changed for whatever reason, rotate the active write index and it will again guess.

To force a specific field type you need to have a custom mapping.

What will not work at all is changing that after ingest.

Hi, @jan!

Thank you for your support: re-indexing made the trick.

I wish I could re-index the others, but there are only 15 msgs on the old index, nothing that we can’t add latter on an external report.

Cheers!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.