Expected numeric type on field but got [keyword]

[wobble_butt]

Hi guys, I see this issue being raised before, so I followed that advice and created a pipeline. The chart still throws the same error:
“Expected numeric type on field [monthly_sales], but got [keyword]”

At the end of my pipeline, i’m using the following print statement:

let debug_message = concat("Is monthly_sales numeric?", to_string(is_number($message.monthly_sales)));
debug(debug_message);

It all confirms as “true” in the logs… so then why would the chart still not recognise that? It doesn’t make sense! Are there any other steps I could take to check the “format” or “type” of a field?

Any help would be appreciated!

[gsmith]

Hello && Welcome

Unfortunately that is not enough data to help us, help you.
As shown below anything that could help us out to troubleshoot your issue.

• Operating system information
• Package versions
• Service logs
• Configuration files
• Environment variables such as Input’s, Extractors, Pipelines, etc…

@wobble_butt

I see this issue being raised before, so I followed that advice and created a pipeline.

Which post are you referring to?

@wobble_butt

The chart still throws the same error:

What kind of chart and how is it configured?

EDIT: I’m not sure exactly how to fix your issue but I found why it’s showing that following error.

Expected numeric type on field but got [keyword]

This is an error when performing Metrics Aggregations such as avg and sum. The calculation of mathematical dimensions is expected to get a numeric field, but the keyword type obtained is a String.

You could try to configure the mapping before the index data, define the value of this field as numeric, or turn on the Numeric detection of the mapping.

Dynamic field mapping | Elasticsearch Guide [8.11] | Elastic

Either revert it to that or try to rotate active write index (System/Indices/Index Set/Maintenance in the Graylog web interface) see if that helps

[wobble_butt]

Thank you @gsmith, I was hoping to fix it using pipelines, is there no other way than mapping? Here are 2 similar answers which I’m basing this on:
2017
2019

The GELF payload is in the “message” field, and I use an JSON extractor to create all the other fields. Even though the monthly_sales value is sent as a float on my end, I assumed the JSON extractor was the one turning it into a string.

So after building the pipeline and forcing this field to be numeric, it’s very disheartening to see the avg() function in the charts still throwing this error.

Does mapping apply to this situation since the field is created by the extractor? Seems like i’m missing something really simple.

[gsmith]

Hello,

@wobble_butt

Does mapping apply to this situation since the field is created by the extractor?

I’m not a 100% sure but think it does. I believe this issue is on the mapping level.

Custom index mappings
EDIT: Did you read this that I posted earlier?

While JSON has support for native floating point and integer data types, some applications or languages may sometimes render numbers as strings. Usually the correct solution is to map these fields explicitly, but numeric detection (which is disabled by default) can be enabled to do this automatically:

curl -X PUT "localhost:9200/my-index-000001?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "numeric_detection": true
  }
}
'
curl -X PUT "localhost:9200/my-index-000001/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
  "my_float":   "1.0", 
  "my_integer": "1" 
}
'

Also I found this.

Convert processor | Elasticsearch Guide [8.11] | Elastic

Hope that helps

[wobble_butt]

I did a mappings call on my index:

curl -X GET "http://localhost:9200/my-index-01/_mapping?pretty"

and it’s all there. I can see all the fields, many that should be numeric say “keyword”, so thank you, that’s progress, I can finally see the problem.

Regarding the solution, they suggest creating a new index and reindexing since existing fields will be invalidated. That sounds like a much more difficult process than I’m confident in completing.

Do you have any experience with this? Is it just a warning, or is it guaranteed to fail?

[tmacgbay]

Once you have set the custom mapping to numeric and/or set the numeric_detection to true (and it works) you can use your current index but you must rotate the index and it will start trapping the field from that point onward as numeric… the previous indexes will still be keyword. If you want to correct your history to use it all, it’s possible… I posted my journey doing that here - it’s in an older version of Graylog/Elastic but I think it still holds. Be careful on production data!!!

[gsmith]

Hello,

@wobble_butt

Do you have any experience with this?

A little with elasticsearch, I’m still learning on the HowTo sections (mappings, templates)
If your having issues there are some really good people here that can help you further.

[wobble_butt]

So just to confirm that pipelines cannot fix this?

So as the project grows in future, every new field that is sent will have some default mapping associated with it (hopefully autodetected to numeric), but if I want to change the types, my options are:

1. Add them to the mapping of the index BEFORE they first arrive, or
2. Migrate to a whole new index with the desired changes.

Is this correct? No amount of extractors or pipelines can change the “type” of a field?

[gsmith]

Hello,

@wobble_butt

Add them to the mapping of the index BEFORE they first arrive, or

What I do know is, once you fix your mapping then I would manually rotate you indices. The new mapping should be good.

@wobble_butt

Migrate to a whole new index with the desired changes.

This is plausible, You could set the new Index as default also when your done.

@wobble_butt

No amount of extractors or pipelines can change the “type” of a field?

I have not see it yet,
Most everyone here helping has day/night jobs so give it a day + maybe someone else here might have a better idea or more information on this.

EDIT:
@tmacgbay stated earlier

@tmacgbay

Once you have set the custom mapping to numeric and/or set the numeric_detection to true (and it works) you can use your current index but you must rotate the index and it will start trapping the field from that point onward as numeric… the previous indexes will still be keyword. If you want to correct your history to use it all, it’s possible… I posted my journey doing that here - it’s in an older version of Graylog/Elastic but I think it still holds. Be careful on production data!!!

[wobble_butt]

After spending the weekend reading docs and building confidence, I created a custom index mapping and with much trepidation, rotated the index. It all went surprisingly smoothly and I’m quite content with the changes only applying from this point on.

Thank you @gsmith and @tmacgbay for your help. Glad to see Graylog handles all this stuff so effortlessly.

For anyone in the future that doesn’t understand why this problem even exists, this video helped me: How can the mapping be changed for an existing Elasticsearch index without downtime? - YouTube - TLDR: if you want to use pipelines and extractors to change an existing field’s data-type, then make sure those pipelines and extractors are creating a new field instead.

[gsmith]

That’s Great @wobble_butt , And thank you for sharing