Possible regression: Geo points and Graylog 7.2 and Opensearch 3.6.0

Graylog Central (peer support)
1

1. Describe your incident:

I have updated my cluster to 7.2 and Opensearch 3.6.0. I was running OS 2.19.6 before.

I have an input that receives JSON data which includes some geo points. An example of a plain JSON received on the input:

{"sequenceNumber": 74116290213, "frequency": 7076059, "mode": "FT8", "sNR": -5, "flowStartSeconds": 1793008588, "senderCallsign": "XXXX", "senderLocator": "DM33WP", "receiverCallsign": "YYYYY", "receiverLocator": "EM74pb92", "receiverDecoderSoftware": "JTDX v2.2.159 i", "band": 40, "senderCountryName": "United States", "receiverCountryName": "United States", "senderAdifCC": 291, "receiverAdifCC": 291, "senderCoords": [33.645833333333336, -112.125], "receiverCoords": [34.05208333333333, -84.67083333333333], "distance": 2528.099522984552, "heading": 81.25059617479047, "tags": []}

When using Opensearch 2.19.6 and Graylog < 7.2 (I am not sure which of both changes caused the trouble, I didn’t use this input for some time) I had a field type profile that converted receiverCoords and senderCoords to a Geo Point. It worked, I was able to represent the data on a map.

Now I have reactvated the input. When checking the field type on the data stream the type appears as “unknown”.

That said, the map aggregation works. But Graylog shows the field type as “unknown”. It also works with a new field I have created converting the receiverCoords field to a string using to_string.

And, indeed, looking at the indexes with Opensearch Dashboards, the receiverCoords and senderCoords fields have the correct geo_point type.

I think in the past Graylog showed the field type properly.

Of course I have made sure that I didn’t have any field contamination rotating the index and deleting all of the old ones.

Anyway, I am worried that at some point in the future the map widget may fail because of the “unknown” field type.

2. Describe your environment:

  • OS Information: Unsupported (FreeBSD)

  • Package Version: 7.2.1

  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?

Ignoring the “unknown” field type the map widget actually works.

4. How can the community help?

Has anyone else observed this behavior?

2

Hi Borjam

This sounds like your field type profile has stopped working, likely something deprecated in there that OS 3.x can’t read. We’ve reproduced your issue locally, luckily it looks like the map widget is not sensitive to the field type and just looks at the data formatting.

I’ll post the associated GH issue to track this problem here once raised. Thanks for letting us know!

3

Thanks for the answer.

Being just a test application, le me know if you need me to test anything. I can destroy the index, recreate, change the mappings, etc without casualties :wink:

4

Can track this issue here.