Search field:<value doesn't give the expected results

Graylog Central (peer support)
1

Hi,

I try to search with range “modifiers” as explained here: http://docs.graylog.org/en/2.5/pages/queries.html#syntax
it “works” as I get an answer(s), but not the correct ones…

The search results I get back are wrong/weird:

If I search on field:<1000, I get back matches for field: 0,1,10,100
If I search on field:>1000, I get back matches of all the other values for that field (both under and over 1000)

2

is that field stored as a number?

3

Yes, as in INT

Input -> Extractor -> Grok -> Grok pattern = %{INT:time_duration}

Graylogs deflector _mapping says:

 "time_duration": {
            "type": "keyword"
          },
4

but type keyword does not mean it is saved as int in elasticsearch - so that the range queries will not work.

change the mapping of that field and those kind of queries will be possible.

5

Yes, I see, I just assumed keyword was a good enough default, since elastic has chosen keyword for almost all fields except for date.
Is it recommended to always create custom mappings for all fields to avoid bad type-guessing by elastic?

6

the recommendation is to create a custom mapping where you expect a specific type of field.

1 Like
7

A follow-up question:

Now that the custom mapping was fixed and queries started to work correctly.
But sorting on the results yields this error:

Could not execute search

There was an error executing your search. Please check your Graylog server logs for more information.

Error Message:

Unable to perform search query No mapping found for [time_duration] in order to sort on

Details:

  • No mapping found for [time_duration] in order to sort on

Search status code:

500

Search response:

cannot GET /api/search/universal/relative?query=http_status_code%3A200&range=300&filter=streams%3A5c2f1e551cbec51028ada85e&limit=150&sort=time_duration%3Aasc (500)

I found this then I searched on the error message, but I don’t think it’s the same issue, as sorting on any “standard” fields (like “timestamp”, “source” etc.) works fine.

8

Do you have different mappings for the same field in the indices you search on?

9

In older indices the old mapping is still in there, but the search was only relative 300 last seconds, so I thought that the old data wouldn’t matter. (and it doesn’t exists in the oldest indices too…)

10

Hmm… old indices can’t be converted by PUTing a new type, it seems I have to reindex instead? How will that affect Graylogs default index if I need to reindex it?
There isn’t much mentioned about this in the Graylog docs?

This is sliding over more and more in elastic-land I guess…

11

I discussed this with another person, and we wonder if this could be a bug in either Graylog or Elastic, causing the error.

The last 10 indices have the correct custom mapping, but yet the relative search sorting fails… any ideas?

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.