Search field:<value doesn't give the expected results


#1

Hi,

I try to search with range “modifiers” as explained here: http://docs.graylog.org/en/2.5/pages/queries.html#syntax
it “works” as I get an answer(s), but not the correct ones…

The search results I get back are wrong/weird:

If I search on field:<1000, I get back matches for field: 0,1,10,100
If I search on field:>1000, I get back matches of all the other values for that field (both under and over 1000)


(Jan Doberstein) #2

is that field stored as a number?


#3

Yes, as in INT

Input -> Extractor -> Grok -> Grok pattern = %{INT:time_duration}

Graylogs deflector _mapping says:

 "time_duration": {
            "type": "keyword"
          },

(Jan Doberstein) #4

but type keyword does not mean it is saved as int in elasticsearch - so that the range queries will not work.

change the mapping of that field and those kind of queries will be possible.


#5

Yes, I see, I just assumed keyword was a good enough default, since elastic has chosen keyword for almost all fields except for date.
Is it recommended to always create custom mappings for all fields to avoid bad type-guessing by elastic?


(Jan Doberstein) #6

the recommendation is to create a custom mapping where you expect a specific type of field.


#7

A follow-up question:

Now that the custom mapping was fixed and queries started to work correctly.
But sorting on the results yields this error:

Could not execute search

There was an error executing your search. Please check your Graylog server logs for more information.

Error Message:

Unable to perform search query No mapping found for [time_duration] in order to sort on

Details:

  • No mapping found for [time_duration] in order to sort on

Search status code:

500

Search response:

cannot GET /api/search/universal/relative?query=http_status_code%3A200&range=300&filter=streams%3A5c2f1e551cbec51028ada85e&limit=150&sort=time_duration%3Aasc (500)

I found this then I searched on the error message, but I don’t think it’s the same issue, as sorting on any “standard” fields (like “timestamp”, “source” etc.) works fine.


(Jan Doberstein) #8

Do you have different mappings for the same field in the indices you search on?


#9

In older indices the old mapping is still in there, but the search was only relative 300 last seconds, so I thought that the old data wouldn’t matter. (and it doesn’t exists in the oldest indices too…)


#10

Hmm… old indices can’t be converted by PUTing a new type, it seems I have to reindex instead? How will that affect Graylogs default index if I need to reindex it?
There isn’t much mentioned about this in the Graylog docs?

This is sliding over more and more in elastic-land I guess…


#11

I discussed this with another person, and we wonder if this could be a bug in either Graylog or Elastic, causing the error.

The last 10 indices have the correct custom mapping, but yet the relative search sorting fails… any ideas?


(system) closed #12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.