Extractors and Searching: numeric range


#1

Hello,
First post but been using graylog for a while and really like it, thanks for the great posts they’ve been so much help. I feel like I’ve missed something so wanted to bring it to the community…

I have a log message coming in and a grok extractor that gets applied on it that should pull out a value, give it a custom field name and store it as a double. I then want to search where the custom field is between two values.
It seems to work as long as the range I search in is 0<= X <= 9.999 but as soon as I use a value like 10 it just returns what seems like random results.

Background
So given a message like this:
16/Nov/2018:15:08:56 +0000 "People who like goat cheese" "3.991"

And a grok extractor like the following:
%{HTTPDATE:timestamp} %{QS:message} "%{BASE10NUM:cheese_eaters:double}"

(Which if I test this in some online grok extractor or via the grok extractor editor in graylog seems to work as expected)

I expect to be able to search like this:
cheese_eaters:>2 AND cheese_eaters:<8

Which does seem to work as expected.

Problem
When I try to search using:
cheese_eaters:>10

I get a number of results, some that are greater than 10 but it seems to be a random selection.

What I’ve tried
Some reading suggested I use a pipeline to write a new field (or override the existing one) as a double, but when I do that using something like the following (note that IRL I’ve used a better condition check, this is just for illustrative purposes)

rule "Convert Values"
when
  has_field("cheese_eaters")
then
  // Initially it was this, but I wasn't convinced it was overwriting the value
  set_field("cheese_eaters", to_double($message.cheese_eaters));
 // So then I added this to see if it was writing them as expected... field just has '0' in so not convinced he above worked, although does prove the rule was running
  set_field("_cheese_eaters", to_double($message.cheese_eaters));
end

So… yeah I expect to be able to search using a range but when I use a numeric value in the condition of 10 or more I just don’t get expected results… am I missing something obvious? … or not obvious?


(Jan Doberstein) #2

are you able to check the elasticsearch mapping for this field? You need to retrieve the mapping of the index the messages are in. How can be seen in the documentation: http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings


(system) #3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.