Documentation for extractors wrong?

[Fl4m3Ph03n1x (Pedro Miguel Pereira Serrano Martins)]

Background

I have been trying to implement some extractors on JSON messages, but I see differences between the documented behavior and app behavior.

Problem / Question

According to the docs:

Using the default settings, that message would be extracted into these fields:

details.tags

So, if I search for details.tags I will get the value of tags.

But in the app, this only works if I do details_tags.
Why?

[jochen]

The documentation is wrong. Well, outdated to be exacted.

See http://docs.graylog.org/en/2.4/pages/upgrade/graylog-2.0.html#dots-in-field-names for details about why dots in field names are not allowed in Graylog 2.x (starting with Graylog 2.0.0).

We’ll probably change this in Graylog 3.0.0 again:

Issue: Allow dots in field names again

And for even more rants context:

Issue: JSON extractor ignore keys with space or other non-alpha num characters

[Fl4m3Ph03n1x (Pedro Miguel Pereira Serrano Martins)]

Ok …
This poses a few interesting questions:

1. When will Graylog 3 come out? Do you have an ETA?
2. Are all the queries performed with the old dot notation replicable with the new underscore notation?
3. What would be the fix for this issue? Another documentation update, or wait for 3.0 ?

[jochen]

@Fl4m3Ph03n1x

• When will Graylog 3 come out? Do you have an ETA?

“When it’s done”, but pretty sure it’s sometime in 2018.

@Fl4m3Ph03n1x

• Are all the queries performed with the old dot notation replicable with the new underscore notation?

What do you mean exactly?

@Fl4m3Ph03n1x

1. What would be the fix for this issue? Another documentation update, or wait for 3.0 ?

We’ll have to update the documentation for Graylog 2.x, since it’s obviously incorrect.

[Fl4m3Ph03n1x (Pedro Miguel Pereira Serrano Martins)]

@jochen

• Are all the queries performed with the old dot notation replicable with the new underscore notation?

What do you mean exactly?

Fl4m3Ph03n1x:

Is the underscore notation as expressive as the dot notation? Or are there some queries that you can do with the dot notation that cannot be done with the underscore one? ( are there any limitations to using the _ notation? )

We’ll have to update the documentation for Graylog 2.x, since it’s obviously incorrect.

Alright, so I just need to update this page and it should all be fine, right?
Or am I missing something else?

http://docs.graylog.org/en/2.4/pages/extractors.html#using-the-json-extractor

[jochen]

@Fl4m3Ph03n1x

Is the underscore notation as expressive as the dot notation? Or are there some queries that you can do with the dot notation that cannot be done with the underscore one? ( are there any limitations to using the _ notation? )

I still don’t know what you mean. Could you provide some examples?

[Fl4m3Ph03n1x (Pedro Miguel Pereira Serrano Martins)]

@jochen

I still don’t know what you mean. Could you provide some examples?

My question comes mainly from my ignorance regarding Graylog2 internals, so it’s normal that perhaps for you it makes no sense at all.

Let’s say that I have the following search query:

obj.bananasCount -> returns the number of bananas in this obj

Now, lets say that when making a search in Graylog2 due to some internal limitation the following query wouldn’t work:

obj_bananasCount -> returns empty, because for some unknown reason to the user, Graylog2 internals can't convert the '_' character to a '.' character equivalent

I see this is possible ( albeit, not sure ) with properties whose name start with ‘_’. Like for example:

obj._orangeCount

How would you convert this?

obj__orangeCount ?

---

Regarding the docs I will see when I have time for a fix and a PR.