Searching special characters

Graylog Central (peer support)
1

Hi all,

I try to understand escaping and wildcards but seem to miss something very basic.

For example I do not even manage to search for a “.” or “-“ character in the message field.
Neither one of the following expressions returns all messages, that contain the character:

.

“.”

“\.”

or

-

“-”

“\-”

Thanks for your help
Kirt

(I am using the virtual appliance Graylog 3.2.4+a407287)

2

Did you check docs?:
https://docs.graylog.org/en/3.2/pages/searching/query_language.html
https://www.elastic.co/guide/en/elasticsearch/guide/2.x/_wildcard_and_regexp_queries.html
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-regexp-query.html#regexp-syntax

3

According to https://docs.graylog.org/en/3.2/pages/searching/query_language.html I would have expected the following:

Searching for string “testing…”

Query: testing…
Expected: returns messages
Result: returns messages

Query: testing
Expected: not returning messages (as “…” is part of the string with no white space between)
Result: still returning messages

Query: *…
Expected: returning messages (as * should act as a wildcard for the substring “testing”)
Result: not returning messages

As my queries do not employ a regular expression, I would have expected “.” to be a standard character.

What am I missing?

Thanks
Kirt

4

Hi all,

any hints on how I do search for “testing…” without finding “testing”? (I.e, require the “…”).

I do not get the concept at all :frowning:

Thanks

Kirt

5

he @Kirt

your leading wildcard search will only work if you have that enabled in Graylog. When you search for the string - you should quote that string. Like described in the docs.

6

Thanks for the asnwer,Jan.

My example with the leading wildcard was not carfully picked. Sorry. Yes, I saw that leading wildcards have to be enabled.

What I do still not understand, is the very simple basic search for a string, that contains “…” (three points).

Text in the log e.g. is “Some text test…” (Without the quotes).

What will I have to enter into the search field to find all lines with “test…” but not with only “test”. (without the quotes).

Thanks
Kirt

7

he @Kirt

the main question is - do you search in the message field or do you search in a different field?

The message field is analyzed field that means that the content is split into terms based on the active tokenizer.

Overview:
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/analysis-tokenizers.html

Default Tokenizer
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/analysis-standard-tokenizer.html

As this is based of words (see links above) your dots are not searchable. Because they are no words.

Should the content be in a non analyzed field where the content is simple seen as a string, your search would be possible.

2 Likes
8

Thanks Jan! This explains what I see. I do indeed search the full_message and message fields.

After having a short look at the tokenizer: Wouldn’t it be better to use the whitespace tokenzier when dealing with log messages? While “test” and “test.” might be very close in a spoken language, it’s likely to be completely different in a log file. This is just asking your opinion.

Will there be another way to search for “test…” in analyzed field?

Thanks again, as I finally know the reason why my search is not working as expected.
Kirt

9

he @Kirt

you could change the tokenizer and check if that is better for you. We did not change the default tokenizer to not change/break the behaviour and make it inconsistence for running setups.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.