Graylog search query - regex

[arnaudluti]

Hi everyone,

I need help about logs queries using regex…
W’ere using Graylog 4.2.3 and ES 7

I’ve followed the documentations Search query language - Searching and specially Regexp Query | Elasticsearch Reference [5.6] | Elastic.

For example, i’ve got in my logs many entries with the field “source” containing exactly
"FB100D-ADV-1" (with the double quotes).

If i take that field and click on “add to query”, the search bar is populated with:
source:\"FB100D\-ADV\-1\"

The documentation says that in regex, the characters “-” and " " " (double quote) must be escaped, so far i understand. Then i try to convert that query as regex, adding the surrounding “/”. I got:
source:/\"FB100D\-ADV\-[1-2]\"/

no results… you’ll understand that i search any entries with the field source equals “FB100D-ADV-1” or “FB100D-ADV-2”…

I tried equally that syntax, without success:

• source:\"FB100D\-ADV\-1\"
• source:/"FB100D\-ADV\-[1-2]"/
• source:\"FB100D\-ADV\-[1-2]\" (causes an error)

I don’t know where is my error !
Any ideas… ?

Thanks in advance!
Arnaud

[tmacgbay]

Played with it a bit but can’t come up with a good test failure… maybe you could try adding a + (one or more) or ? (zero or one) or {1} to the [1-2] to define it further?

source:/\"FB100D\-ADV\-[1-2]{1}\"/

[gsmith]

Hello @arnaudluti

I did 2 tests in my lab with a field that has double quotes. I also used regex to remove the quotes.

Not sure if that’s where you going with this.

Example 1:

Field with double quotes.

Created an extractor using regex to remove quotes.

Example 2:

Global search for 7 days.

test_forum:\"Enseva\-Lab\"

Results

I also have in my Graylog configuration file I added this line

allow_leading_wildcard_searches = true That was found here

[cawfehman]

try

source:/“FB100D-ADV-[1-2]”/

The issue you’re having is that you need to escape the hyphens when you are searching normally, but they don’t need to be escaped when you are doing a RegEx search as they are not listed as characters requiring escaping for RegEx in ES.

[arnaudluti]

Hello everyone,

Thanks for your answers.

@cawfehman @tmacgbay i tried your propositions, no results

Arnaud

[arnaudluti]

(i wanted to group with message with the other, but "new users cannot mention more that 2 users )

Hello @gsmith,

Thanks for your answer. I’ll give a try to extractors, but i don’t know if it can have a impact on the performances, since these messages are firewall traffic logs, with approximately 500 messages/second.

[cawfehman]

Have you tried expanding your time window, searching All Messages?

Also, what specific version of ES are you using? Graylog does not support 7.11+

[gsmith]

Hello,

@arnaudluti

but i don’t know if it can have a impact on the performances, since these messages are firewall traffic logs, with approximately 500 messages/second.

I have like 15 on my firewall Input about 300 to 500 mps.

I didn’t notice any performance issue yet Or have seen any major increase of CPU or Memory on the server. I have seen some community members talk about CPU issues but there GROK/Regex configuration were incorrect. To be honest from what I have seen @tmacgbay do with pipelines can be pretty impressive also. No matter what you do, if you have to use either one there maybe a chance of some type of increase of resources but if you keep it at a minimal level It shouldn’t be noticeable.

FYI… Here are my statistics on my ( only one ) lab firewall but I also some more and a tone of switches.