Graylog 3.2 Regex Extractors FAILING BADLY

Graylog Central (peer support)
1

So I’ve looked at all community resolution for this, and not a single valid answer. I’m hoping someone could hint out what I’m doing wrong.

I’ve created a regex extractor against a line like this:

May 07 12:20:15 [matchmakinga175e0] [Game] [d0d27210-c0b2-4d59-97ee-8dbb627128e9]Network Error: True

using Regex:
^.matchmaking.Game].[.](.+?) Error: True.*$

  1. Confirms Regex Success
  2. Selected Only attempt extraction if field matches regular expression

There’s TWO MAJOR ISSUES here:

  1. The field " mmstagneterror" doesn’t appear under search, (well it doesn’t appear anywhere)
  2. I’m seeing in the Extractor Details: 0 hits, 1604 misses
    sc1
    sc11465×777 72.3 KB

I even doubted myself and checked my regex here: https://www.freeformatter.com/java-regex-tester.html and it works perfectly.

So Anyone knows what’s happening here, cuz this is honestly driving me insane.

Thanks in Advance, this is going to be big help indeed. Thanks again

2

See if I can help, but I need a bit of a clarification.

Which part of the the message string are you trying to extract? Just the piece before the word Error? In this case network “Network” or something else/more?

Can you expand on it doesn’t appear anywhere? Is it not listed in your fields on your Search tab if click the field list button on the left navigation bar? Make sure you are not filtering on any streams.

image

I believe it’s not showing up because you haven’t had any hits against it and it hasn’t had cause to create it. But I’m not 100% on that.

3

Hi @cawfehman, thanks for replying.
Yes, It’s not showing in:

  1. fields where your arrow is pointing
  2. Not “autocompleting” when entering the field name in query search bar
  3. The line does indeed exist and is there, and I can find it by simply typing “Network Error”
  4. Yes, the word I’m looking for is “Network” … not that I am, but for the sake of this example issue.

Do you need more clarifications, screenshots … etc … Im happy to provide anything you’d need. Thanks again.

4

I discovered why that’s the case and as simple as this would sound, it was a major issue for me.

So the punchline is … if you create extractors, they’ll never work on previously ingested logs, they ONLY and ONLY work on new logs coming in AFTER the extractor has been created. So yes all previous log data cannot be filtered based on new extractor fields. So this is NOT a bug, it’s just the way graylog is. and It’d make sense really otherwsie, the system will require massive CPU and memeory resources to run your extractors against every single log line ever ingested.

Thanks @cawfehman for replying … I appreciate the time you took to respond. Many thanks man.

1 Like
5

Glad you figured it out… this is where I was going. I was going to mention that the hit/misses increasing tell you that the extractor is being run against messages, just nothing is matching the pattern in your case.

good luck

1 Like
6

Hey @cawfehman , the pattern matches perfectly, re-read my reply bro … it’s the fact that extractors don’t parse “ALREADY INGESTED” logs, they only work on new ones.

so don’t expect the fields to show up when new lines haven’t yet been ingested.

1 Like
7

If it makes you feel any better this is also true of other log management tools such as ManageEngine Event Log Analyzer. I spent a couple of days pulling my hair out before it was explained to me.

8

@bungo63 … that’s true.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.