Creating Extractor

Hello everyone,

can someone explain to me what I’m doing wrong?

I wanted to write an Extractor for the line:

nginx: 172.20.0.66 - - [25/Mar/2022:13:18:57 +0100] "POST /ifstats.php HTTP/2.0" 200 215 "https://172.20.0.1/" "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"

and setup my extractor the following way:

After that it adds the field source, but nothing else. Any Hints what I’m doing wrong?

Cheers,
Gamie

Hello && Welcome @Gamienator

It very hard to see you screen shoot even after I enlarge it.

What kind of extractor are you using (i.e. regex, grok, etc…)?
I assume you only want the field created if it has a specific string?
When this extractor was configured did you test it out , and were the results conclusive?

For example /w regex, shown in red box.

If this is correct, then I would look into Date/time when the message was sent and received.
Perhaps some testing on a message. Find the message that would have these details and test it against the extractor again, also look at timestamp. It might be extracting it in the future, so perhaps that’s why unable to see it. Just a guess.

EDIT: I put my glasses on and found something. Try using another field “NAME” instead of source, perhaps something like source1 or database1. It might be conflicting with your index template which also by default has a field called source.

Hey there,

I tried it. Still not working. Do I have to escape the spacebar? I don’t think so. I already checked for future messages and still not working. Maybe I should invest some time into GROK Pattern. :thinking: But this is also irritating for me :smiling_face_with_tear:

What did you try? and what was the out come?
Did you wait after the new configuration and give it enough time to generate the new fields?

Not sure what you mean, can you explain this?

Its your configuration made, show us what you did and perhaps a clearer screenshot. If your having this issue with now , I don’t think GROK might help you, Its a derivative from regular Regular expression.

To help you further we would need more information then " I tried it and does work".
Thanks

Sorry @gsmith,

was a long day for me yesterday but happy to see you answering :slight_smile:

So to get you up to date my upper example with nginx got a solution from the marketplace, but now I have to write an extractor for dhcp. Example Message:

<190>Mar 26 08:12:38 dhcpd[29645]: DHCPOFFER on 10.0.2.125 to 56:fa:3e:4a:a9:42 (UPCGW_A942) via vtnet0

As you suggested I tried a different NAME:

And after the time, it generates in the result only the field dhcp_offer

which indicates to me, that it only progress the index, that was written in the Target index and discards everything else on the message.

I hope this time the screenshots are clearer :slight_smile:

Hello,

Now I see your using a split Index extractor? I haven’t used that before.

I’m also seeing two different logs, One from Nginx and another that like like a firewall.

Correct me if I’m wrong, what needs to be done is extract the string DHCPD from the message field and create a field. Could use a regex extractor.

What I did was upload you last message above to my lab server.
Created a regex extractor.
Tested to insure the configuration were correct.

Example Completed

This is all I can do for you, perhaps it will help. If not you have to explain and show what your doing.

Thanks

Thanks again!

I invested a lot of time aswell and found the way to use GROK patterns yesterday. For me I thought Split Index is the same way python uses Split, by putting every word into a list :slight_smile:

But since Getting an idea how GROK works I can write my own extractors now :slight_smile: Thank you again!

1 Like

Awesome, Glad you got it to work :+1:

Couple things you should be aware of is sometimes if a GROK pattern is configured incorrect you may experience some issues like memory, dashboards loading slowly, etc… One thing I do like about GROK is I can save the patterns and use them on other INPUTs and/or Pipelines if needed. Lately I’ve have been only using Regex.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.