Grok Pattern ExtractorNot Extracting

Graylog Central (peer support)
1

Interesting issue I am having right now. I have a Grok Pattern, tests perfect with current messages in the editor, Details show:

Metrics

210,743 total invocations since boot, averages: 362.98, 313.82, 172.58.

210968 hits, 0 misses

But, I am not seeing the extracted fields in search. Any thoughts?

Thanks.

2

Are you naming your captures in your Grok Pattern and have “Named captures only” checked in the configuration? Can you post the GROK and relevant extractor settings?

3

Grok Pattern is:
%{DATA:source} %{DATA:dvc} Protocol: %{WORD:proto}, SrcIP: %{IP:src_addr}(, OriginalClientIP: %{DATA:UNWANTED})?, DstIP: %{IP:dst_addr}, SrcPort: %{INT:src_port}, DstPort: %{INT:dst_port}, TCPFlags: %{BASE16NUM:UNWANTED}, IngressZone: %{DATA:in_zone}, EgressZone: %{DATA:out_zone}, DE: %{DATA:UNWANTED}, Policy: %{DATA:policy}, ConnectType: %{WORD:UNWANTED}, AccessControlRuleName: %{DATA:ac_rule_name}, AccessControlRuleAction: %{WORD:ac_action}

I do not have Named Captures Only checked.

Other Settings are:
Extractor type: Grok pattern

Source field : message

Condition: Only attempt extraction if field contains string

Field contains string: DstIP

Extraction strategy: Copy

And then a title for the Extractor.

I have tried both Cut and Copy, and Have tried with Named Checked and Unchecked.

I also have started to notice that when I restarted Graylog, Initially I would see the fields and data, until I started getting a Kafka file lock error in the /var/log/graylog-server/server.log

4

We aren’t set up with Kafka so I guess I won’t be much help - I poked around a bit and found more on Kafka file locks Graylog Kafka file lock hope that gives you some leads! Good luck!

5

So it seems, I just have to wait like 12 hours for the fields to start showing data and extraction. Weird that it would take that long. but it is working now.

6

yoou should check how much data is in your journal of Graylog and if you have all servers in the same timezone … and the messages are stored with the correct date.

7

I am double checking the sources timezone, but I was also overwriting with current date/time. Only one Graylog server is what we have.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.