GROK Formatting

[fnk]

I have a basic authentication log formatted as follows:

2018-03-19 15:11:46,181 https-jsse-nio-443-exec-246 JiraServiceAccount 911x8252778x1 - 192.168.0.1 /rest/api/2/search The user ‘ServiceAccount’ has PASSED authentication.
2018-03-19 15:11:46,744 https-jsse-nio-443-exec-251 JiraServiceAccount 911x8252779x1 - 192.168.0.1 /rest/api/2/search The user ‘ServiceAccount’ has PASSED authentication.

I’m finding it easy enough to pull out the date but I just can’t figure out how to pull/ignore the https-jsse-nio-443 after it?

[macko003]

a. don’t log it in your app
b. cut extractor
c. pipeline

[jan (Jan Doberstein)]

Read the Docs: http://docs.graylog.org/en/2.5/pages/extractors.html#using-grok-patterns-to-extract-data

[fnk]

Thanks, somehow missed that in the documentation, it’s been helpful. Much further but banging my head against the wall at the moment! So close though!

I’m using the following pattern which works…

%{TIMESTAMP_ISO8601} %{DATA:UNWANTED} %{USERNAME:UNWANTED} %{DATA:UNWANTED} %{DATA:UNWANTED} %{IPV4} %{PATH}

I need to extract the username between quotes from “The user ‘ServiceAccount’ has PASSED authentication.” To do this i’ve set a grok pattern up called JIRASECUSER which is the following regex

(?<=The user ').*?(?=') DeleteEdit

Again this works in isolation as does the %{JIRASECUSER} on it’s own in the extractor.

As soon as I try to use the following the filter refuses to run:

%{TIMESTAMP_ISO8601} %{DATA:UNWANTED} %{USERNAME:UNWANTED} %{DATA:UNWANTED} %{DATA:UNWANTED} %{IPV4} %{PATH} %{JIRASECUSER}

Was looking for the server.log file on the OVA appliance but I can’t seem to find the log I need either.

Determined to get this sorted

[jan (Jan Doberstein)]

To find your logfile: http://docs.graylog.org/en/2.5/pages/configuration/file_location.html#omnibus-package

Check the docs …

---

Personal I would try to make the Grok Patterns, not that complex (you will want to modify the following and make it fit your local needs!)

%{TIMESTAMP_ISO8601:time} %{DATA:UNWANTED} - %{IPV4:request} %{PATH:path} %{GREEDYDATA:message} 

Check “capture named only” to have the following fields:

As I do not know how other possible messages can look I did not extract the username from the field message - that should be done in a second Pattern on the field - this way you get always the basics and you are able to work with the real message separately.