I am trying to write my first pipeline. This being a grok to extract the apache log message into fields.
Here is my pipeline attempt:
rule "apache grok"
when
has_field("application_name")
then
let apache_grok = grok(%{COMMONAPACHELOG}, to_string($message.message), true);
set_fields(apache_grok);
end
For some reason I am getting many errors to do with the formatting of the message. I am not sure what to do in this situation as I have pretty much copied the example from the documentation.
If anyone has any pointers for me that would be great
The error message also changes depending on where I insert whitespace. (even if it is not modifying the line) I am not sure if this is expected behaviour or not.
I use the following in my setup - as you can see you need to put the Grok Pattern into quotes (")
rule "extract nginx"
when
has_field("tags") AND contains(to_string($message.type), "nginx")
then
let message_field = to_string($message.message);
let action = grok(pattern: "%{COMBINEDAPACHELOG}", value: message_field, only_named_captures: true);
set_fields(action);
end
I am not sure if this is the right place to ask this or if I should start a new thread, but is it recommended to have another pipeline rule after this to route it to a stream ? Or to change the process ordering so that “message filter chain” runs before pipeline processor ?
it might also be worth pointing out that the graylog docs don’t say to include quotes for the grok function.
// Let "nginxaccessfields" hold the Map returned by the grok function
// Use the "set_fields" function to use the "nginxaccessfields" object to set individual field names and values.
let nginxaccessfields = grok(%{NGINXACCESS}, to_string($message.message), true);
set_fields(nginxaccessfields);
