Pipeline rule to set facility and log level categories

Description of your problem

I was having issues with the default settings for the Syslog UDP input so I switched to using the Raw/Plaintext UDP input and will set the fields I want using pipelines/rules. One issue I have is that I can’t seem to find a way to set the facility and log level fields. These values are found in the header portion of the packet so ideally I should be able to pull the values from there (RFC5424). The log level is in the message so technically I could get it from there, but that doesn’t solve the issue of getting the facility as well. Is there a variable like $header available in pipeline rules similar to $message where the facility and log level values are stored? As far as I can tell, the only variable available in pipeline rules is $message and a few sub indices like gl2_remote_ip, source, etc.

Description of steps you’ve taken to attempt to solve the issue

I’ve looked through the pipeline documentation and the community forums but have not been able to find a solution to this issue.

Environmental information

Integration with LibreNMS. Mostly Cisco devices.

Operating system information

Ubuntu 20.04

Package versions

Graylog - 4.1.3+9d79c05
MongoDB - v4.0.26
Elasticsearch - 7.10.2

Hi @pepperoni-pi

You can easily extract facility and severity from PRI (priority) header from syslog still if you use Raw input. But your message should contain <PRI> in the beggining of your message. You can you either regex or GROK to extract it and then use pipeline function expand_syslog_priority to extract facility and severity.

https://docs.graylog.org/en/4.1/pages/pipelines/functions.html#expand-syslog-priority

rule "Cisco FirePower priority parsing"
when
  contains(to_string($message.message), "%FTD", true)
then
  set_fields(grok(pattern: "<%{NONNEGINT:syslog_pri:int}>", value: to_string($message.message), only_named_captures: true)); 
  let priority = expand_syslog_priority(to_long($message.syslog_pri));
  set_fields({facility: priority.facility, level: priority.level });
end

You can expand grok to also contains more details like data time mnemonic and so on if you want.

Or if you want to have text representation of facility and priority use function expand-syslog-priority-as-string
https://docs.graylog.org/en/4.1/pages/pipelines/functions.html#expand-syslog-priority-as-string

1 Like

Aha, that’s what I was looking for. So the value between the “<” and “>” is the priority. I tried out your rule and it works. However, I tried it with “expand_syslog_priority_as_string” and it doesn’t seem to work that way. When I hit apply, immediately the messages stop showing up in the stream. It works in the simulator though.

rule "priority parsing"
when
  true
then
  set_fields(grok(pattern: "<%{NONNEGINT:syslog_pri:int}>", value: to_string($message.message), only_named_captures: true)); 
  let priority_string = expand_syslog_priority_as_string(to_long($message.syslog_pri));
  set_fields({facility: priority_string.facility, log_level: priority_string.level});
end

Simulation runs successfully

Messages stop getting processed after rule is applied

What I would like to get

Screen Shot 2021-09-08 at 1.42.05 PM

Found one that slipped in with this error

gl2_processing_error
For rule 'priority parsing': In call to function 'expand_syslog_priority_as_string' at 7:24 an exception was thrown: For input string: "null"

Why would it be null in the “expand_syslog_priority_as_string” function, but not “expand_syslog_priority” function?

It also seems that I can’t set “facility” as anything except an integer, but I know that the Syslog UDP input uses the named string forms of the facility values such as local0 or local7.

Okay, the issue seems to be with setting the “facility” value. If I change it to something like “facility_string” the set_field command works. It just doesn’t let me set the “facility” value as a string, but I can set it as an integer. What I want the facility field to be though is the named value like local7, etc. just like the Syslog UDP input does. It looks like someone else may have come across this same issue, but it doesn’t appear to have been resolved.

Submitted a Github issue here.

Hi @pepperoni-pi

I not a bug/issue or problem, it is a feature. ElasticSearch uses dynamic mapping to determine type of data, so if you first use one type of data (e.g. numeric), that type if remembered for this field. So another time you can’t insert another type of data (e.g. string) to same field. If you want to change type of data of the field, you have to use custom index mapping and then rotate index. Or save data to new field with different name, as you’ve already done.

https://docs.graylog.org/en/4.1/pages/configuration/elasticsearch.html#custom-index-mappings
https://megamorf.gitlab.io/2021/01/20/graylog-and-elasticsearch-troubleshooting/#update-custom-field-mappings

Okay, so because Elasticsearch first received an integer for the “facility” field, the dynamic mapping set the field to “type” : “long” and as a result, only integers can be stored there now.

$ curl -X GET 'http://localhost:9200/graylog_deflector/_mapping?pretty'
...
      "properties" : {
        "facility" : {
          "type" : "long"
        },
        "facility_num" : {
          "type" : "long"
        },
        "facility_string" : {
          "type" : "keyword"
        },
...

To correct this, I would have to force the “type” to “keyword” or “text” with a custom index mapping and then rotate the index so the new mapping will take effect. Do the dynamic mappings reset every time the index rotates?

Yes, you have to change it from long to keyword. No index mapping won’t reset after index rotation.

@tmacgbay has just posted nice article about custom mapping, check this for better explanation:

The dynamic mappings do seem to reset when the index rotates. I set my pipeline rule back to saving the facility as the string value. The stream stopped showing any logs. Then I rotated the index and logs started pouring in again. I checked the dynamic mappings and now the facility is “type” : “keyword”. I’m sure you were referring to the custom mappings not resetting on rotation, but it looks like the dynamic ones do.

Before index rotation

$ curl -X GET 'http://localhost:9200/graylog_deflector/_mapping?pretty'
...
      "properties" : {
        "facility" : {
          "type" : "long"
        },
...

After index rotation

$ curl -X GET 'http://localhost:9200/graylog_deflector/_mapping?pretty'
...
      "properties" : {
        "facility" : {
          "type" : "keyword"
        },
...