Change source name through pipeline example?

Graylog Central (peer support)
1

I’m trying to change the presented name of the Ubiquiti controller messages coming into Graylog through syslog. The controller is presenting itself as (“U7PG2,802aa8d014b6,v3.9.3.7537”) and I’m trying to get it to show up as simply unifi.[domain.com] which is how we access it.

I’m very new to Graylog and tried a pipeline rule like this.

rule “UbiquitiSwapSource”
when
$message.source == “(“u7pg2,802aa8d014b6,v3.9.3.7537”)”
then
set_field("$message.source", “unifi.xxx.yyy”);
end

The pipeline is connected to ‘all messages’ and other pipelines appear to work. Is this a bad way to attempt this? Are there better ways?

2 
rule "UbiquitiSwapSource"
when
  to_string($message.source) == "u7pg2,802aa8d014b6,v3.9.3.7537"
then
  set_field("source", "unifi.xxx.yyy");
end

If you have lots of devices and want to replace the “source” message field for each of them, take a look at lookup tables: http://docs.graylog.org/en/2.4/pages/lookuptables.html

3

I managed to get a lookup table so when I query (“u7pg2,802aa8d014b6,v3.9.3.7537”) the lookup comes back with:

{
“single_value”: “UniFi Cloud Controller”,
“multi_value”: {
“value”: “UniFi Cloud Controller”
},
“ttl”: 9223372036854776000
}

I added a table extractor to the input stream and from inside the extractor it appears to show a match but never gets triggered.

The store as field should be “source” to overwrite it yes?

4

well after an hour or so the name on two devices began to get swapped with the lookup table but the one source that arrives as (“u7pg2,802aa8d014b6,v3.9.3.7537”) stayed as (“u7pg2,802aa8d014b6,v3.9.3.7537”).

5

This sounds like a problem with the timestamps in your messages.

If the timestamp in the log messages doesn’t include any timezone information, Graylog will treat it as if it was UTC.

6

I have some unifi devices myself and use the following rules to get a “valid” hostname:

rule "unifi split source"
when
  has_field("source") AND
  contains(to_string($message.source), "U7LT") OR 
  contains(to_string($message.source), "US8P60") OR 
  contains(to_string($message.source), "US16P150")
  // didn't yet find a better way doing this
then

 // clean up source and get device information out of the fucking string that is provided as source
 // needs the following defined grok pattern in the system
 // UNIFIYSOURCES = ("%{WORD:device_type},%{WORD:device_mac},%{NOTSPACE:device_version}")
 
 let source_field = to_string($message.source);
 let source_split = grok(pattern: "%{UNIFIYSOURCES}", value: source_field, only_named_captures: true); 
 set_fields(source_split);
end

That helps me to have all three available information in a single field (and I can spot easy when a software update is not yet rolled out complete).

After that you have two options, depending of the number of devices you have. Create a second stage in the pipeline and write one rule for each device like

rule "unifi set hostname (802aa893d851)"
when
    has_field("device_mac") AND contains(to_string($message.device_mac), "802aa893d851")
then
    set_field("source", "ap-office.lan");
end

Or create a lookup table that contains the device MACs and hostnames and use that in one rule to fit all devices.

rule "unifi set hostname from LUT"
when
	// use as much fields as possible to 
	// remove false lookups if device_mac might be present
	// on other messages
    has_field("device_mac") AND
    has_field("device_type")
then
    // get hostname based on MAC for unifi devices
    let update_source = lookup_value("unifi-hostname-lookuptable", $message.device_mac);
    set_field("source", update_source);
end
7

I’ll test that approach. The lookup table did start working about 9 hours after I set it up. The hostnames began to appear instead of the (“name”) that was showing up.

8
9

The firewall was sending out a timestamp set to the wrong timezone, working well now.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.