Pipeline rule to truncate long URLs

TLDR
I just want to truncate URLs so my dashboard widget does not have crazy long URLs and scrollbars on super long URLs

Description of your problem

I am trying to truncate long URLs in my logs so they do not cause scrolling on my dashboard widget. Attempting to use a pipeline rule for this however running into an issue getting it to work.

Description of steps you’ve taken to attempt to solve the issue

Built pipeline rule and applied it to the stream, however nothing happens, my URLFilename field is still displaying crazy long URLs on my dashboards

Environmental information

Graylog 4.2.1
Ubuntu 20.04

This is the simple pipeline rule I am trying to use

```
 rule "test rule"
when
 has_field("URLFilename")
then
 let new_URLFilename = abbreviate(to_string($message.URLFilename), 25);
 set_field("URLFilename", new_URLFilename);
end
```

You can use the debug function to find out what is going on in the rule:


rule "test rule"
when
 has_field("URLFilename")
then
 let new_URLFilename = abbreviate(to_string($message.URLFilename), 25);
 set_field("URLFilename", new_URLFilename);
 // find below  in Gryalog log
 debug(concat("Origional-URL: ",to_string($message.URLFilename)));
 debug(concat("smaller-URL: ",new_URLFilename));
end

Use this command to monitor the Graylog log:

$ tail -f /var/log/graylog-server/server.log

1 Like

@tmacgbay thank you! that was very helpful. according to the debug log the has_field condition is not met so it skips processing, funny thing is. I know URLFilename has the URL in it and I can query on it in the search and build dashboards from it…

so next to just test it I changed the has_field to check for “source_ip” because I have another pipeline processing on that and its now processing the rule and now the debug log shows a match to the field and the debugging entries you suggested show up in the log however the URLFilename before and after values are empty?! :confused:

I must be missing something here because the field exists, has values in it, and is searchable. It is just not being seen by my pipeline rule for some reason.

There are two things that I can think of that could cause it to not have a field at that point in time in the pipeline. The first one is your message processor configuration under System/Configurations likely you want “Message Filter Chain” to come before “Pipeline Processor”. The other is that rules in a particular stage in the pipeline are processed in parallel. Meaning that if you have a rule in a stage that creates the field “URLFilename” you are best of testing for it’s existence in subsequent pipeline stages.

1 Like

aha! yes, thank you. It was the order of operations. I just popped those into place and it started working. Problem solved thank you!

1 Like

Great! Could you mark the answer or maybe add an answer post with the answer and any details missed so it is easily searchable for future question? Glad it worked out!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.