Brackets not escaped in alert emails

[inventor96 (Caleb)]

1. Describe your incident:

I’ve created an alert that runs daily to alert me of any pipeline processing errors (indicated by the presence of the gl2_processing_error field). Here’s an example of the contents of this field:

Error evaluating action for rule <parse httpd access fields/611b70ac278fe11a3fe8ed2c> (pipeline <Apache httpd Access Log Parsing/611b67d8278fe11a3fe8e394>) - In call to function 'parse_date' at 20:13 an exception was thrown: Invalid format: ""

In Graylog, this is just fine, but when the email is received, the brackets are not escaped, and so the email software and/or the web browser client will treat everything within the brackets as an HTML tag, causing it to be hidden. With the example above, this is what it looks like in the email:

Error evaluating action for rule (pipeline ) - In call to function 'parse_date' at 20:13 an exception was thrown: Invalid format: ""

Is there a function in the alert email template language that would allow me to escape the brackets so the field contents can come through as seen in the Graylog UI?

2. Describe your environment:

• OS Information: Rocky Linux 8.6

• Package Version: 4.3.1

• Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?

I’m not aware of any steps I could take at this point. I’ve thought about using a pipeline rule to do the escaping for me, but my understanding is that pipelines run simultaneously, and I’m not aware of a way to force one pipeline to run after another without moving the message between streams…

4. How can the community help?

Is there a function like the one I just described? Or does this need to be a bug report or feature request?

[gsmith]

Hello @inventor96

@inventor96

Error evaluating action for rule <parse httpd access fields/611b70ac278fe11a3fe8ed2c> (

I’m taking a guess, that message is under the field called “messages”, is this correct?

@inventor96

HTML tag, causing it to be hidden. With the example above, this is what it looks like in the email:

I’m assuming again, the notification template HTML looks something like this?

<tr><td>Fields</td><td><ul style="list-style-type:square;">${foreach event.fields field}<li>${field.key}:${field.value}</li>${end}<ul></td></tr>
</tbody></table>
${if backlog}
<br /><table width="100%" border="0" cellpadding="10" cellspacing="0" style="background-color:#f9f9f9;border:none;line-height:1.2"><tbody>
<tr><th style="background-color:#e6e6e6;line-height:1.5">Backlog (Last messages accounting for this alert)</th></tr>
${foreach backlog message}
<tr><td>${message}</td></tr>
${end}
</tbody></table>
${end}

If not, can you show us you notification template, either I can find an error or create a solution for you.

EDIT: Also you could expand your pipeline to add another stage ( i.e. rule) if need be. There are many examples of this in the forum. Showing how you created this pipeline could help troubleshoot this issue.

[gsmith]

I did some research and couldn’t find a way to set HTML to show those full message with angle brackets in it, but I did find a regex. It might help.

Example:

<([^>]*)>

From here.

regex101: build, test, and debug regex

[inventor96 (Caleb)]

@gsmith

I’m taking a guess, that message is under the field called “messages”, is this correct?

If I understand your guess, that’s not correct. The text I quoted was from the field called gl2_processing_error.

@gsmith

I’m assuming again, the notification template HTML looks something like this?

Basically, yes. The only significant difference is that instead of ${message}, it’s ${message.fields.gl2_processing_error}.

NOTE: This issue is not specifically tied to this field. The issue would happen for any field whose content could include open and closing brackets.

@gsmith

could expand your pipeline to add another stage

Adding another stage to any of our pipelines isn’t a problem, but I don’t think that would help in this situation. Here are my observations when doing some testing in our Graylog instance.

Whenever there’s an error during pipeline processing, the field gl2_processing_error is automatically created by Graylog and contains the error message. Any remaining rules in the same stage where the error occurs do not get processed, so I can’t reliably stick another rule in each stage.

Even though the remainder of the stage does not execute, the pipeline processor does move on to the next stage as needed. So I could put a rule at the end of every pipeline we have, but that would only work for pipelines where the entire pipeline gets executed every time, or I would have to add a lot of complexity to every rule to allow this without breaking the existing pipeline rule functionality.

@gsmith

I did find a regex. It might help.

Yes, that regex would help in general, but I still need a way to do a regex find-and-replace in the dynamic parts of the email template.

[tmacgbay]

I think what you want is the HTML preformatted text marker for your field in the HTML alert code.

<pre> YOUR STUFF HERE </pre>

Or here is an example from some of my alert code:

<td><pre>${message.fields.event_detail}</pre></td>

You didn’t post your alert code so I can’t tell exactly…

[gsmith]

Hello @inventor96

Ok I get it now.

I went through the HTML code/s to find a solution, which you know is because of these brackets < >. Unfortantly I was unable to find a way to solve this issue. Most if not all forums I was in stated they ended up using some type of script.

• HTML Tutorial

@inventor96

Yes, that regex would help in general, but I still need a way to do a regex find-and-replace in the dynamic parts of the email template.

Understood, so this is a global issue. If a message/log has brackets the email notification template with HTML will not show parts of the message.

This is tough one,

@tmacgbay is good at Pipelines , I’m sure he may come up with something if you post the rule that was created.

Couple suggestions.
If you know which INPUT these logs are received on , maybe create an REGEX extractor to filter out those messages, and/or create a Grok pattern to save, then apply it to any or all inputs you wish. Yes, you will end up creating a new field for the messages that are re-formatted.

Having messages/logs with HTML code in them and trying to send these messages/logs in HTML I don’t think is going to work, unless they are formatted/removed. I would use pipeline/extractor to clean up the messages/logs before there sent via email.

I know this is not ideal situation but if all fails then yes, I would put in a feature request with messages that have HTML code in them. Maybe dev’s can come up with something.

Last have you tried not to use HTML on the notification template?

@tmacgbay
I did test the tags showed, didn’t work on plain text, not sure if my testing was completely accurate thou. Since there are macros on the template compared to have just plaintext. I have not upload a test message yet to be completely 100% sure. I’m just working of @inventor96 sample above.

<td><pre>${message.fields.event_detail}</pre></td>

[inventor96 (Caleb)]

@tmacgbay

I think what you want is the HTML preformatted text marker for your field in the HTML alert code.

The brackets would still need converted to > and <, respectively. See @gsmith’s example.

@gsmith

If you know which INPUT these logs are received on , maybe create an REGEX extractor to filter out those messages, and/or create a Grok pattern to save, then apply it to any or all inputs you wish.

In general that is a good idea, but in this case, the messages with brackets are created during pipeline processing; the messages don’t come in with the brackets. The message comes in, goes through the respective pipeline(s), generates an error from one of the pipeline rules, and then comes out with the new gl2_processing_error field that contains the brackets.

@gsmith

Last have you tried not to use HTML on the notification template?

I have thought about that (I’m assuming that’s done by simply removing all content from the HTML portion of the alert notification), but it would be nice to have the HTML formatting in our email for easier layout and reading.

[gsmith]

@inventor96

I have thought about that (I’m assuming that’s done by simply removing all content from the HTML portion of the alert notification),

Correct,

@inventor96

but it would be nice to have the HTML formatting in our email for easier layout and reading.

Agree

To be honest, I haven’t had/seen this issue but I am curious about it how to solve it.

[inventor96 (Caleb)]

Where would I go to make a feature request?

[gsmith]

Sorry, you go here @inventor96

• Sign in to GitHub · GitHub

[inventor96 (Caleb)]

I created Issue 12854.

[tmacgbay]

Interesting - a reply on the issue from zhu:

you can use ${message;htmlsafe} to escape the message.

[gsmith]

@tmacgbay

Interesting -

Agree, Did not know this…

[zhu]

That PR has not been merged yet. You can build a test version by yourself or wait until some one merge it.

[tmacgbay]

Well, darnit.

How about a textarea element:
<textarea id="w3review" name="w3review" rows="4" cols="50">Error evaluating action for rule <parse httpd access fields/611b70ac278fe11a3fe8ed2c> (pipeline <Apache httpd Access Log Parsing/611b67d8278fe11a3fe8e394>) - In call to function 'parse_date' at 20:13 an exception was thrown: Invalid format: ""</textarea>

[gsmith]

@zhu

That PR has not been merged yet.

Well… that sucks, btw thanks for responding here.

[nisow95612]

Thank you for the work @zhu. I hope they will merge it, because all other solutions look like band-aids.

If graylog copies unescaped strings from logs into HTML, I don’t think any wrapping in message template will help against malicious logs. For example attackers can easily undo @tmacgbay 's <textarea> by input like </textarea><script>alert("Attacked by javascript embedded in logs")</script><textarea>

And for anybody who says attackers can’t put stuff in the logs… what about access logs from your web servers?