Odd Pipeline/Stream Behavior

If you click on the “Show received messages” the recent messages come in with fields and values as you would expect

Are you able to provide a screenshot or copy/paste (it can be redacted)? The syslog spec doesn’t support fields beyond timestamp, level, hostname, message. I’m curious how other fields are being added. There has to be something parsing those fields weather its an extractor or a pipeline rule.

if you set up Stream (call it “Fortigate”) to catch the inputs (via gl2_source_input number) the fields and values are no longer mapped.

Can you provide screenshots or a step by step of how you are setting this up?

If you create a Pipeline Rule and attach it to a Pipeline that is attached to the Stream Fortigate, suddenly there are no recently messages in either the Stream view

Can you share the pipeline rule? Is it ANY rule? Are there any errors in the graylog server.log?

system is usually pegged at 95% use (RAM)

This could be a potential cause of slowness but without data to measure it it is difficult to say for certain. Elasticsearch and OpenSearch are very memory intensive. Also searching for “All Messages” can be a very intense query.

1 Like