Pipeline problem in search

Hello all,

I don’t understand why I can’t showing up pipeline in search tab, the pipeline test is ok and the order in system-configurations is:

|1|AWS Instance Name
|2|GeoIP Resolver
|3|Message Filter Chain
|4|Pipeline Processor

Pipeline is connected to streams “all messages”
any suggestions?


My Message Processors Configuration looks like this.

Maybe give that a try. Unless you using your AWS lookup mine is disabled.
Hope that helps.

thanks but unfortunately doesn’t work.

I try the same pipeline for some syslog messages and it works but I need this for a “Raw/Plaintext AMQP” input, so it’s the graylog that reads the Rabbit queue and I think I need to do something different for this input.

Can you share your INPUT, maybe a message and Pipeline configuration?


this is an example message:

i need to create a field named event when match 0x10000 and set “createfile”


the input


If i tested with simulator all work


If you just need to create a field from the message have you tried to use Extractors? Here is one example that I have.


My extractor created for this INPUT.

The end result.

Hope that helps

yes i try, but i need to change the value in the field

if i have 0x10000 in message the value of the extractors will be “create file”

in extractors i didn’t find anything to do this


I tried to create an example for you. What I’m using is Graylog 4.0.
Created a new field using a Regular expression from what information you gave us as shown below,
First I used this site for testing my Regular expression → here.

Then I created the extrator.

Using a pipeline to convert the your_new_field to a different value.

rule "event"
    has_field("your_new_field") AND contains(to_string($message.your_new_field), "0x10000")

Still fairly new at using pipelines and Regular expression. This worked for me to converting numbers into a name. I have seen other members/staff here work with pipelines better then i can.
Hope this helps

I have updated graylog to 4.0 but still not working even with extractors.
If I use on message the funcion “add to query” this is the result:


the original message is in a strange format so, probably, did not match with extractors,pipelines ecc…
do you ever seen something like this?


Agree, if the mesages looks like that, the extractor are not going to help.

Yes I have, when one of my INPUTs were incorrect. For an example: Using Syslog UDP INPUT for my frewalls I had to change my INPUT to Raw/Plaintext UDP.

You still using Raw/Plaintext AMQP? is so try using Raw/Plaintext UDP as an INPUT see if that helps. The order of what I would do is make sure messages are coming through correct. Then I would add the extractors to create unique fields. And last I would apply the pipeline. Just a thought…
Hope that helps


unfortunately I have to use Raw/Plaintext AMQP because the source support only rabbitmq for third party solutions and I can’t normalize logs.


How are you sending those messages to RabbitMQ? Do you happen to be sending Windows logs through RabbitMQ to Graylog? I ask because I ran into a similar issue (sans RabbitMQ) that seemed to indicate that it was actually an encoding issue on the Windows side. If it is in fact a Windows system, you’ll want to look at how that file is encoded. IANA Windows admin, but from what Google tells me, Windows uses UTF-16. If you’re using Filebeat, you should be able to set the encoding like so:

encoding: utf-16-bom

logs arrive from a EMC unity, rabbitmq is installed on a windows machine and talk with the EMC CEE service installed on the same machine.

I configure a Splunk endpoint that ricognized all the fileds of EMC logs without rabbitmq beacause CEE has a dedicated configuration for Splunk and these services communicate each other via HTTP.

I didn’t find anything like this in graylog so I try to do the same thing with rabbitmq.


This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.