# Pipeline problem in search **URL:** \ **Category:** Graylog Central (peer support)\ **Tags:** sidecar, filebeat-windows\ **Created:** [March 30, 2021, 12:38pm UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314 "2021-03-30T12:38:37Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [March 30, 2021, 12:38pm UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/1 "2021-03-30T12:38:37Z")
Hello all, I don’t understand why I can’t showing up pipeline in search tab, the pipeline test is ok and the order in system-configurations is: |1|AWS Instance Name |2|GeoIP Resolver |3|Message Filter Chain |4|Pipeline Processor Pipeline is connected to streams “all messages” any suggestions? thanks ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [March 31, 2021, 1:36am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/2 "2021-03-31T01:36:43Z")
Hello, My Message Processors Configuration looks like this. ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/0/07065069781c27417612d3650bf410e1f9d14916.png) Maybe give that a try. Unless you using your AWS lookup mine is disabled. Hope that helps. ---
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [March 31, 2021, 7:13am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/3 "2021-03-31T07:13:31Z")
thanks but unfortunately doesn’t work. I try the same pipeline for some syslog messages and it works but I need this for a “Raw/Plaintext AMQP” input, so it’s the graylog that reads the Rabbit queue and I think I need to do something different for this input. ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 1, 2021, 1:25am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/4 "2021-04-01T01:25:45Z")
Hello, Can you share your INPUT, maybe a message and Pipeline configuration? Thanks ---
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [April 1, 2021, 2:00pm UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/5 "2021-04-01T14:00:25Z")
this is an example message: ![immagine](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/c/c946dd34ffd7e33f50b4faae211ea1b2ef2dda8b.png) i need to create a field named event when match 0x10000 and set “createfile” ![immagine](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/23ac6cc961f698477b1401921964daebe4ac4eda.png) the input ![immagine](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/8/8ce9ecba8e0af0e9daa8fc1e1fcc5ba4f8981873.png) If i tested with simulator all work thanks ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 1, 2021, 10:10pm UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/6 "2021-04-01T22:10:19Z")
Hello, If you just need to create a field from the message have you tried to use Extractors? Here is one example that I have. Message ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/d/d183e610b70963e06aa5f43aac2924086019ce78.png) My extractor created for this INPUT. ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/8/88b17dfd69ab8698bce1d702078a34b4fa238e07.png) The end result. ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/2cf6c150c2939b7e5f5e66196166751b0a040395.png) Hope that helps ---
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [April 2, 2021, 6:37am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/7 "2021-04-02T06:37:30Z")
yes i try, but i need to change the value in the field if i have 0x10000 in message the value of the extractors will be “create file” in extractors i didn’t find anything to do this thanks ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 2, 2021, 10:58pm UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/8 "2021-04-02T22:58:31Z")
@nsecurity Hello, I tried to create an example for you. What I’m using is Graylog 4.0. Created a new field using a Regular expression from what information you gave us as shown below, First I used this site for testing my Regular expression → [here](https://regex101.com/). Then I created the extrator. ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/e/e09f16a8e9483b2b88ceabf3f7664dee08bd6b37.png) Using a pipeline to convert the **your\_new\_field** to a different value. ``` rule "event" when has_field("your_new_field") AND contains(to_string($message.your_new_field), "0x10000") then set_field("your_new_filed","new_value"); end ``` Still fairly new at using pipelines and Regular expression. This worked for me to converting numbers into a name. I have seen other members/staff here work with pipelines better then i can. Hope this helps ---
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [April 7, 2021, 10:42am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/9 "2021-04-07T10:42:02Z")
@gsmith Hello, I have updated graylog to 4.0 but still not working even with extractors. If I use on message the funcion “add to query” this is the result: ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/0/06cec76907fac5c43bdd947ba106801e29169b5d.png) ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/a/afe1e79dcc83fb55fae7fafb43140aa128aca264.png) the original message is in a strange format so, probably, did not match with extractors,pipelines ecc… do you ever seen something like this? thanks ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 7, 2021, 9:07pm UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/10 "2021-04-07T21:07:47Z")
@nsecurity > [@nsecurity](#): > > the original message is in a strange format so, probably, did not match with extractors,pipelines ecc Agree, if the mesages looks like that, the extractor are not going to help. > [@nsecurity](#): > > do you ever seen something like this? Yes I have, when one of my INPUTs were incorrect. For an example: Using Syslog UDP INPUT for my frewalls I had to change my INPUT to Raw/Plaintext UDP. You still using Raw/Plaintext AMQP? is so try using Raw/Plaintext UDP as an INPUT see if that helps. The order of what I would do is make sure messages are coming through correct. Then I would add the extractors to create unique fields. And last I would apply the pipeline. Just a thought… Hope that helps ---
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [April 9, 2021, 11:00am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/11 "2021-04-09T11:00:05Z")
@gsmith unfortunately I have to use Raw/Plaintext AMQP because the source support only rabbitmq for third party solutions and I can’t normalize logs. thanks ---
**Author:** ![aaronsachs](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/aaronsachs/32/7180_2.png) [@aaronsachs](https://community.graylog.org/u/aaronsachs)\ **Post date:** [April 10, 2021, 1:48am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/12 "2021-04-10T01:48:08Z")
How are you sending those messages to RabbitMQ? Do you happen to be sending Windows logs through RabbitMQ to Graylog? I ask because I ran into a similar issue (sans RabbitMQ) that seemed to indicate that it was actually an encoding issue on the Windows side. If it is in fact a Windows system, you’ll want to look at how that file is encoded. IANA Windows admin, but from what Google tells me, Windows uses UTF-16. If you’re using Filebeat, you should be able to set the encoding like so: `encoding: utf-16-bom` ---
**Author:** ![nsecurity](https://avatars.discourse-cdn.com/v4/letter/n/3be4f8/32.png) [@nsecurity](https://community.graylog.org/u/nsecurity)\ **Post date:** [April 10, 2021, 7:53am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/13 "2021-04-10T07:53:14Z")
logs arrive from a EMC unity, rabbitmq is installed on a windows machine and talk with the EMC CEE service installed on the same machine. I configure a Splunk endpoint that ricognized all the fileds of EMC logs without rabbitmq beacause CEE has a dedicated configuration for Splunk and these services communicate each other via HTTP. I didn’t find anything like this in graylog so I try to do the same thing with rabbitmq. thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [April 24, 2021, 7:53am UTC](https://community.graylog.org/t/pipeline-problem-in-search/19314/14 "2021-04-24T07:53:36Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.