Hello all,
I don’t understand why I can’t showing up pipeline in search tab, the pipeline test is ok and the order in system-configurations is:
|1|AWS Instance Name
|2|GeoIP Resolver
|3|Message Filter Chain
|4|Pipeline Processor
Pipeline is connected to streams “all messages”
any suggestions?
thanks
Hello,
My Message Processors Configuration looks like this.
Maybe give that a try. Unless you using your AWS lookup mine is disabled.
Hope that helps.
thanks but unfortunately doesn’t work.
I try the same pipeline for some syslog messages and it works but I need this for a “Raw/Plaintext AMQP” input, so it’s the graylog that reads the Rabbit queue and I think I need to do something different for this input.
Hello,
Can you share your INPUT, maybe a message and Pipeline configuration?
Thanks
this is an example message:
i need to create a field named event when match 0x10000 and set “createfile”
the input
If i tested with simulator all work
thanks
Hello,
If you just need to create a field from the message have you tried to use Extractors? Here is one example that I have.
Message
My extractor created for this INPUT.
The end result.
Hope that helps
yes i try, but i need to change the value in the field
if i have 0x10000 in message the value of the extractors will be “create file”
in extractors i didn’t find anything to do this
thanks
@nsecurity
Hello,
I tried to create an example for you. What I’m using is Graylog 4.0.
Created a new field using a Regular expression from what information you gave us as shown below,
First I used this site for testing my Regular expression → here.
Then I created the extrator.
Using a pipeline to convert the your_new_field to a different value.
rule "event"
when
has_field("your_new_field") AND contains(to_string($message.your_new_field), "0x10000")
then
set_field("your_new_filed","new_value");
end
Still fairly new at using pipelines and Regular expression. This worked for me to converting numbers into a name. I have seen other members/staff here work with pipelines better then i can.
Hope this helps
@gsmith
Hello,
I have updated graylog to 4.0 but still not working even with extractors.
If I use on message the funcion “add to query” this is the result:
the original message is in a strange format so, probably, did not match with extractors,pipelines ecc…
do you ever seen something like this?
thanks
Agree, if the mesages looks like that, the extractor are not going to help.
Yes I have, when one of my INPUTs were incorrect. For an example: Using Syslog UDP INPUT for my frewalls I had to change my INPUT to Raw/Plaintext UDP.
You still using Raw/Plaintext AMQP? is so try using Raw/Plaintext UDP as an INPUT see if that helps. The order of what I would do is make sure messages are coming through correct. Then I would add the extractors to create unique fields. And last I would apply the pipeline. Just a thought…
Hope that helps
unfortunately I have to use Raw/Plaintext AMQP because the source support only rabbitmq for third party solutions and I can’t normalize logs.
thanks
How are you sending those messages to RabbitMQ? Do you happen to be sending Windows logs through RabbitMQ to Graylog? I ask because I ran into a similar issue (sans RabbitMQ) that seemed to indicate that it was actually an encoding issue on the Windows side. If it is in fact a Windows system, you’ll want to look at how that file is encoded. IANA Windows admin, but from what Google tells me, Windows uses UTF-16. If you’re using Filebeat, you should be able to set the encoding like so:
encoding: utf-16-bom
logs arrive from a EMC unity, rabbitmq is installed on a windows machine and talk with the EMC CEE service installed on the same machine.
I configure a Splunk endpoint that ricognized all the fileds of EMC logs without rabbitmq beacause CEE has a dedicated configuration for Splunk and these services communicate each other via HTTP.
I didn’t find anything like this in graylog so I try to do the same thing with rabbitmq.
thanks
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.