1. Describe your incident:
I created pipeline rules and a pipeline to convert various FortiGate syslog fields from strings to doubles and IP addresses, with pipeline rules like this:
The pipeline rules page is showing message throughput of greater than zero.
The pipeline is connected to the stream.
And yet, all of the fields are still stored in the index as strings.
What am I doing wrong?
2. Describe your environment:
OS Information: Debain 12
Package Version: Graylog 5.2.4+ec33db8
Do you have a step in those rules where you are writing the output of that conversion back to a field? you will want to have a set field command using that $output_1
D’oh!
Also, Is there a better/easer way to convert all of these field types instead of one rule at a time?
First off, what’s the reason you are converting them?
So I can do range queries on numerical values, and CIDR queries on IP addresses
Thought so. So that doesn’t have a direct connection to how the data is actually stored. What you need to do is change the field data type. You access that option by clicking the field name in any message (search results) that has that field in it, then you can manually select the data type the index uses. You then need to rotate the index after you do that for it to apply.
Good to know. Thanks. Is using a pipeline in an exported content pack the best way to set the data types the best way to set things up so others won’t need to do that?
Others as in other users in your cluster?
Nah, I mean for users setting up a fresh cluster and using my content pack.
Ah, I don’t think the field mappings can be deployed though a content pack, even if they could it would depend on the index they route the data to. You may do better to lay out the data schema in a readme file and say this field should be this data type etc
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
