Problem with to_ip set_field()

Graylog Central (peer support)
1

Hi all,

I’ve created some rules to create new fields and everything is working as expected. I want now to be able to “type” the specific fields (boolean, integer,… IP).

When I try to specify the IP type (through the to_ip function): set_field(“my_IP”, to_ip(values.[“myIP”]))) , while running through the Simulator, I get the following error Something went wrong with the error: "this.props.simulationResults is undefined"

When I try to implement this directly in the rule, the rule stops working.

What am I doing wrong.

For infos about versions:
Graylog Version: 3.1.2+9e96b08, codename Quantum Dog
ElasticSearch version: 6.8.5

Thanks in advance

2

Please send complete pipeline rule you use…

3

Here it is. Without the to_ip(), field is stored without problem

rule "Test NAC Connector - All key=value - all"
when
    to_string($message.source) == "127.0.0.1"
then
  
   // Key_value parsing and stores results in values Map
   let values= key_value(value:to_string($message.message), allow_dup_keys:true, trim_key_chars:" ", trim_value_chars:",", delimiters:", ");
   
   set_field("adIpAddress", to_ip(values["AD-IP-Address"]));
   set_field("adIpAddressBlackListed", values["AD-IP-Address-Black-Listed"]);
   set_field("adSite", values["AD-Site"]);
   set_field("adTrustedDomain", values["AD-Trusted-Domain"]);
end
4

I don’t think, that it is a correct way to map a type of field. Graylog by default uses Elastics’ dynamic mapping, so if you want to use specific schema check this:

https://docs.graylog.org/en/3.1/pages/configuration/elasticsearch.html#custom-index-mappings

5

Thanks for the reply.

I must say that I’m curious about your answer. If I take a look at many other topics in this forum, I can get many results of the usage of the to{ip, string, double, …}()_ function, e.g:

As soon as I modify my rule to use another of the to_double(), in the set_field(), the rule runs smoothly.

I have many IP possible data fields; are you telling me that I will have to create a template to handle all of them? It seems to me that IP is a common data type.

Thanks for the help.

6

the function to_SOMETHING is not given to make Graylog during the processing aware of a specific data type. This way you can give Graylog notice that you have a number/string given.

In addition you could use that to check the data type during processing. Means you proof if something is from a specific data type.

To enforce a specific datatype to be used to store a message you need to understand how that is working at all.

Elasticsearch - what is used to store the messages - is guessing what something is from a data perspective and is setting the field type based on this. Means on first ingest is decided if a field is a string, a number or a ip address.

So you hve two options. Rotate the index and pray that your first ingested value is recognized as the value it should or you create a custom mapping in elasticsearch that forces to be a specific value.

Does this make sense for you?

7

Dear Jan,

Thanks for the answer and sorry, your reply got lost among multiple mails.

I finally understood the to_SOMETHING purpose and I started some reading about the dynamic mapping.

As you said, I think I have to investigate (I already started :slightly_smiling_face:) more deeply the way that Elasticsearch stores data and I have to check with my colleagues how we are going to build our custom mappings (we are storing many fields from multiple (several tens) log sources).

Anyway, thanks for the help.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.