CSV Lookup error

Graylog Central (peer support)
1

Hi all,
I’m using csv lookup to enrich the email domain.
I created a DataAdapter that use a already present csv file, the Cache and the Lookup Table too

image
image1280×710 38.4 KB

where I add a default single value when the domain is not present in the csv file.

Here a snippet of my pipeline rule:

let domain = lookup_value("email-freemail-domains", to_string($message.email_source_domain));
  
  set_field("public_src_domain", domain);

Now I have this problem on Graylog:
If I use the lookup function in my pipeline rule, the message is prenset but is present in this way:
image

While my expected value should be just 1. So I throught that the result is a map and I update the code using domain[“value”] in my rule but no messages arrive!

If I use lookup_value no messages are present on Graylog stream and there is an error on the graylog log:

WARN : org.graylog2.indexer.messages.Messages - Failed to index message: index=<graylog_3> id=<858d0570-fcc5-11ea-9410-c2dd6581dc42> error=<{"type":"mapper_parsing_exception","reason":"object mapping for [public_src_domain] tried to parse field [public_src_domain] as object, but found a concrete value"}>

This is the same message that I have if I use lookup and domain[“value”]
So what does this error message means?
How I can complete my scenario, where am I doing wrong?

2
  1. Try to use custom mapping for field, or use new field name. After create custom mappping rotate index.
  2. Alternatively you can also save value as number using
    set_field("public_src_domain", to_long(domain));
3

Hi @shoothub
Changing the field name it works…but I are very surprised about this workaround

4

It’s a way how Elastic Search works. It will try to guess format of data (dynamic mapping) and assign it to the field first time it was saved. Then if you try to save another type of data it won’t work. You can only save one data type, for example if you create field with numeric value, you can’t insert string value afterwards.

5

Hi @shoothub
thanks a lot for your clarification :smiley:

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.