Indexer failures and regex with set_field in Pipeline

[Grakkal (Roger Mier)]

I can not figure out what I’m doing wrong here for the life of me. I’m trying to use a regex to look at the source field and determine which QA node the message came from, then set a field with the qa number so I can sort the message to a stream in a later pipeline.
My code looks fine, and I get no errors when I’m in the Pipeline rule editor. But when I save the rule and attach the pipeline to a Stream, I get this Indexer error:
{"type":"mapper_parsing_exception","reason":"object mapping for [qa_node] tried to parse field [qa_node] as object, but found a concrete value"}
And then no messages go into that stream anymore.

At this point, my code looks like this:
    rule "what qa node"
    when
        (has_field("source") AND contains(to_string($message.source), "qa"))
    then
        let n = regex("^qa(\\d+).?", to_string($message.source), ["num"]);
        let node = n["num"];
        set_field("qa_node", node);
    end

I’ve tried using let node = n.num;
I’ve tried set_field(“qa_node”, n.num);
I’ve tried set_field(“qa_node”, n[“num”]);
None have worked so far.

It’s worth noting that if I use:
set_field(“qa_node”, n);
The regex works, and it sets “qa_node” to {“num”, “11”} (Assuming it saw qa11 in the $message.source field). How can I get it to be just ‘11’?

[jochen]

The node variable in your rule already only contains the captured value (“11”) but the dynamic mapping of Elasticsearch expects a JSON object.

You’ll have to create a custom index mapping for the “qa_node” field (and any other custom field you’re interested in) in Elasticsearch:
http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings

[Grakkal (Roger Mier)]

Many thanks to you! After adding the custom mapping and rotating the index, my field now has the correct number and I can continue with my pipelines.

Feel free to close this.