Remove Field function in pipeline not working

I have created some fields and I want to remove it, i used remove_field() function in pipeline, but it doesn’t work. i change message processor order, Message Filter Chain -> Pipeline Processor. but it still doesn’t work. @jan can you help give me some tips to remove unused fields?
Thanks

Please post your pipeline rule, and information of configured pipeline…

Here it is:
rules configure

Blockquote rule “Remove Fields”
when
true
then
remove_field(“any”);
end

Screen Shot 2020-08-05 at 3.34.32 PM

  1. Graylog can remove only fields that coming inside, not fields for older messages already stored in ES.
  2. Best way to debug is to use debug() function, and check logs, if the pileline function was called or not:
    debug("in function Remove Fields")
    https://docs.graylog.org/en/3.3/pages/pipelines/functions.html#debug
    And afterthat check graylog logs, which you should see message from debug function:
    sudo tail -f /var/log/graylog-server/server.log
  3. If you want to remove only one field, better way is to use condition for field existence, also for speed:
rule "Remove field"
when
    has_field("source_ip")
then
     // added debug message to be notified about the removed fields
    debug( concat("dropped field from ", to_string($message.source)));
    remove_field("source_ip");
end

understood, Thank you.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.