# Extractor causes low out message performance **URL:** \ **Category:** Graylog Central (peer support)\ **Tags:** grok-patternspl\ **Created:** [November 9, 2023, 11:30am UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609 "2023-11-09T11:30:54Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mka](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@mka](https://community.graylog.org/u/mka)\ **Post date:** [November 9, 2023, 11:30am UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/1 "2023-11-09T11:30:54Z")
**1. Describe your incident:** If we create an extractor on one of our inputs our out log performance goes down with something like 99%. Right now our in and out is around 5000 to 12000 messages. If i create this extrator our out goes to 5-10 messages and our processbuffer goes to 100%. The uncomitted messages builds up, and if i delete the extrator the out goes from the 5-10 to 50k and the build up is gone in seconds. What could cause this? This is the grok pattern we use: SrcIP: %{IPV4:ciscoftd\_src\_ip}, DstIP: %{IPV4:ciscoftd\_dst\_ip}, SrcPort: %{INT:ciscoftd\_src\_port}, DstPort: %{INT:ciscoftd\_dst\_port} I have also just tried with SrcIP: %{IPV4:ciscoftd\_src\_ip} but its the same. **2. Describe your environment:** - OS Information: Verson: 22.04.3 LTS CPU: 12 vCPU RAM: 32GB - Package Version: 5.2.0 - Service logs, configurations, and environment variables: /etc/security/limits.d/elasticsearch.conf -Xms14g -Xmx14g /etc/graylog/server/server.conf processbuffer\_processors = 8 outputbuffer\_processors = 4 inputbuffer\_ring\_size = 65536 inputbuffer\_processors = 4 **3. What steps have you already taken to try and solve the problem?** I have tried the following: - Restarting - Increasing processors (We have tried a range og options, like 8, 10, 12, 20 for each or some but its the same) - Increasing RAM and CPU on the VM - Disabling Geo and plugins - Disabling all pipelines ---
**Author:** ![drewmiranda-gl](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/drewmiranda-gl/32/13376_2.png) [@drewmiranda-gl](https://community.graylog.org/u/drewmiranda-gl)\ **Post date:** [November 9, 2023, 8:41pm UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/2 "2023-11-09T20:41:44Z")
Greetings! You are correct in that extractors can sometime incur severe performance penalties. Extractors are a legacy feature that predates [Processing Pipelines](https://go2docs.graylog.org/5-2/making_sense_of_your_log_data/pipelines.html). I recommend exploring the use of Processing Pipeines and using those in place of extractors. This blog post is a great place to get started: [https://graylog.org/post/graylog-parsing-rules-and-ai-oh-my/](https://graylog.org/post/graylog-parsing-rules-and-ai-oh-my/) . Let us know if you have any specific questions. ---
**Author:** ![faen](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@faen](https://community.graylog.org/u/faen)\ **Post date:** [November 10, 2023, 2:53pm UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/3 "2023-11-10T14:53:46Z")
I made the switch to pipelines earlier in the year - it wasn’t that hard and performance is so much better, plus you can do all the enrichment right there at the same time. ---
**Author:** ![mka](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@mka](https://community.graylog.org/u/mka)\ **Post date:** [November 13, 2023, 4:02pm UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/4 "2023-11-13T16:02:16Z")
Thanks you two, i tried the pipelines and they work great. However now I see new fields like the following. Are these just from the pipeline? MINUTE MONTHDAY MONTHNUM SECOND YEAR ISO8601\_TIMEZONE IPV4 IPORHOST ---
**Author:** ![ihe](https://avatars.discourse-cdn.com/v4/letter/i/a88e57/32.png) [@ihe](https://community.graylog.org/u/ihe)\ **Post date:** [November 14, 2023, 3:24pm UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/5 "2023-11-14T15:24:35Z")
> [@mka](#): > > MINUTE > MONTHDAY > MONTHNUM > SECOND > YEAR > ISO8601\_TIMEZONE > IPV4 > IPORHOST those look like unnamed fields captured from a grok - just as a guess. ---
**Author:** ![frantz](https://avatars.discourse-cdn.com/v4/letter/f/dbc845/32.png) [@frantz](https://community.graylog.org/u/frantz)\ **Post date:** [November 14, 2023, 4:26pm UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/6 "2023-11-14T16:26:59Z")
Did you configure the “Condition” ? Because if you select “Always try to extract” it will try to apply the grok pattern on all logs received by the input. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [November 28, 2023, 4:27pm UTC](https://community.graylog.org/t/extractor-causes-low-out-message-performance/30609/7 "2023-11-28T16:27:57Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.