# Pipeline rule key/value with specific delimiter not working **URL:** \ **Category:** Graylog Central (peer support)\ **Tags:** pipeline-rules\ **Created:** [September 7, 2023, 11:09am UTC](https://community.graylog.org/t/pipeline-rule-key-value-with-specific-delimiter-not-working/30040 "2023-09-07T11:09:15Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![s0p4L1N](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/s0p4l1n/32/15104_2.png) [@s0p4L1N](https://community.graylog.org/u/s0p4L1N)\ **Post date:** [September 7, 2023, 11:09am UTC](https://community.graylog.org/t/pipeline-rule-key-value-with-specific-delimiter-not-working/30040/1 "2023-09-07T11:09:16Z")
**1. Describe your incident:** I use winlogbeat to ingest Windows Defender Security Event to Graylog. The message field is not parsed correctly as it does not get all the data. I would like to parse it with the key/value pipeline but it give me some weird result: - The message looks like this: ```auto { "tags": [ "windowsdefender" ], "message": "Antivirus Microsoft Defender a détecté un logiciel malveillant ou potentiellement indésirable.\n Pour plus d’informations, reportez-vous aux éléments suivants :\nhttps://go.microsoft.com/fwlink/?linkid=37020&name=Virus:DOS/EICAR_Test_File&threatid=2147519003&enterprise=0\n \tNom : Virus:DOS/EICAR_Test_File\n \tID : 2147519003\n \tGravité : Grave\n \tCatégorie : Virus\n \tChemin : containerfile:_C:\\Users\\adm.user\\Downloads\\eicar_com.zip; file:_C:\\Users\\adm.user\\Downloads\\eicar_com.zip->eicar.com; webfile:_C:\\Users\\adm.user\\Downloads\\eicar_com.zip|https://secure.eicar.org/eicar_com.zip|pid:1352,ProcessStart:133385545470169081\n \tOrigine de la détection : Internet\n \tType de détection : Concret\n \tSource de détection : Téléchargements et pièces jointes\n \tUtilisateur : LAB\\adm.user\n \tNom du processus : Unknown\n \tVersion de la veille de sécurité : AV: 1.397.528.0, AS: 1.397.528.0, NIS: 1.397.528.0\n \tVersion du moteur : AM: 1.1.23080.2005, NIS: 1.1.23080.2005" } ``` And i want to extract all the field starting from `\n \tNom : ` - Where `\n \t` is the delimiter, - `: ` is the separator - as there is space before and after the separator, I tried many things but nothing work - The pipeline rule: ```auto when contains(to_string($message.tags),"windowsdefender") then set_fields( fields: key_value( value: to_string($message.message), trim_value_chars: " ", trim_key_chars:" ", delimiters:"\\n \\t", kv_delimiters:":" ) ); end ``` Here’s the result… ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/21845e19b85ed660ebf051e2ce82191d889ea0a9.png) (EventDescription comes from another pipeline) The trace logs of the simulations shows no errors. **2. Describe your environment:** - OS Information: docker - Package Version: Graylog 5.1.4 / Opensearch 2.9.0 **3. What steps have you already taken to try and solve the problem?** I already try to set the space in the trim or in separator but does not change the problem. **4. How can the community help?** Is it possible to set return carriage and tabulation as separator for this case ? ---
**Author:** ![s0p4L1N](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/s0p4l1n/32/15104_2.png) [@s0p4L1N](https://community.graylog.org/u/s0p4L1N)\ **Post date:** [September 7, 2023, 1:31pm UTC](https://community.graylog.org/t/pipeline-rule-key-value-with-specific-delimiter-not-working/30040/2 "2023-09-07T13:31:46Z")
I found a workaround with another pipelines function: I used the replace function, by replacing: - `\n \t` with `||` and `:` with `=` - replacing with `_` key name containing space - replacing `é` by `e` ```auto rule "Winlogbeat - W-Defender - Message Replace" when contains(to_string($message.tags),"windowsdefender") then let test6 = replace(to_string($message.message), "; ", ";"); set_field("message", test6); let test0 = replace(to_string($message.message), "Antivi", "Information = Antivi"); set_field("message", test0); let test4 = replace(to_string($message.message), ".\n", " || Detail_Info = "); set_field("message", test4); let test3 = replace(to_string($message.message), ":\nhtt", " || URL = htt"); set_field("message", test3); let test = replace(to_string($message.message), "\n \t", " || "); set_field("message", test); let test2 = replace(to_string($message.message), " : ", "="); set_field("message", test2); let test9 = replace(to_string($message.message), "Origine de la détection", "Origine_de_la_detection"); set_field("message", test9); let test10 = replace(to_string($message.message), "Type de détection", "Type_de_detection"); set_field("message", test10); let test11 = replace(to_string($message.message), "Source de détection", "Source_de_detection"); set_field("message", test11); let test12 = replace(to_string($message.message), "Nom du processus", "Nom_du_processus"); set_field("message", test12); let test13 = replace(to_string($message.message), "Version de la veille de sécurité", "Version_de_la_veille_de_securite"); set_field("message", test13); let test13 = replace(to_string($message.message), "Version du moteur", "Version_du_moteur"); set_field("message", test13); end ``` The message look like this after the first pipeline rule: ```auto "message": "Information = Antivirus Microsoft Defender a detecte un logiciel malveillant ou potentiellement indesirable || Detail_Info = Pour plus d’informations, reportez-vous aux elements suivants || URL = https://go.microsoft.com/fwlink/?linkid=37020&name=Virus:DOS/EICAR_Test_File&threatid=2147519003&enterprise=0 || Nom = Virus:DOS/EICAR_Test_File || ID = 2147519003 || Gravite = Grave || Categorie = Virus || Chemin = containerfile:_C:\Users\adm.user\Downloads\eicar_com.zip;file:_C:\Users\adm.user\Downloads\eicar_com.zip->eicar.com;webfile:_C:\Users\adm.user\Downloads\eicar_com.zip|https://secure.eicar.org/eicar_com.zip|pid:6316,ProcessStart:133385640422091285 || Origine_de_la_detection = Internet || Type_de_detection = Concret || Source de detection = Telechargements et pièces jointes || Utilisateur = ISS\adm.user|| Nom_du_processus = Unknown || Version_de_la_veille_de_securite = AV: 1.397.528.0, AS: 1.397.528.0, NIS: 1.397.528.0 || Version_du_moteur = AM: 1.1.23080.2005, NIS: 1.1.23080.2005", ``` And with a second stage pipeline with key value function pipeline: ```auto rule "W-DEFENDER Message Parser" when has_field("message") then set_fields( fields: key_value( value: to_string($message.message), trim_value_chars: "", trim_key_chars:"", delimiters:"||", kv_delimiters:"=" ) ); end ``` The results are here : ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/f/f6a7c56958fd988327d308187f2fe0df7ac41c64.png) I wonder why Winlogbeat does not parses automatically the XML Event ?? Because it already contains all the key/value pairs ! ` ---
**Author:** ![Joel\_Duffield](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Joel\_Duffield](https://community.graylog.org/u/Joel_Duffield)\ **Post date:** [September 7, 2023, 3:51pm UTC](https://community.graylog.org/t/pipeline-rule-key-value-with-specific-delimiter-not-working/30040/3 "2023-09-07T15:51:04Z")
Your workaround is the correct approach, the key value function accepts multiple characters as delimiters and seperators, so it doesnt see /n it sees / or n. I had the same problem as you once and pulled my hair out for way to long until I found that out. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [September 21, 2023, 3:51pm UTC](https://community.graylog.org/t/pipeline-rule-key-value-with-specific-delimiter-not-working/30040/4 "2023-09-21T15:51:29Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.