Json nested and lsit

Templates and Rules Exchange Pipeline Rules
1

Hello,

I try to format my json log, but so far without success.
My goal is to be able to parse the nested field like :

httpRequest, ruleListGroup

My current setup is quite easy, I defined a JSON extractor for an raw-tcp input.
And then I tried to create a pipeline with a rule.

Any help to understand what I do wrong is appreciated .

SCR-20231112-bcsb
SCR-20231112-bcsb1758Ă—9357 543 KB

2

Hi, flep

please check syntax
In the rule, use the “parse_json” function to parse the nested fields. Here’s an example rule:


rule "Parse Nested Fields"
when
    has_field("message")
then
    let json_message = parse_json(to_string($message.message));
    let http_request = json_message.httpRequest;
    let rule_list_group = json_message.ruleListGroup;

    set_field("httpRequest", http_request);
    set_field("ruleListGroup", rule_list_group);
end

This pipeline rule attempts to parse the JSON in the message field and extract the httpRequest and ruleListGroup values into separate fields. Now you need to adjust the rule as needed based on your actual JSON structure.

Remember to check the Graylog server and processing pipeline simulator logs for any errors or warnings if things aren’t working as expected.

1 Like
3

Hello bahram,

Thank you for your reply.
I tried multiple times with different codes, and the simulator showed me the same.
The server.log returns no errors too (I’m using docker) but it does if I make a mistake.

I appreciate your first help :slight_smile:

image
image1920Ă—2643 287 KB

The sample I used to test the simulator:

{
    "timestamp": 1698624183281,
    "formatVersion": 1,
    "webaclId": "arn:aws:xxxxxxxxxx77a",
    "terminatingRuleId": "Default_Action",
    "terminatingRuleType": "REGULAR",
    "action": "ALLOW",
    "terminatingRuleMatchDetails": [],
    "httpSourceName": "ALB",
    "httpSourceId": "xxxxxxxxxx",
    "ruleGroupList": [
        {
            "ruleGroupId": "xxxxxxx",
            "terminatingRule": null,
            "nonTerminatingMatchingRules": [],
            "excludedRules": null,
            "customerConfig": null
        },
        {
            "ruleGroupId": "xxxxxxxx",
            "terminatingRule": null,
            "nonTerminatingMatchingRules": [],
            "excludedRules": null,
            "customerConfig": null
        }
    ],
    "rateBasedRuleList": [
        {
            "rateBasedRuleId": "xxxxxx",
            "rateBasedRuleName": "xxxxxx",
            "limitKey": "IP",
            "maxRateAllowed": 998,
            "limitValue": "xxxxxx"
        }
    ],
    "nonTerminatingMatchingRules": [],
    "requestHeadersInserted": null,
    "responseCodeSent": null,
    "httpRequest": {
        "clientIp": "xxxxx",
        "country": "US",
        "headers": [
            {
                "name": "content-length",
                "value": "1503"
            },
            {
                "name": "sec-ch-ua",
                "value": "\"Chromium\";v=\"118\", \"Google Chrome\";v=\"118\", \"Not=A?Brand\";v=\"99\""
            },
            {
                "name": "sec-ch-ua-platform",
                "value": "\"Windows\""
            },
            {
                "name": "sec-ch-ua-mobile",
                "value": "?0"
            },
            {
                "name": "user-agent",
                "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
            },
            {
                "name": "content-type",
                "value": "application/x-www-form-urlencoded"
            },
            {
                "name": "accept",
                "value": "*/*"
            },
            {
                "name": "sec-fetch-site",
                "value": "cross-site"
            },
            {
                "name": "sec-fetch-mode",
                "value": "cors"
            },
            {
                "name": "sec-fetch-dest",
                "value": "empty"
            },
            {
                "name": "accept-encoding",
                "value": "gzip, deflate, br"
            },
            {
                "name": "accept-language",
                "value": "en-US,en;q=0.9"
            }
        ],
        "uri": "/track",
        "args": "",
        "httpVersion": "HTTP/2.0",
        "httpMethod": "POST",
        "requestId": "1-653ef2b7-xxxxx"
    },
    "requestBodySize": 1503,
    "requestBodySizeInspectedByWAF": 1503,
    "ja3Fingerprint": "xxxxxx"
}
1 Like