Hello,
is there a way to recursively parse JSON?
Thoughtful steps:
-
Extract JSON with Regex/Grok
-
Parse JSON and save to object
-
Automatically map individual elements to fields
{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40490,"time":{"ms":73}},"total":{"ticks":75650,"time":{"ms":115},"value":75650},"user":{"ticks":35160,"time":{"ms":42}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":9},"info":{"ephemeral_id":"4d41e9a1-2b3b-4bd8-9359-ce715d0c936e","uptime":{"ms":77880024}},"memstats":{"gc_next":7091120,"memory_alloc":5574160,"memory_total":7759230120},"runtime":{"goroutines":37}},"filebeat":{"events":{"added":3,"done":3},"harvester":{"files":{"203bc89d-c0c7-43a1-b0fd-670d7f157548":{"last_event_published_time":"2020-05-29T09:28:05.967Z","last_event_timestamp":"2020-05-29T09:28:05.967Z","read_offset":199},"9398b2be-686d-47ea-a7ee-5deb1b5bfbcd":{"last_event_published_time":"2020-05-29T09:27:45.974Z","last_event_timestamp":"2020-05-29T09:27:45.974Z","read_offset":1418,"size":1418}},"open_files":2,"running":2}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":3,"batches":2,"total":3},"read":{"bytes":12},"write":{"bytes":1471}},"pipeline":{"clients":2,"events":{"active":0,"published":3,"total":3},"queue":{"acked":3}}},"registrar":{"states":{"current":2,"update":3},"writes":{"success":2,"total":2}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
This should for example result in fields:
monitoring.metrics.beat.cpu.system.ticks: 6630
monitoring.metrics.filebeat.evets.added: 4
and so on…
Is there a possibility?
The existing JSON parser function creates only one field from the parent element:
monitoring
{"metrics":{"beat":{"cpu":{"system":{"ticks":40490,"time":{"ms":73}},"total":{"ticks":75650,"time":{"ms":115},"value":75650},"user":{"ticks":35160,"time":{"ms":42}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":9},"info":{"ephemeral_id":"4d41e9a1-2b3b-4bd8-9359-ce715d0c936e","uptime":{"ms":77880024}},"memstats":{"gc_next":7091120,"memory_alloc":5574160,"memory_total":7759230120},"runtime":{"goroutines":37}},"filebeat":{"events":{"added":3,"done":3},"harvester":{"files":{"203bc89d-c0c7-43a1-b0fd-670d7f157548":{"last_event_published_time":"2020-05-29T09:28:05.967Z","last_event_timestamp":"2020-05-29T09:28:05.967Z","read_offset":199},"9398b2be-686d-47ea-a7ee-5deb1b5bfbcd":{"last_event_published_time":"2020-05-29T09:27:45.974Z","last_event_timestamp":"2020-05-29T09:27:45.974Z","read_offset":1418,"size":1418}},"open_files":2,"running":2}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":3,"batches":2,"total":3},"read":{"bytes":12},"write":{"bytes":1471}},"pipeline":{"clients":2,"events":{"active":0,"published":3,"total":3},"queue":{"acked":3}}},"registrar":{"states":{"current":2,"update":3},"writes":{"success":2,"total":2}},"system":{"load":{"1":0,"5":0,"15":0,"norm":{"1":0,"5":0,"15":0}}}}}
Similar question: Parsing nested JSON message