I am ingesting data that comes in the following form:
{
"severity": "CRITICAL",
"gl2_accounted_message_size": 407,
"gl2_receive_timestamp": "2024-06-20 17:19:03.600",
"level": 2,
"gl2_remote_ip": "192.168.128.253",
"gl2_remote_port": 60415,
"streams": [
"66743c7017398b40b69e9950"
],
"gl2_message_id": "01J0VAHBTR00000V7QK3NT46X7",
"source": "librenms",
"message": "librenms librenms[179]: 10.3.4.38: [Critical] Ping Latency - sysObjectID = .1.3.6.1.4.1.311.1.1.3.1.2; sysDescr = Hardware: AMD64 Family 25 Model 1 Stepping 1 AT/AT COMPATIBLE - Software: Windows Version 6.3 (Build 19045 Multiprocessor Free); location_id = 2;",
"gl2_source_input": "6658d16c17398b40b6914b7b",
"gl2_processing_timestamp": "2024-06-20 17:19:03.600",
"facility_num": 3,
"gl2_source_node": "9eeb2b4c-7ed7-4e24-8caa-658ed92957a0",
"_id": "311e2b01-2f29-11ef-a8c2-bc241192d178",
"facility": "system daemon",
"gl2_processing_duration_ms": 0,
"timestamp": "2024-06-20T17:19:03.000Z"
}
Graylog does a good job parsing out of the box, but “message” needs further parsing. I’d like a pipeline rule that splits at the “;'s” within “message” and creates new fields.
For instance, in the above example, I would like:
"message": "librenms librenms[179]: 10.3.4.38: [Critical] Ping Latency - sysObjectID = .1.3.6.1.4.1.311.1.1.3.1.2; sysDescr = Hardware: AMD64 Family 25 Model 1 Stepping 1 AT/AT COMPATIBLE - Software: Windows Version 6.3 (Build 19045 Multiprocessor Free); location_id = 2;",
To:
message_2: librenms librenms[179]: 10.3.4.38: [Critical] Ping Latency
sysObjectID: .1.3.6.1.4.1.311.1.1.3.1.2
sysDescr: Hardware: AMD64 Family 25 Model 1 Stepping 1 AT/AT COMPATIBLE - Software: Windows Version 6.3 (Build 19045 Multiprocessor Free)
Any thoughts?
Thank you!