Can Graylog parse logs like this?

Graylog Central (peer support)
1

I’m a system adminstrator for a document management system, and everyday we generate about 2,000 errors through a process where inbound data is sent to our system. It’ll give you an error and then the inbound message that generated the error.

ERROR: Primary keyword not associated with A^04 to complete HAR Autofill

MSH|^~&|AccMgr|1|||20050110045504|| ADT^A01 |599102|P|2.3||| EVN|A01|20050110045502||||| PID|1||10006579^^^1^MRN^1||DUCK^DONALD^D||19241010|M||1|111 DUCK ST^^FOWL^CA^999990000^^M|1|8885551212|8885551212|1|2||40007716^^^AccMgr^VN^1|123121234|||||||||||NO NK1|1|DUCK^HUEY|SO|3583 DUCK RD^^FOWL^CA^999990000|8885552222||Y|||||||||||||| PV1|1|I|PREOP^101^1^1^^^S|3|||37^DISNEY^WALT^^^^^^AccMgr^^^^CI|||01||||1|||37^DISNEY^WALT^^^^^^AccMgr^^^^CI|2|40007716^^^AccMgr^VN|4|||||||||||||||||||1||G|||20050110045253|||||| GT1|1|8291|DUCK^DONALD^D||111^DUCK ST^^FOWL^CA^999990000|8885551212||19241010|M||1|123121234||||#Cartoon Ducks Inc|111^DUCK ST^^FOWL^CA^999990000|8885551212||PT| DG1|1|I9|71596^OSTEOARTHROS NOS-L/LEG ^I9|OSTEOARTHROS NOS-L/LEG ||A| IN1|1|MEDICARE|3|MEDICARE|||||||Cartoon Ducks Inc|19891001|||4|DUCK^DONALD^D|1|19241010|111^DUCK ST^^FOWL^CA^999990000|||||||||||||||||123121234A||||||PT|M|111 DUCK ST^^FOWL^CA^999990000|||||8291 IN2|1||123121234|Cartoon Ducks Inc|||123121234A|||||||||||||||||||||||||||||||||||||||||||||||||||||||||8885551212 IN1|2|NON-PRIMARY|9|MEDICAL MUTUAL CALIF.|PO BOX 94776^^HOLLYWOOD^CA^441414776||8003621279|PUBSUMB|||Cartoon Ducks Inc||||7|DUCK^DONALD^D|1|19241010|111 DUCK ST^^FOWL^CA^999990000|||||||||||||||||056269770||||||PT|M|111^DUCK ST^^FOWL^CA^999990000|||||8291 IN2|2||123121234|Cartoon Ducks Inc||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||8885551212 IN1|3|SELF PAY|1|SELF PAY|||||||||||5||1

The problem with this is that out of those 2,000 or so errors there really are only a very few that are relevant and truly an error. Will gray log be able to pull the errors I want out with the corresponding message as shown above so I can more easily go through my daily logs?

2

the question is - is that above one line or multiple? What is the identifier? What is the structure of the log? is any pattern given?

If a pattern is present parsing isn’t a problem - well, with no pattern it is nearly impossible.

3

Hi
is this HL7 Protocol? if so, i might be able to help you here. if so, we can continue off topic

thanks
Hana

4

the community would learn when you share how to parse that.

5

Hi

i agree. i did hl7 parsing using other tool and have java code for it. quick solution with graylog will be setting a pipeline function to extract all fields based on hl7 spec, and init them on the message level .

thanks
Hana

6

do you mind, creating a feature request over at github that Graylog can parse tthe hl7 spec messages?

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.