Well, it does say that “each message needs to be delimited with a null byte”. So even if you’re just sending one
You’re doing some great work here! And it sounds like you’re getting very close to the answer.
I would actually suggest running a Netcat listener to do the other test I suggested. Send it both the test message through Netcat and the test message from your Oracle app.
and still I have the "“GELF message is too short…” error and the 45 records.
I gave the same even when double quotes are removed, the way of the documentation.
why should the same string from Oracle generate error and debug?
Just for shits and giggles, does Oracle treat \"0" the same as it would, say a bare \0 or even"\0" - good chance it may be that it figures you want to escape the single quote and not add a null byte.
quoted:
“The newline character sequence is appended to the message before it is transmitted.”
which means the newline is added as a last (!) action.
While in your doc, on GELF TCP, paragraph “Example payload”, it says:
Note
Newlines must be denoted with the `\n` escape sequence to ensure the payload is valid JSON as per [RFC 7159]
which tells me it is looking for some escape of newlines in the original text, which cannot be done since Oracle’s package by default adds it as a last action before delivery.
I think the newlines it’s talking about for the payload is if you are embedding newlines in your JSON message, and not the separator between messages. You want to set “newline” to the null byte (\0 ) and see if that makes the difference.
As far as I can tell from the docs, a GELF message is in JSON, any newlines inside the JSON payload must be escaped properly, but the separator between GELF messages is a null byte.
and it did work, I don’t have the cosmetic messages again.
thank you very much for suggesting Chr(0).
I had tried that before, but not inside UTL_TCP, but as an added string in the end.
A guy at stackoverflow recalled me that.
Important:
what troubles me most is the inability to identify the problem by Graylog own logs. I cannot find anything on the 45 messages to indicate me what is the error and how to be corrected.
Is there anything more to debug on Graylog? have more messages in hope of identifying the problem?
The thing is that you are trying to send GELF formatted output, using a raw TCP library - while it’s possible, nobody ever said it’d be easy
Set up a raw TCP input, have Oracle send the JSON data that way, at that point it won’t care what the format is, as long as each message is separated by a newline, after that you can attach processor/pipeline to parse the JSOn and sort it out from there - at this point this seems to be the better option
I didn’t make any extractor. My GELF TCP Input has:
Configured extractors
This input has no configured extractors.
I just send the JSON from Oracle PL/SQL and it gets written to Graylog.
This has attracted my attention, because in Splunk I had to set some KV_MODE=XML when
sending data by such (XML). While at Graylog I have none anywhere for the JSON.
May be the JSON extractor is default - and works even unconfined?
What you see is the raw json payload in the “message” field - if that works for you, then leave it as is. If you want to parse the JSON and convert the keys in the payload to actual fields in an event, you’ll have to run an extractor over it
but it would be best if you told me why I can get my JSON loaded without “declaring” it as a type somewhere. Is it a Graylog default? A GELF TCP Input default? Or a Graylog’s default index default?
You have the JSON “loaded” because it’s treated as a raw bit of text in the ‘message’ field of an event - but that’s all it is, right now. Graylog has not assigned meaning to any of it, as far as Graylog is concerned, it’s a text message (that just happens to be formatted in JSON). If you want to actually parse it as JSON and do useful things with the individual fields, that’s where the extractor comes in.
Also, the raw TCP input takes anything it receives and creates an event consisting of a timestamp (when the input was received), and a message field which is whatever content was sent.
But altink told us altink did not use raw tcp input, but GELF input. And GELF input does not necessarily need extractors, if the data sent to it is formatted properly.
GELF input still doesn’t parse the JSON - what I said about the raw tcp one also goes for GELF, if the json data is part of the “message” field then it gets stored, but not parsed.
I am using GELF TCP input. With data formatted as JSON. The later I have not declared anywhere.
Indeed, the GELF TCP Input was the only thing I needed to do inside Graylog to get the data in.
And I also do have all my custom _foo1, _foo2… _fooX fields.