Error when using syslog-ng graylog2() with Graylog Central

I’m using syslog-ng graylog2() with Graylog GELF TCP input.
The conf of syslog-ng is as below.

destination redbull_graylog_test {
graylog2( host("") port(9002) persist-name("redbull_graylog_test"));

But I got the error log on graylog server as below.

2022-10-21T09:27:22.477+08:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=8309cca0-50df-11ed-8dff-0242c2d85535, messageQueueId=2068049, codec=gelf, payloadSize=171, timestamp=2022-10-21T01:27:22.474Z, remoteAddress=/} on input <6351ef6dfb510b3ff8b947ad>.
2022-10-21T09:27:22.477+08:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=8309cca0-50df-11ed-8dff-0242c2d85535, messageQueueId=2068049, codec=gelf, payloadSize=171, timestamp=2022-10-21T01:27:22.474Z, remoteAddress=/}
com.fasterxml.jackson.core.JsonParseException: Unexpected character (',' (code 44)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (String)"{"version":"1.1","timestamp":1666315642,"short_message":"java.lang.NullPointerException","level":5,"host":"localhost","_program":"","_pid":,"_facility":"user","_class":""}"; line: 1, column: 141]
        at com.fasterxml.jackson.core.JsonParser._constructError( ~[graylog.jar:?]
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError( ~[graylog.jar:?]
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar( ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue( ~[graylog.jar:?]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextFieldName( ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject( ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize( ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize( ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose( ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper.readTree( ~[graylog.jar:?]
        at org.graylog2.inputs.codecs.GelfCodec.decode( ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage( ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent( [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent( [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent( [graylog.jar:?]
        at [graylog.jar:?]
        at com.codahale.metrics.InstrumentedThreadFactory$ [graylog.jar:?]
        at [?:?]

It seems the “” empty string is missing after “_pid”:.

Could anyone tell me what is wrong about my issue? Thank you very much.

Hello @daemondong

From what I see in the logs

Unable to decode raw message RawMessage

And this

 codec=gelf, payloadSize=171

So when you get this…

[Source: (String)"{"version":"1.1","timestamp":1666315642,"short_message":"java.lang.NullPointerException"

The NullPointerException occurs due to a situation in application code where an uninitialized object is attempted to be accessed or modified. Essentially, this means the object reference does not point anywhere and has a null value.

Try using a different input.

EDIT: I looked into this a little more. Unfortunately, I don’t use syslog-ng but during my Googling for this issue


After looking at this…

destination redbull_graylog_test {graylog2( host(“”) port(9002) persist-name(“redbull_graylog_test”));};

I came across a statement about double quotes might be the issue, if so, then preprocess the string with one of the following before passing it to Jackson:

Remove the double quoting: newValue = initialValue.replaceAll("\"\"", "\"");

1 Like

I think the problem is when the Graylog analysis the string "

{"version":"1.1","timestamp":1666315642,"short_message":"java.lang.NullPointerException","level":5,"host":"localhost","_program":"","_pid":,"_facility":"user","_class":""}" it found there is NO "" after "_pid":, so it showed "Unexpected character (',' (code 44))". You can find there is "" after "_program":

I couldn’t find the way to add the _pid parameter is the source conf as below.

source redbull_log {
       file("/var/jxzc/enjoyment/logs/starter.log" follow_freq(1) multi-line-mode(regexp) multi-line-prefix("(ERROR|java)") flags(no-parse));

Maybe if the Graylog think the [:,] is the same as [:“”,] the problem will be fixed.
Thanks for your response.

I think you are correct, most, if not all the time "," need to escape those.

Hi gsmith

What should I do about this issue for now? Will the Graylog fix this issue on next version ?

Not sure, if you think its a bug you can post your issue here

As stated before, I have not used Syslog-ng, but after googling that error from above either the syslog-ng needs to different configuration or perhaps try a different input on graylog. Either way Graylog does not like it. Maybe someone else here uses Syslog-ng,

I was looking at this.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.