Unexpected end-of-input - size limit?

Graylog Central (peer support)
1

Hi,

we send our apache accesslogs with our own logformat in GELF to graylog, e.g.

LogFormat “{ “version”: “1.1”,
“host”: “%h”,
”_GEOIP_COUNTRY_CODE": " %{GEOIP_COUNTRY_CODE}e",\

…

“short_message”: “%r” }" extended_gelf

Sometimes graylog can’t parse the shortmessage with an Unexpected end-of-input:

com.fasterxml.jackson.core.JsonParseException: Unexpected end-of-input: was expecting closing quote for a string value

The error always occurs at column 4097. Is there a size limit of 4096 characters for GELF / JSON here ?

Thanks
Jürgen

2

What’s the exact LogFormat configuration of your Apache httpd?
How are you shipping these manually built GELF messages to Graylog?
How did you take care of properly escaping the variable values (e. g. %r)?

Also take a look at

3

Hi Jochen,

We can’t use the mod_log_gelf because it use mpm_prefork :frowning:

The complete LogFormat directive is

#extended_gelf für Graylog
LogFormat "{ \"version\": \"1.1\",\
 \"host\": \"%h\",\
 \"_GEOIP_COUNTRY_CODE\": \" %{GEOIP_COUNTRY_CODE}e\",\
 \"_User\": \"%u\",\
 \"timestamp\": %{%s}t,\
 \"_Status\": \"%>s\",\
 \"_user_agent\": \"%{User-Agent}i\",\
 \"_X-Forwarded-For\": \"%{X-Forwarded-For}i\",\
 \"_ServerName\": \"%v\",\
 \"_Local_IP-address\":  \"%A\",\
 \"_Port\": \"%p\",\
 \"_Response-Handler\": \"%R\",\
 \"_BALANCER_WORKER_ROUTE\": \"%{BALANCER_WORKER_ROUTE}e\",\
 \"_Connection_status\": \"%X\",\
 \"_Cookie\":\"%{cookie}n\",\
 \"_UNIQUE_ID\": \"%{UNIQUE_ID}e\",\
 \"_SSL_PROTOCOL\": \"%{SSL_PROTOCOL}x\",\
 \"_SSL_CIPHER\": \"%{SSL_CIPHER}x\",\
 \"_Bytes_received\": \"%I\",\
 \"_Bytes_send\": \"%O\",\
 \"_ratio\": \"(%{ratio}n%%)\",\
 \"_Request_time\": \"%D\",\
 \"_ModSecTimeIn\": \"%{ModSecTimeIn}e\",\
 \"_ApplicationTime\": \"%{ApplicationTime}e\",\
 \"_ModSecTimeOut\": \"%{ModSecTimeOut}e\",\
 \"_ModSecAnomalyScoreIn\": \"%{ModSecAnomalyScoreIn}e\",\
 \"_ModSecAnomalyScoreOut\": \"%{ModSecAnomalyScoreOut}e\",\
 \"short_message\": \"%r\" }" extended_gelf

Shipping per Apache:

CustomLog  "|/bin/nc -u XXX.XXXX.de 9333"  extended_gelf

If there was a problem with the escaping, there would be other error messages, too.

4

You’re using UDP which, depending on the underlying network, might have a restriction in packet size.
Try using TCP.

FWIW, Graylog itself doesn’t have a size restriction for GELF messages and Elasticsearch happily ingests message fields up to 32 KB each.

The problem with unescaped variable values remains.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.