REST API present in syslog comes in distorted format

Graylog Central (peer support)
1

Hi all,

I am using Graylog 2.4.3.

My flow is as follows:

Logs(mobile,computer etc)—>Filebeat------------->Graylog–>Log Analysis server

When the logs reaches from the filebeat to the graylog it comes in a right format to /var/log/syslog folder in graylog server.

But the graylog output is coming in distorted format in multiline and sent to Log analysis server.

Content-Type: application/json;charset=UTF-8;ver=0.9
Response-Code: 200
Content-Type: application/json;charset=UTF-8;ver=0.9
ID: 11122610
Headers: {Content-Length=[0], Date=[Tue, 12 Dec 2018 08:49:15 GMT]}
ID: 11122611
ID: 11122603
Content-Type: application/json
Headers: {Date=[Tue, 12 Dec 2018 08:49:18 GMT], Content-Type=[application/json;charset=UTF-8;ver=0.9]}

Looking at the above output it looks like there is some problems while the graylog output format which fails to parse certain REST APIs. You can see there are 3 message IDs(11122610,11122611,11122603) comes in different lines. It only happens for certain REST APIs.

Is it something to do with maximum message length or message type of output as mentioned in below:

Referring to the GitHub link :

  1. If I need to send any message length >16384 bytes in Maximum message length field can I pass as 0 or anything else? Similarly can i send message type as “full”

  2. Can anyone provide some clues what could be the wrong which results ?

Any help is really appreciated.

Regards
Pradeep

2

When the logs reaches from the filebeat to the graylog it comes in a right format to /var/log/syslog folder in graylog server.

But if logs are in /var/log/syslog you are not using Graylog at all.

3

I mean whatever logs I receive from the filebeat to graylog and it is passed from graylog to the log analyzer. When the log reaches the log analyzer the logs are in multiple lines which is difficult to parse. I am suspecting that there is some problem either the format that we are sending from graylog server to the log analyzer server. As from the filebeat(/var/log/*) to the graylog it comes properly in a single line.

Currently message format it is using as per the below Output plugin is “plain”. May be I need to use “structured” or “full” and the maximum message length is exceeding the default length that is set.

Can anyone suggest what could be the issue?

4

he @PRADEEP13039356

did you checked if the events are proper formatted in Graylog? Means a multiline event is a single event in Graylog?

If that is the case - the output plugin might have a problem with that. More important syslog messages can’t be multiline, what could be the root.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.