Combining split logs

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
Is there a way to tell graylog when the line has the timestamp at the start its a new message. Graylog is making these logs into 10 different messages when it should just be 3.

  • OS Information:
    Graylog 5.2.3+9aee303
Example:
2024-01-29 09:46:24,8566 [13] DEBUG XML from client: <?xml version="1.0" encoding="utf-16"?><Transaction commitRollbackYN="N"><WhoIsResponse xmlns:xsd="###" xmlns:xsi="###" MessageDateTime="2024-01-29T07:59:25.7958798-06:00" TrackingId="##" Successful="true" ExceptionCode="None" Signature="###06:00&lt;/ExpirationDate&gt;&lt; reqTypCd="###" reqNbr="" /></Requests></Input></Transaction>

2024-01-29 09:46:24,8877 [13] DEBUG XML from server: <Transaction>
  <Output>
    <UserAuthentication successYN="Y" persNbr="###"/>
    <Responses>
      <Response reqNbr="" reqTypCd="###" successYN="Y">
        <NextBusinessDate>01/30/2024</NextBusinessDate>
        <PreviousBusinessDate>01/28/2024</PreviousBusinessDate>
        <CurrentVersion>####</CurrentVersion>
        <CurrentBundleVersion>###</CurrentBundleVersion>
        <CurrentServiceVersion>###</CurrenServiceVersion>
      </Response>
    </Responses>
  </Output>
</Transaction>

2024-01-29 09:46:24,88167 [13] INFO   | 0 | ### | ### | ### | True
2

Can you share how you are shipping these logs to graylog?

3

Through a filebeat input, it pulls from a .txt file on a windows server.

4

I recommend configuring filebeat to use multiline parameters which will have filebeat combine the messages before shipping to graylog.

5

If anyone is interested this is the multiline parameters I added

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\the\tree\of\life
  multiline.type: pattern
  multiline.pattern: ((?:19|20)\d\d)[- /.](0[1-9]|1[012])[- /.](0[1-9]|[12][0-9]|3[01]) (\d\d:\d\d:\d\d)
  multiline.negate: true
  multiline.match: after
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.