Number of lines in gralog messages section at once

Graylog Tech Challenges
1

Description of your problem

Does graylog have a limit on the number of lines it can ingest into 1 log file in the message section of the log file in graylog?

Description of steps you’ve taken to attempt to solve the issue

I added the multi line options in my filebeat config so that it can ingest all my log file in 1 log file in filebeat. The log file on my server is an active log file so that means data gets appended to that file. While it is being appended that log file is sent via filebeat to graylog.

I did a test and copied over the same file but this time it was not active and had all the data in the log file. But then it only copied over certain amount of data not the whole file in 1 log file in graylog.

Operating system information

  • CentOS 7.9

Package versions

  • Graylog 4.1.
  • MongoDB 4.2.14
  • Elasticsearch 7.10.2-1
  • Filebeat 7.14.0-1

My multi line added to logfile

```
  multiline.type: pattern
  multiline.pattern: '-*.TRANSFER START'
  multiline.negate: true
  multiline.match: after
```
2

Hello,

Not that I know of.

multiline.type
Defines which aggregation method to use. The default is pattern. The other option is count which lets you aggregate constant number of lines.

Here is some information for you if you haven’t seen it already.

test_your_regexp_pattern_for_multiline

Here is the link for that Below for testing.

Hope that helps

3

I got it to work by using the multiline.max_lines option.

1 Like
4

Good Job.:slight_smile:
I assume this is what solved your issue?

multiline.max_lines
The maximum number of lines that can be combined into one event. If the multiline message contains more than max_lines, any additional lines are discarded. The default is 500

5

Yes that’s correct. I put 5000 just to see if it does strip out more than 500 or not.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.