# FileBeat multiline How to

**URL:** https://community.graylog.org/t/filebeat-multiline-how-to/24529
**Category:** Graylog Central (peer support)
**Tags:** debuggingpl, documentation
**Created:** [June 28, 2022, 4:38pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529 "2022-06-28T16:38:52Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [June 28, 2022, 4:38pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/1 "2022-06-28T16:38:53Z")

</div>

Hi all,  
I’m using Graylog 4.1.6 and I’m using filebeat to send leg message into Graylog.  
My problem is how to configure fielbeat in order to send multiline messages.

Consider a simple json text

```auto
{
  "preview": false,
  "offset": 2491152,
  "result": {
    "_raw": "row message",
    "_time": "2022-04-01T00:00:13.000+0000",
    "host": "127.0.0.0",
    "index": "source.net",
    "source": "tenant",
    "sourcetype": "gianluca.localmachine",
    "_server": "server.ip"
  }
}
{
  "preview": false,
  "offset": 2491111,
  "result": {
    "_raw": "row message",
    "_time": "2022-04-01T00:00:13.000+0000",
    "host": "127.0.0.0",
    "index": "source.net",
    "source": "tenant",
    "sourcetype": "mike.localmachine",
    "_server": "server.ip"
  }
}

```

and so on…

Can somebody can help me to understand how set the sidercar.yml in order to get and parse correctly the multiline log file?  
Thanks a lot  
Gianluca

---

<div class="post-metadata">

### Author: ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)
#### Post date: [June 28, 2022, 7:33pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/2 "2022-06-28T19:33:38Z")

</div>

You can handle that in your configuration here are the docs from Elasticsearch on [filebeat-\>multiline](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html)

I found a random example of the multiline commands being used [here](https://community.graylog.org/t/if-you-wanted-to-use-filebeat-with-add-docker-metadata-processor-rejoice-theres-a-dirty-solution-to-make-it-work/7862)… that post doesn’t solve the problem is is just an example of multiline processing in the sidecar configuration.

---

<div class="post-metadata">

### Author: ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)
#### Post date: [June 28, 2022, 11:47pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/3 "2022-06-28T23:47:28Z")

</div>

Adding on to @tmacgbay , I also found another example. This post did look familiar.

- [Filebeat Windows - Java Multiline Pattern](https://community.graylog.org/t/filebeat-windows-java-multiline-pattern/16732)

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [June 30, 2022, 7:05am UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/4 "2022-06-30T07:05:15Z")

</div>

Hi @gsmith @tmacgbay  
thanks a lot for your help  
I’m going to solve my scenario and I will share the solution here hoping that could help someone in the future

Thanks

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [June 30, 2022, 2:36pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/5 "2022-06-30T14:36:59Z")

</div>

Hi all  
I tested

```auto
{
	"test": true
}

```

using the configuration below:

```auto
multiline.pattern: '^\{'
multiline.negate: true
multiline.match: after

```

as suggested [here](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#multiline)

but it does not works. I tested multiline.pattern: ‘^{’ too without result.  
The json is splitted in more then one message  
 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/8/80a72d7a0af63587beb78c79f1c0ddb42c95e643.png)

Any suggestions?

UPDATE  
I add my collector configuration

```auto
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: ["myhost:5044"] 
path:
  data: C:\Program Files\Graylog\sidecar\cache\filebeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - s-aad 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:/splunk-logs/test/*.json
# Multiline https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html
multiline.type: pattern
multiline.pattern: '^{'
multiline.negate: false 
multiline.match: after

```

---

<div class="post-metadata">

### Author: ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)
#### Post date: [June 30, 2022, 3:06pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/6 "2022-06-30T15:06:31Z")

</div>

Your post and it’s edit conflict in what your multiline pattern settings are, as I read it the top one where it says this:

```auto
multiline.pattern: '^\{'
multiline.negate: true
multiline.match: after

```

should work the way that you want.

The other important thing to note is that yml configuration files are picky about indentation. Your multiline configurations should be indented at the same level as `paths:`

```auto
...
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:/splunk-logs/test/*.json
  # Multiline https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html
  multiline.type: pattern
  multiline.pattern: '^{'
  multiline.negate: true
  multiline.match: after
...

```

The indentation means that the multiline applies specifically to the `paths:` above it… this comes into play if you have a configuration file the has different settings for different paths. Some paths: may not be multiline.

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [June 30, 2022, 3:08pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/7 "2022-06-30T15:08:00Z")

</div>

Thanks @tmacgbay  
let me try

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [June 30, 2022, 5:15pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/8 "2022-06-30T17:15:56Z")

</div>

Hi,  
finally I solve it

I just add the configuration from filebeat.inputs:

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:/splunk-logs/aad-test/test.json

# Multiline https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html
  multiline.type: pattern
  multiline.pattern: '^{'
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 5000
  multiline.timeout: 10

processors:
 - decode_json_fields:
     fields: ['message']
     target: ""
     process_array: true
     max_depth: 8
     overwrite_keys: true

```

Now I can send a json multifile to a single message text into my graylog and finally can start the parsing operation

---

<div class="post-metadata">

### Author: ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)
#### Post date: [June 30, 2022, 6:10pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/9 "2022-06-30T18:10:09Z")

</div>

Mark it as solved for future searchers! 😃

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)
#### Post date: [July 14, 2022, 6:10pm UTC](https://community.graylog.org/t/filebeat-multiline-how-to/24529/10 "2022-07-14T18:10:23Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
