# FileBeat and Sidecar: who trim the message?

**URL:** https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728
**Category:** Graylog Central (peer support)
**Tags:** sidecar
**Created:** [October 10, 2017, 3:57pm UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728 "2017-10-10T15:57:15Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [October 10, 2017, 3:57pm UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/1 "2017-10-10T15:57:16Z")

</div>

Hi,  
I’d like to share with you this strange (for me) filebeat behaviour.

I’m using filebeat to read a log file (MySql generic log).  
this is an extract of the log file:

```
171009 14:04:26 7 Connect	root@10.10.100.53 on 
		            7 Query	select @@version_comment limit 1
171009 14:06:26 7 Connect	root@10.10.100.55 on 

```

Splitting the message using \t we can parse the message.

The strange situation is that on graylog the massage value that I can use and parse is **trimmed**

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/1X/f9eb670cefee7bf980dec93dad6ac0272a8c21a0.png)

The empty spaces are missing! Why?

So, can somebody explain me what happen? Who trim the message?  
Is this the correct behaviour?

Thanks  
Gianluca

---

<div class="post-metadata">

### Author: ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)
#### Post date: [October 10, 2017, 4:08pm UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/2 "2017-10-10T16:08:59Z")

</div>

What’s your Filebeat Configuration?

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [October 10, 2017, 8:10pm UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/3 "2017-10-10T20:10:03Z")

</div>

Hi @jochen  
this is my filebeat.yml

```
filebeat:
  prospectors:
  - encoding: UTF-8
    fields:
      collector_node_id: graylog-collector-sidecar
      gl2_source_collector: 6439f440-9cd8-4793-948e-ff3937d73d45
      source_name: DBMS
      type: log
    ignore_older: 0
    paths:
    - /var/lib/mysql/mysql-mock.log
    scan_frequency: 10s
    tail_files: true
    type: log
output:
  logstash:
    hosts:
    - 10.10.100.211:5047
path:
  data: /var/cache/graylog/collector-sidecar/filebeat/data
  logs: /var/log/graylog/collector-sidecar
tags:
- mysqlmock

```

that comes from this Graylog collectors configurations:

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/1X/f28cb81a691171b2e1f48d75819f2eaf02a4da8e.png)  
 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/1X/3cc373852b831aaa6e3851fe793fdf39540e98b3.png)

Do you think that there could be a misconfiguration?

Thanks  
Gianluca

---

<div class="post-metadata">

### Author: ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)
#### Post date: [October 11, 2017, 6:46am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/4 "2017-10-11T06:46:02Z")

</div>

> [@gianluca-valentini](#):
>
> Who trim the message?
> 
> Is this the correct behaviour?

Graylog trims string values in messages automatically:

> <https://github.com/Graylog2/graylog2-server/blob/2.3.1/graylog2-server/src/main/java/org/graylog2/plugin/Message.java#L323>

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [October 11, 2017, 7:52am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/5 "2017-10-11T07:52:56Z")

</div>

Ok thanks.  
But I don’t understand why the messagge is trimmed.  
How you can manage a message that comes from a positional log?

In my case, but there are others example like SAP log, the position is as important as the presence of white spaces in the row.

So how I could resolve my problem? Is possible to find a workaround?  
Gianluca

---

<div class="post-metadata">

### Author: ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)
#### Post date: [October 11, 2017, 8:22am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/6 "2017-10-11T08:22:02Z")

</div>

I think you probably want to merge the two log lines instead of having them as separate messages in Graylog.

You can use the multiline feature (see the Graylog Collector Sidecar configuration) for this.

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [October 11, 2017, 8:43am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/7 "2017-10-11T08:43:42Z")

</div>

Hi @jochen

I’m reading the MySql general query log (tha is a general record of what mysq is doing)  
So each row is different from the others and could be not related with the before one.  
I don’t know if merging the timestamp could be a solution even if it is intuitively correct.

Can you explain me how does the multiline features works? Can you give me an example please?

For example, the MySql file ends with the SQL command executed and sometimes it can be null

`171011 10:09:11 11 Quit`

Is the the possibility that the multiline adding something different then the ‘null’ sql command (that is the correct value)?

Thanks  
Gianluca

---

<div class="post-metadata">

### Author: ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)
#### Post date: [October 11, 2017, 8:58am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/8 "2017-10-11T08:58:25Z")

</div>

> [@gianluca-valentini](#):
>
> Can you explain me how does the multiline features works? Can you give me an example please?

Please refer to [Managing Multiline Messages | Filebeat Reference [5.6] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/5.6/multiline-examples.html)

---

<div class="post-metadata">

### Author: ![gianluca-valentini](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gianluca-valentini/32/1592_2.png) [@gianluca-valentini](https://community.graylog.org/u/gianluca-valentini)
#### Post date: [October 11, 2017, 8:59am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/9 "2017-10-11T08:59:49Z")

</div>

Thanks a lot.

I will take a look to your suggested link

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)
#### Post date: [October 25, 2017, 9:00am UTC](https://community.graylog.org/t/filebeat-and-sidecar-who-trim-the-message/2728/10 "2017-10-25T09:00:06Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
