Sidecar Multiline config

Hi!

I’ve been working on a workaround for the limitations of the pipeline-grok in terms of multiline messages and I’ve successfully come up with a somewhat hacky solution that at least works in the pipeline simulator.

However, as it seems, my Beats sidecar is not even transmitting the multiline messages to GrayLog for further processing, even when disabling all my pipelines and filters, Beats will not send any of the multiline java exceptions I throw at it into GrayLog.

My Beats config looks like that:

filebeat.inputs:
- type: log
  multiline:
    pattern: '^\d{4}-'
    negate: true
    match: after
  enabled: true
  max_bytes: 33554432

  paths:
    - "/PATH/PATH/log1.log"
  scan_frequency: 1s
  fields.log_type: "glassfish"
    
- type: log
  multiline:
    pattern: '^\d{4}-'
    negate: true
    match: after
  enabled: true
  max_bytes: 33554432
  
  paths:
    - "/PATH/PATH/log.log"
  fields.log_type: "jetty"
    
output.logstash:
   hosts: ["localhost:5044"]
   codec.format:
     string: '%{[message]}'
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

Anything wrong with that? It’s basically the same four lines of config that make my ElasticStack read multiline messages and the config, apart from the GrayLog specific aspects, should be the same, right?

He Erik,

without knowledge about your log - this should work. But the pattern might be wrong for your logs.

It does not make a difference what receive the logs, if filebeat reads the message correctly, it will be received by Graylog the same way.

Hi Jan!

To test my filebeat config, I’ve installed LogStash and rerouted the Beats output there; it works fine.

However, when I fire up Graylog again, there’s no incoming messages, even after disabling all filtered streams and letting everything pass through All Messages (it’s just a test install).

If I manually write one line messages and echo them into my test file, the Beat will quickly transmit them.

Is the Graylog sidecar functionality the same as the legacy collector sidecar? Because as I’ve seen from previous forum posts about multiline, there’s a GUI in the legacy version that let’s you do the multiline configuration outside of the actual config file. Maybe I should try that?

Ok, I’ve tried around a bit constructing my own pseudo-log messages and I noticed a strang behaviour. Without any applied filters or such the “log”

1337-01-01 13:37:12,345 Freude schöner Götterfunken Tochter aus Elysium
wir betreten feuertrunken
himmlische dein Heiligtum

Will be transferred to Graylog, it pops up in my search tab, but the message that actually reaches Graylog is only

ysium
wir betreten feuertrunken
himmlische dein Heiligtum

Same behaviour in Logstash. Beats cuts off the beginning of the line for no apparent reason. I removed the max_bytes setting and it seems to send full messages now, I’ll post an edit if this solved my problem.

EDIT: Ok, I guess my problem is solved, full multiline messages are sent into Graylog. My pipeline does not trigger, even though according to the Simulator it should work well, but that’s another topic. /thread