Capture part of log starting from to as entire event

Please advise is there any possibility to capture stream in graylog for future processing?
I mean starting “from” “to” using pattern and.

For example:


Currently graylog breaks into pieces entire event. Which should not be, but so as it is something like JSON (multiline) graylog consider each line of JSON lile a separate message.

Which function i should use this in order to avoid it and get each event intact from: info:START_REQUEST
To: info: END_REQUEST?


What data are you shipping and via what method?

Hello @megan201296
It is data from docker container api.
I am using fluentd agent for shipping.
Here is my fluentd config file.

  type tail
  read_from_head true
  pos_file fluentd-docker.pos
  path /var/lib/docker/containers/*/*-json.log
  time_format %Y-%m-%dT%H:%M:%S
  tag docker.*
  format json

<match docker.**>
  type copy
  buffer_type memory
  buffer_chunk_limit 256m
  buffer_queue_limit 128
  flush_interval 1s
  disable_retry_limit false
  retry_limit 17
  retry_wait 1s
    type gelf
    host #host name was changed
    protocol udp
    port 12201
    type stdout

Data structure (see example in attachment):

1 Like

Ah, I apologize as I do not have experience with fluentd. Hopefully someone else will be able to assist.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.