Logs are not in the order


(Kieulam141) #1

Hi Everyone.

I use collector-sidecar and filebeat to collect logs. I got 1 problem about the order of the logs. In my file the order still right order. Here is the picture.

Then in Graylog UI, I got other order of logs. It’s very hard for us to trace log. Here is the picture.

It’s my filebeat configuration:

`filebeat:
prospectors:

  • document_type: log
    encoding: plain
    fields:
    collector_node_id: graylog-collector-sidecar
    gl2_source_collector: xxx
    ignore_older: 0
    input_type: log
    paths:
    • /var/log/*
      symlinks: true
      scan_frequency: 10s
      tail_files: true
      output:
      logstash:
      hosts:
    • xxx
      path:
      data: /var/cache/graylog/collector-sidecar/filebeat/data
      logs: /var/log/graylog/collector-sidecar
      tags:
  • xxx`

Any one can help me to fix this problem? Thanks so much !!!


(Jan Doberstein) #2

That is the reason why you split your logfile into single fields and after that you are able to group and search the information in the way you need them to be worked on.


(Kieulam141) #3

You mean I should split our logfile as GELF.


(Jan Doberstein) #4

that would be one solution. Or you use Regex or Grok (or something that is working for you) to normalize the data you already have.

This way you would have a field that only holds the component of your stack, another the version, and so on.


(Kieulam141) #5

OK Thanks.

Maybe I should use pipeline to convert timestamp.


(system) #6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.