How to handle node failure for Graylog Kafka and aws input plugin

Graylog Central (peer support)
1

Hi,

We are using two input plugins such as KAFKA and AWS given below,

Kafka:


AWS:

  • If we create input with global option selected, then kafka thread will be created in one of the randomly selected graylog node
  • If we choose any node while created the input, then kafka thread will be created in selected node

Currently we have 2 node graylog cluster and we found that whenever any graylog node with kafka thread running goes down then other node will not start kafak thread.
Also if we want to restart graylog node with kaka thread running, then we may loose the messages b/w restart time interval.

Please let us know how we can support kafka input heigh availability, so that we do not loose any log messages on graylog node failure or on graylog node restart.

Thanks in advance,
Umesh Dc

2

Which version of Graylog are you using?
Which version of the AWS plugin are you using?
Which version of the 3rd party Kafka plugin are you using?

If you have problems with the 3rd party Kafka plugin, please create a bug report in the respective repository:
https://github.com/fbalicchia/graylog-plugin-kafka-inout

3

Thanks Jochen for quick replay.

We are using,
Graylog: 2.3.2
AWS: 2.3.1
Kafka Version: 0.8.0

We are not facing any problem with plugin as such. Our concern is with respect to plugin run. Since it runs on only one node and if it goes down, how to handle this single point of failure.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.