Graylog cluster not load balancer for all node

Graylog Central (peer support)
1

Hi all,
i setup graylog cluster for 4 node graylog
elasticsearch cluster: 4 node
mongo replica set: 4 node
kafka cluster: 3 node: 1,2,3
i create a topic on kafka cluster and i push log from logstash to kafka cluster
but, only 2 node to receive log from kafka cluster, no log for troubleshoot this proplem.
image
please help me check.

2

Hello @ducna09

Would like to help you but the lack of information is preventing that. perhaps take a look here

3

Operating system information :

ubuntu 20.04

Package versions

  • Graylog: Graylog 4.3.9+e2c6648
  • MongoDB cluster : v4.4.18
  • Elasticsearch cluster : 7.10.2
    ####my input: raw/plaintext kafka
    Legacy mode: none
    Bootstrap Servers (optional): ip:port kafka cluster
    ZooKeeper address (legacy mode only) (optional): none
    Topic filter regex: osquery
    Fetch minimum bytes: 5
    Fetch maximum wait time (ms): 100
    Processor threads:4
    Allow throttling this input: none
    Auto offset reset (optional): automatically reset the offset to the lastest offset
    Consumer group id (optional): graylog-sec-prod
    Custom Kafka properties (optional): none
    , this input traffic message only route to 2 node , and 2 node no message.i dont know why,
    let’s me know if you need more information.
4

Did this work before?

5

no, this is first time i install graylog cluster, i have another input work with 4 node, but this itput only work with 2 node, i think main cause at kafka cluster ? input kafka work correctly is another kafka, and kafka work wrong i install on the same server install graylog.

6

Hey,

It might be, not sure how you set it up or what configuration you have made.

7

i install confluent-kafka on 3 node, this is ckma config topic

image
image1808×836 38.5 KB

8

Hey,

Ok i see now, my applogies I have not worked with that software, I was refering to the configuration file for Elasticsearch/Opensearch. along with Graylogs configuration file. Unfortantly the screen shot does not show how the Cluster was configured, perhaps to get a better idea take a look here

https://go2docs.graylog.org/4-x/setting_up_graylog/multi-node_setup.html?Highlight=cluster

9

Hi Gsmith,
how graylog cluster multi node work with load balancers?
4 node Load balancer indication: ALIVE but message only route to 2 node.
no log for debug…

10

Hello,

https://go2docs.graylog.org/5-0/setting_up_graylog/web_interface.htm#making-the-web-interface-work-with-load-balancersproxies

Unforantly I do not know how you configure your setup, so im not sure what you did.

11

https://go2docs.graylog.org/5-0/setting_up_graylog/web_interface.htm#making-the-web-interface-work-with-load-balancersproxies
=> this is web interface lb, i dont know how to graylog handle route message to multi node for share traffic to multi node. no UI to config that.

Unforantly I do not know how you configure your setup, so im not sure what you did. => simple: i create input of kafka cluster and one topic listen port 9092 on 4 node, push message to input is ok, but message only route to 3 node
image
…

12

hello,

my apologies the screenshot only shows that node with red box has no activites. This might be a configuration/connection issue. not sure.

Perhaps someone else here has a better idea.

13

[quote=“gsmith, post:12, topic:26848”]
apologies the screenshot only shows that node with red box has no activites. This might be a configuration/connection issue. not sure.

Perhaps someone else here has a better idea.
[/quote] Thanks @gsmith for your prompt response my issue

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.