How to configure kafka output with topic and credentials

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
how to configure kafka output with sasl.jaas.config with kafka topic name and it’s credentials to send logs in Kafka from Graylog server

2. Describe your environment:

  • OS Information:
    I have Graylog server running with 5.x version on RHE8 OS

  • Package Version:
    graylog-server-5.0.7-1.x86_64

3. What steps have you already taken to try and solve the problem?
I have created the kafka output but didn’t find option to fill sasl credentials, hence updated in server.conf as below
output_module = kafka
output_module_kafka_topic = topic1
output_module_kafka_bootstrap_servers = jann2.kf2.net:9093
output_module_kafka_security_protocol = SASL_PLAINTEXT
output_module_kafka_ssl_truststore_location = /etc/pki/kafka/RootCA01.crt
output_module_kafka_sasl_mechanism = PLAIN
output_module_kafka_sasl_jaas_config = org.apache.kafka.common.security.plain.PlainLoginModule required username=“abc” password=“azxswq”;

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

To make sure I understand fully, are you looking to send logs FROM kafka TO graylog (kafka → graylog), OR
FROM graylog TO kafka (graylog → kafka).

Thanks.

3

I’m looking to forward logs from graylog to Kafka topic with SASL authentication.

4

Could you please assist me to transfer logs from Graylog to Kafka topic with SASL credentials.

5

As far as I know there is no official or supported way to output messages from graylog to kafka. I did find an unofficial output plugin, but i have not tested it and its several years old so may no longer work. GitHub - asnowfox/graylog-kafka-output . I’m also not clear on if it supports SASL.

I also found this document graylog-guide-syslog-kafka/README.md at master · Graylog2/graylog-guide-syslog-kafka · GitHub but it more so talks about ingesting messages in kafak and then sending to graylog.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.