Elasticsearch Indexing Issue

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
I am trying to link Wazuh Indexer and Graylog.

I am encountering an issue with bulk indexing in Graylog. The error message suggests a problem with parsing the response body from Elasticsearch, resulting in a NullPointerException. This error occurs repeatedly during the indexing process.

Error message :

graylog-1  | 2024-07-26 13:10:12,102 WARN : org.graylog2.indexer.messages.Messages - Caught exception during bulk indexing: ElasticsearchException{message=ElasticsearchException[An error occurred: ]; nested: IOException[Unable to parse response body for Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1, host=https://X.X.X.X:9200, response=HTTP/1.1 200 OK}]; nested: NullPointerException;, errorDetails=[]}, retrying (attempt #23).

2. Describe your environment:

  • OS Information:
    Ubuntu 22.04

  • Package Version:
    Graylog 6.0
    MongoDB Version: 6.0.14
    Wazuh Indexer Version : 4.8.1

  • Service logs, configurations, and environment variables:
    Docker-Compose Configuration :

version: '3'
services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: "mongo:6.0.14"
    container_name: mongodb
    volumes:
      - "mongodb_data:/data/db"
    restart: on-failure

  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    hostname: "server"
    image: "graylog/graylog:6.0"
    # To install Graylog Open: "graylog/graylog:6.0"
    depends_on:
      mongodb:
        condition: service_started
    entrypoint: ["/usr/bin/tini", "--", "wait-for-it", "X.X.X.X:9200", "--", "/docker-entrypoint.sh"]
    environment:
      GRAYLOG_REPORT_DISABLE_SANDBOX: "true"
      GRAYLOG_SERVER_JAVA_OPTS: -Djavax.net.ssl.trustStore=/usr/share/graylog/custom_keystore.jks -Dlog4j2.formatMsgNoLookups=true -Djavax.net.ssl.trustStorePassword=changeit
    ports:
      - "9000:9000/tcp"
      - "5044:5044/tcp"
      - "5140:5140/tcp"
      - "5140:5140/udp"
      - "12201:12201/tcp"
      - "12201:12201/udp"
      - "13301:13301/tcp"
      - "13302:13302/tcp"
      - "5555:5555/tcp"
      - "5555:5555/udp"
    volumes:
      - ./graylog/config:/usr/share/graylog/data/config
      - "graylog_data:/usr/share/graylog/data/data"
      - "graylog_journal:/usr/share/graylog/data/journal"
      - ./custom_keystore.jks:/usr/share/graylog/custom_keystore.jks
    restart: on-failure

volumes:
  mongodb_data:
  graylog_data:
  graylog_journal:

4. How can the community help?

  • Provide guidance on resolving the bulk indexing issue with Elasticsearch.
  • Any insights into what might be causing the NullPointerException during the bulk indexing process.
  • Suggestions for additional debugging steps or configurations to check.

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

It looks like you’re missing GRAYLOG_ELASTICSEARCH_HOSTS. See docker-compose/open-core/docker-compose.yml at 4df601dd5988cc01b26371889820b1447cce46dd · Graylog2/docker-compose · GitHub

This configures graylog to connect to the desired OpenSearch cluster.

See Server.conf for more information.

3

Hey @Som-gif,

Could you give a little more insight into what exactly it is you are attempting here?

4

Hello, Thank you for your answer!

In my docker-compose.yml, I’m mounting as a volume the graylog.conf file, and in that file I have that line :

elasticsearch_hosts = https://admin:XXXXXX@XX.XX.XX.XX:9200

Here

./graylog/config:/usr/share/graylog/data/config

I’ve done what is indicated in “Custom configurations File” section of :
Graylog docker installation

5

Hello, thanks for your answer, I m actually trying to link my Wazuh Indexer to my Graylog, to forward and save my logs directly into my Wazuh Indexer.

6

Hello everyone,

I was able to resolve my issue, and I’m sharing the solution in case anyone else encounters the same problem:

In your Wazuh Indexer configuration file (wazuh.indexer.yml for a Docker deployment), you need to disable the compatibility override version option. This option allows Wazuh Indexer to override the OpenSearch version name with the Elasticsearch version. Therefore, you should change the parameter from true to false:

compatibility.override_main_response_version: false
1 Like
7

Nice work and thanks for sharing @Som-gif!

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.