# Change from Elasticsearch to opensearch: Node ID problem

**URL:** <https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370>\
**Category:** Graylog Central (peer support)\
**Tags:** docker\
**Created:** [May 11, 2024, 9:29pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370 "2024-05-11T21:29:45Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 11, 2024, 9:29pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/1 "2024-05-11T21:29:45Z")

</div>

Dear community,

I am running Graylog 5.2 with Elasticsearch in a Docker Stack. Everything is fine.  
Last night, I wanted to upgrade to Graylog 6.0 and make the change to opensearch.

Unfortunately, I cannot get it running.

- How can I verify, if the containers can reach each other? “ping” is not available within the graylog-container.
- When googling, this seems to be an old issue, but I did not find a working solution. How do I debug this?

I see this error log in the graylog container:

```auto
wait-for-it: waiting 15 seconds for elasticsearch:9200

wait-for-it: timeout occurred after waiting 15 seconds for elasticsearch:9200

2024-05-11T21:19:54.598201098Z main ERROR Reconfiguration failed: No configuration found for '50a4e294' at 'null' in 'null'

23:19:54.806 [main] INFO org.graylog2.featureflag.ImmutableFeatureFlagsCollector - Following feature flags are used: {default properties file=[frontend_hotkeys=on, field_types_management=on, cloud_inputs=on, data_tiering_cloud=off, scripting_api_preview=on, composable_index_templates=off, search_filter=on, preflight_web=on, data_node_migration=off, instant_archiving=off]}

23:19:54.813 [main] ERROR org.graylog2.bootstrap.CmdLineTool - Couldn't load configuration: Properties file /usr/share/graylog/data/config/graylog.conf doesn't exist!

```

my compose.yaml

```auto
# version: as May 2024 https://docs.docker.com/compose/compose-file/

services:
# MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: mongo:6.0.14
    container_name: mongodb
    hostname: mongodb
    environment:
      - TZ=Europe/Zurich
    restart: unless-stopped
    #DB in share for persistence
    volumes:
      - type: bind
        source: /home/uadmin/Docker/Graylog6/mongo_data/mongo_db
        target: /data/db
      - type: bind
        source: /home/uadmin/Docker/Graylog6/mongo_data/mongo_configdb
        target: /data/configdb
    deploy:
      resources:
        limits:
          memory: 500mb
    networks:
      graylog_backend:
        ipv4_address: 10.10.10.13

  opensearch:
    image: opensearchproject/opensearch:2.12.0
    container_name: opensearch
    hostname: opensearch
    volumes:
      - /home/uadmin/Docker/Graylog6/opensearch-data:/usr/share/opensearch/data
    environment:
      - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
      - "bootstrap.memory_lock=true"
      - "discovery.type=single-node"
      - "action.auto_create_index= false"
      - "plugins.security.ssl.http.enabled=false"
      - "plugins.security.disabled=true"
      # Can generate a password for `OPENSEARCH_INITIAL_ADMIN_PASSWORD` using a linux device via:
      # tr -dc A-Z-a-z-0-9_@#%^-_=+ < /dev/urandom | head -c${1:-32}
      - "OPENSEARCH_INITIAL_ADMIN_PASSWORD=lOULLLfs2h3TKicWH7tVAOaP4qvNu3UcYA7BfCxxxx"
      - TZ=Europe/Zurich
    ulimits:
      memlock:
        hard: -1
        soft: -1
      nofile:
        soft: 65536
        hard: 65536
    restart: unless-stopped
    networks:
      graylog_backend:
        ipv4_address: 10.10.10.14

# Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    image: graylog/graylog:6.0
    container_name: graylog
    hostname: graylog
    domainname: internal
    #journal and config directories in local NFS share for persistence
    volumes:
      - /home/uadmin/Docker/Graylog6/graylog_data:/usr/share/graylog/data
    entrypoint: "/usr/bin/tini -- wait-for-it elasticsearch:9200 -- /docker-entrypoint.sh"
    environment:
      - GRAYLOG_NODE_ID_FILE= "/usr/share/graylog/data/config/node-id"
      - GRAYLOG_HTTP_BIND_ADDRESS= "192.168.60.12:9000"
      - GRAYLOG_ELASTICSEARCH_HOSTS= "http://opensearch:9200"
      - GRAYLOG_MONGODB_URI= "mongodb://mongodb:27017/graylog"
      # To make reporting (headless_shell) work inside a Docker container
      - GRAYLOG_REPORT_DISABLE_SANDBOX= "true"
      # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_PASSWORD_SECRET= "y9H2-muMO-qtAS-xxxx"
      # Password: "admin"
      - GRAYLOG_ROOT_PASSWORD_SHA2= "52993186fc321cf6969fc160d99ed41869703000868814d30deddff4e795xxxx"
      - GRAYLOG_HTTP_EXTERNAL_URI= "http://192.168.60.3:9000/"
      - TZ=Europe/Zurich
    deploy:
      resources:
        limits:
          memory: 2gb
    networks:
      macvlan60:
        ipv4_address: 192.168.60.13
      graylog_backend:
        ipv4_address: 10.10.10.12
    links:
      - mongodb:mongo
      - opensearch
    restart: unless-stopped
    depends_on:
      mongodb:
        condition: service_started
      opensearch:
        condition: service_started
    ports:
      # Graylog web interface and REST API
      - "9000:9000/tcp"
      # Beats
      - "5044:5044/tcp"
      # Syslog TCP
      - "5140:5140/tcp"
      - "1514:1514" #(e.g. Unifi)
      - "1515:1515" #(e.g. Tasmota)
      # Syslog UDP
      - "5140:5140/udp"
      # GELF TCP
      - "12201:12201/tcp"
      # GELF UDP
      - "12201:12201/udp"
      # Forwarder data
      - "13301:13301/tcp"
      # Forwarder config
      - "13302:13302/tcp"

# Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/
volumes:
  mongo_db:
  mongo_configdb:
  opensearch-data:
  graylog_data:
    driver: local

# Network specifications
networks:
  macvlan60:
    external: true
  graylog_backend:
    internal: true
    ipam:
      driver: default
      config:
        - subnet: "10.10.10.0/24"

```

Any help is very much appreciated!

---

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 11, 2024, 11:07pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/2 "2024-05-11T23:07:10Z")

</div>

Update: I followed this chapter  
[https://go2docs.graylog.org/current/downloading\_and\_installing\_graylog/docker\_installation.htm#Configuration](https://go2docs.graylog.org/current/downloading_and_installing_graylog/docker_installation.htm#Configuration)

and got the missing graylog.conf from there.  
And I had to map the port 9200 for the opensearch container.

Now, I have this error message:

```auto
wait-for-it: waiting 15 seconds for opensearch:9200

wait-for-it: opensearch:9200 is available after 0 seconds

2024-05-12 01:05:29,261 INFO : org.graylog2.featureflag.ImmutableFeatureFlagsCollector - Following feature flags are used: {default properties file=[frontend_hotkeys=on, field_types_management=on, cloud_inputs=on, data_tiering_cloud=off, scripting_api_preview=on, composable_index_templates=off, search_filter=on, preflight_web=on, data_node_migration=off, instant_archiving=off]}

2024-05-12 01:05:29,466 ERROR: org.graylog2.bootstrap.CmdLineTool - Invalid configuration

com.github.joschi.jadconfig.ValidationException: Parent path "/usr/share/graylog/data/config for Node ID file at "/usr/share/graylog/data/config/node-id" is not a directory

	at org.graylog2.Configuration$NodeIdFileValidator.validate(Configuration.java:572) ~[graylog.jar:?]

	at org.graylog2.Configuration$NodeIdFileValidator.validate(Configuration.java:560) ~[graylog.jar:?]

	at com.github.joschi.jadconfig.JadConfig.validateParameter(JadConfig.java:227) ~[graylog.jar:?]

	at com.github.joschi.jadconfig.JadConfig.processClassFields(JadConfig.java:143) ~[graylog.jar:?]

	at com.github.joschi.jadconfig.JadConfig.process(JadConfig.java:101) ~[graylog.jar:?]

	at org.graylog2.bootstrap.CmdLineTool.processConfiguration(CmdLineTool.java:505) [graylog.jar:?]

	at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:293) [graylog.jar:?]

	at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:267) [graylog.jar:?]

	at org.graylog2.bootstrap.Main.main(Main.java:55) [graylog.jar:?]

```

Any ideas?  
I would expect it to create a random node-id?

---

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 12, 2024, 7:53am UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/3 "2024-05-12T07:53:42Z")

</div>

I have spend the last night on this and I am stuck.

I still keep this error:

```auto
2024-05-12 09:49:44,403 ERROR: org.graylog2.bootstrap.CmdLineTool - Invalid configuration

com.github.joschi.jadconfig.ValidationException: Parent path "/usr/share/graylog/data/config for Node ID file at "/usr/share/graylog/data/config/node-id" is not a directory

	at org.graylog2.Configuration$NodeIdFileValidator.validate(Configuration.java:572) ~[graylog.jar:?]

	at org.graylog2.Configuration$NodeIdFileValidator.validate(Configuration.java:560) ~[graylog.jar:?]

	at com.github.joschi.jadconfig.JadConfig.validateParameter(JadConfig.java:227) ~[graylog.jar:?]

	at com.github.joschi.jadconfig.JadConfig.processClassFields(JadConfig.java:143) ~[graylog.jar:?]

	at com.github.joschi.jadconfig.JadConfig.process(JadConfig.java:101) ~[graylog.jar:?]

	at org.graylog2.bootstrap.CmdLineTool.processConfiguration(CmdLineTool.java:505) [graylog.jar:?]

	at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:293) [graylog.jar:?]

	at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:267) [graylog.jar:?]

	at org.graylog2.bootstrap.Main.main(Main.java:55) [graylog.jar:?]

```

The user rights for the persistant folder on my server look like this:

```auto
uadmin@ubuntu:~/Docker/Graylog6/graylog_data/config$ ls -lah
total 48K
drwxrwxrwx 2 graylog graylog 4.0K May 12 09:47 .
drwxrwxrwx 7 graylog graylog 4.0K May 11 23:07 ..
-rwxrwxrwx 1 graylog graylog 34K May 12 00:56 graylog.conf
-rwxrwxrwx 1 graylog graylog 2.5K May 12 00:56 log4j2.xml

```

Graylog is able to read the graylog.conf, but it does not write the node-id file.

If I remove an persistence from the graylog container, I still get the same error. Is there something wrong with `image: graylog/graylog:6.0`?

---

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 12, 2024, 6:29pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/4 "2024-05-12T18:29:40Z")

</div>

for the record: I finally got it up and running. I will sumarize everything and post a link.

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [May 12, 2024, 6:54pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/5 "2024-05-12T18:54:57Z")

</div>

Hey @schneich

Judging from the error

```auto
main] ERROR org.graylog2.bootstrap.CmdLineTool - Couldn't load configuration: Properties file /usr/share/graylog/data/config/graylog.conf doesn't exist!

```

Graylog is looking for the configuration file. By chance did you have this ocnfigured already.

---

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 12, 2024, 7:21pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/6 "2024-05-12T19:21:26Z")

</div>

Hi @gsmith,

thank you for your feedback. Yes, this was one of my errors.

- It started with Graylog not being able to reach Opensearch.
- Then, I realized that my graylog.conf was missing.
- Then, the environment variable for the node-id file did not work.
- In between, I had several issues with folder access permissions.  
…

Here is a brief summary of my findings and a working solution.

> [@How to run Graylog 6, MongoDB and Opensearch in a Docker Stack](https://community.graylog.org/t/how-to-run-graylog-6-mongodb-and-opensearch-in-a-docker-stack/32377):
>
> Dear community, I have spend the last two nights and a great part of today to figure out, how to run Graylog 6 in a Docker environment. The [official Graylog documentation](https://go2docs.graylog.org/current/downloading_and_installing_graylog/docker_installation.htm) was partly helpful, partly misleading. I will summarize my findings here. Hopefully, this will benefit others as well. Background I usually run my docker container in different docker networks to separate them from each other. (-\> increase security) My containers have a public facing network (-\> macvlan), others have only an…

Please allow me a followup question: is there a guide on how to migrate from Elasticsearch to Opensearch? And MongoDB 5 to MongoDB 6?

---

<div class="post-metadata">

**Author:** ![patrickmann](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/patrickmann/32/9091_2.png) [@patrickmann](https://community.graylog.org/u/patrickmann)\
**Post date:** [May 13, 2024, 6:57am UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/7 "2024-05-13T06:57:02Z")

</div>

There is a section in the docs:  
[https://go2docs.graylog.org/5-2/planning\_your\_deployment/upgrading\_to\_opensearch.htm?tocpath=Upgrading%20Graylog|OpenSearch|Moving%20from%20Elasticsearch%20to%20OpenSearch|\_\_\_\_\_0](https://go2docs.graylog.org/5-2/planning_your_deployment/upgrading_to_opensearch.htm?tocpath=Upgrading%20Graylog%7COpenSearch%7CMoving%20from%20Elasticsearch%20to%20OpenSearch%7C _____ 0)

Best to start a new topic if you are having issues with migration.

---

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 13, 2024, 5:24pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/8 "2024-05-13T17:24:12Z")

</div>

Hi @patrickmann ,

Thank you. I have seen that page. It explains about elasticsearch to opensearch, but does not mention mongodb… I might have found other sources for that. If it works, I am happy to share my procedures. 🙂

---

<div class="post-metadata">

**Author:** ![schneich](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/schneich/32/13807_2.png) [@schneich](https://community.graylog.org/u/schneich)\
**Post date:** [May 14, 2024, 9:36pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/9 "2024-05-14T21:36:48Z")

</div>

for the record: this is how I finally upgraded from Graylog 5.2 to 6.0.1:

> [@Update Graylog Docker Stack from 5.2 to 6.0.1, incl. change from Elasticsearch to Opensearch and upgrade mongodb](https://community.graylog.org/t/update-graylog-docker-stack-from-5-2-to-6-0-1-incl-change-from-elasticsearch-to-opensearch-and-upgrade-mongodb/32411):
>
> Dear community, during the last couple of days, I was trying to upgrade my Docker Graylog Stack. I ran into several issues and finally managed to solve them all. I thought it worth while to document, so that others can benefit from my learning curve. slight_smile Background This was my initial post: [Change from Elasticsearch to opensearch: Node ID problem](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370) Then I got it running successfully - as a new, empty environment: [How to run Graylog 6, MongoDB and Opensearch in a Docker Stack](https://community.graylog.org/t/how-to-run-graylog-6-mongodb-and-opensearch-in-a-docker-stack/32377) My Do…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [May 28, 2024, 9:37pm UTC](https://community.graylog.org/t/change-from-elasticsearch-to-opensearch-node-id-problem/32370/10 "2024-05-28T21:37:14Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.