# Datanode not listening on 9200 - no opensearch

**URL:** <https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921>\
**Category:** Graylog Central (peer support)\
**Tags:** dashboards\
**Created:** [February 13, 2025, 3:35pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921 "2025-02-13T15:35:53Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 13, 2025, 3:35pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/1 "2025-02-13T15:35:53Z")

</div>

I’ve followed the [Debian guide](https://go2docs.graylog.org/current/downloading_and_installing_graylog/debian_installation.htm) to install a standalone instance.

However, I see that my machine is not listening on 9200. The datanode service is started correctly and it listens on 8999. I’ve the impression that OpenSearch is not running at all.

Here are the logs

> **datanode.log**
>
> ```auto
> 2025-02-13T16:35:06.620+01:00 INFO [ServerBootstrap] Graylog datanode 6.1.6+a644883 starting up
> 2025-02-13T16:35:06.620+01:00 INFO [ServerBootstrap] JRE: Eclipse Adoptium 17.0.14 on Linux 6.1.0-31-cloud-amd64
> 2025-02-13T16:35:06.621+01:00 INFO [ServerBootstrap] Deployment: deb
> 2025-02-13T16:35:06.621+01:00 INFO [ServerBootstrap] OS: Debian GNU/Linux 12 (bookworm) (debian)
> 2025-02-13T16:35:06.622+01:00 INFO [ServerBootstrap] Arch: amd64
> 2025-02-13T16:35:06.646+01:00 INFO [PeriodicalsService] Starting 6 periodicals ...
> 2025-02-13T16:35:06.647+01:00 INFO [PeriodicalsService] Delaying start of 1 periodicals until this node becomes leader ...
> 2025-02-13T16:35:06.648+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.MetricsCollector] periodical in [0s], polling every [60s].
> 2025-02-13T16:35:06.658+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.OpensearchNodeHeartbeat] periodical in [0s], polling every [10s].
> 2025-02-13T16:35:06.663+01:00 INFO [SearchableSnapshotsConfigurationBean] Opensearch snapshots not configured, skipping search role and cache configuration.
> 2025-02-13T16:35:06.664+01:00 INFO [OpensearchDistributionProvider] Found following opensearch distributions: [/usr/share/graylog-datanode/dist/opensearch-2.15.0-linux-x64]
> 2025-02-13T16:35:06.667+01:00 INFO [Periodicals] Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
> 2025-02-13T16:35:06.670+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeCertRenewalPeriodical] periodical in [0s], polling every [1800s].
> 2025-02-13T16:35:06.676+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeConfigurationPeriodical] periodical in [0s], polling every [2s].
> 2025-02-13T16:35:06.701+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.NodePingPeriodical] periodical in [0s], polling every [1s].
> 2025-02-13T16:35:06.702+01:00 INFO [OpensearchProcessService] 
> 
> ========================================================================================================
> It seems you are starting Data node for the first time. The current configuration is not sufficient to
> start the indexer process because a security configuration is missing. You have to either provide http
> and transport SSL certificates or use the Graylog preflight interface to configure this Data node remotely.
> ========================================================================================================
> 
> 2025-02-13T16:35:06.734+01:00 INFO [JerseyService] Starting Data node REST API
> 2025-02-13T16:35:06.740+01:00 INFO [ServerBootstrap] Services started, startup times in ms: {GracefulShutdownService [RUNNING]=3, OpensearchProcessService [RUNNING]=4, OpensearchConfigurationService [RUNNING]=84, PeriodicalsService [RUNNING]=93}
> 2025-02-13T16:35:06.739+01:00 INFO [CsrRequesterImpl] Triggered certificate signing request for this datanode
> 2025-02-13T16:35:06.740+01:00 INFO [ServerBootstrap] Graylog DataNode datanode up and running.
> 2025-02-13T16:35:07.059+01:00 INFO [Version] HV000001: Hibernate Validator 8.0.1.Final
> 2025-02-13T16:35:07.224+01:00 INFO [NetworkListener] Started listener bound to [0.0.0.0:8999]
> 2025-02-13T16:35:07.225+01:00 INFO [HttpServer] [HttpServer] Started.
> 2025-02-13T16:35:07.226+01:00 INFO [JerseyService] Started REST API at <0.0.0.0:8999>
> 
> ```

How could I troubleshoot this ?

The graylog-server service is properly installed, I’ve completed the preflights checks but then it fails to connect on the datanode port and I get his error in the logs (+WebUI not available)

```log
2025-02-13T16:36:36.995+01:00 ERROR [VersionProbe] Unable to retrieve version from indexer node: Failed to connect to graylog.fqdn/172.28.14.4:9200. - Connection refused.
2025-02-13T16:36:36.995+01:00 INFO [VersionProbe] Indexer is not available. Retry #134

```

---

<div class="post-metadata">

**Author:** ![Sinan](https://avatars.discourse-cdn.com/v4/letter/s/f4b2a3/32.png) [@Sinan](https://community.graylog.org/u/Sinan)\
**Post date:** [February 13, 2025, 6:53pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/2 "2025-02-13T18:53:57Z")

</div>

I had the very same issue. The fix was to remove ‘elasticsearch\_hosts’ line from server.conf. it would be useful for everyone if you could provide details of the conf files (after removing commented & empty lines via grep like `grep -E '^(#|$)' -v server.conf`

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 14, 2025, 7:48am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/3 "2025-02-14T07:48:09Z")

</div>

Thanks for the suggestion.

However this settings was unset, and as per [the docs](https://go2docs.graylog.org/current/setting_up_graylog/initial_configuration_settings.html)

> - Default: `https://127.0.0.1:9200 `You may retain the default setting only if Elasticsearch is installed on the same host as the Graylog server.

Which I believe is my case.  
Anyway, here are the 2 config files

> **server.conf**
>
> ```bash
> is_leader = true
> node_id_file = /etc/graylog/server/node-id
> password_secret = REDACTED
> root_password_sha2 = REDACTED
> bin_dir = /usr/share/graylog-server/bin
> data_dir = /var/lib/graylog-server
> plugin_dir = /usr/share/graylog-server/plugin
> http_bind_address = 0.0.0.0:9000
> selfsigned_startup = true
> stream_aware_field_types=false
> disabled_retention_strategies = none,close
> allow_leading_wildcard_searches = false
> allow_highlighting = false
> field_value_suggestion_mode = on
> output_batch_size = 500
> output_flush_interval = 1
> output_fault_count_threshold = 5
> output_fault_penalty_seconds = 30
> processor_wait_strategy = blocking
> ring_size = 65536
> inputbuffer_ring_size = 65536
> inputbuffer_wait_strategy = blocking
> message_journal_enabled = true
> message_journal_dir = /var/lib/graylog-server/journal
> lb_recognition_period_seconds = 3
> mongodb_uri = mongodb://localhost/graylog
> mongodb_max_connections = 1000
> integrations_scripts_dir = /usr/share/graylog-server/scripts
> 
> ```

> **datanode.conf**
>
> ```bash
> node_id_file = /etc/graylog/datanode/node-id
> config_location = /etc/graylog/datanode
> password_secret = REDACTED
> root_password_sha2 = REDACTED
> mongodb_uri = mongodb://localhost/graylog
> opensearch_location = /usr/share/graylog-datanode/dist
> opensearch_config_location = /var/lib/graylog-datanode/opensearch/config
> opensearch_data_location = /var/lib/graylog-datanode/opensearch/data
> opensearch_logs_location = /var/log/graylog-datanode/opensearch
> 
> ```

---

<div class="post-metadata">

**Author:** ![Tdvorak](https://avatars.discourse-cdn.com/v4/letter/t/a88e57/32.png) [@Tdvorak](https://community.graylog.org/u/Tdvorak)\
**Post date:** [February 14, 2025, 8:02am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/4 "2025-02-14T08:02:33Z")

</div>

Hello,  
I’d recommend removing content of the mongodb. Then commented out the `selfsigned_startup` configuration option and restart both graylog server and data node.

You should also check datanode logs, there will be most likely recorded a reason why the data node can’t start the opensearch process. Feel free to post your data node log here and I’ll try to suggest something else, if this won’t help.

Best regards,  
Tomas

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 14, 2025, 8:30am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/5 "2025-02-14T08:30:41Z")

</div>

Implemented your suggestion.

On the pre-flight interface, the datanode (same system as graylog-server) is listed, but it still does not listen on 9200.

Here are its logs

> **datanode.log**
>
> ```bash
> 2025-02-14T09:28:28.647+01:00 INFO [ServerBootstrap] Graylog datanode 6.1.6+a644883 starting up
> 2025-02-14T09:28:28.647+01:00 INFO [ServerBootstrap] JRE: Eclipse Adoptium 17.0.14 on Linux 6.1.0-31-cloud-amd64
> 2025-02-14T09:28:28.648+01:00 INFO [ServerBootstrap] Deployment: deb
> 2025-02-14T09:28:28.648+01:00 INFO [ServerBootstrap] OS: Debian GNU/Linux 12 (bookworm) (debian)
> 2025-02-14T09:28:28.648+01:00 INFO [ServerBootstrap] Arch: amd64
> 2025-02-14T09:28:28.701+01:00 INFO [PeriodicalsService] Starting 6 periodicals ...
> 2025-02-14T09:28:28.701+01:00 INFO [PeriodicalsService] Delaying start of 1 periodicals until this node becomes leader ...
> 2025-02-14T09:28:28.701+01:00 INFO [Periodicals] Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
> 2025-02-14T09:28:28.784+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.NodePingPeriodical] periodical in [0s], polling every [1s].
> 2025-02-14T09:28:28.794+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.OpensearchNodeHeartbeat] periodical in [0s], polling every [10s].
> 2025-02-14T09:28:28.795+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeConfigurationPeriodical] periodical in [0s], polling every [2s].
> 2025-02-14T09:28:28.801+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeCertRenewalPeriodical] periodical in [0s], polling every [1800s].
> 2025-02-14T09:28:28.802+01:00 INFO [SearchableSnapshotsConfigurationBean] Opensearch snapshots not configured, skipping search role and cache configuration.
> 2025-02-14T09:28:28.803+01:00 INFO [OpensearchDistributionProvider] Found following opensearch distributions: [/usr/share/graylog-datanode/dist/opensearch-2.15.0-linux-x64]
> 2025-02-14T09:28:28.805+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.MetricsCollector] periodical in [0s], polling every [60s].
> 2025-02-14T09:28:28.814+01:00 INFO [OpensearchProcessService] 
> 
> ========================================================================================================
> It seems you are starting Data node for the first time. The current configuration is not sufficient to
> start the indexer process because a security configuration is missing. You have to either provide http
> and transport SSL certificates or use the Graylog preflight interface to configure this Data node remotely.
> ========================================================================================================
> 
> 2025-02-14T09:28:28.815+01:00 INFO [ServerBootstrap] Services started, startup times in ms: {GracefulShutdownService [RUNNING]=1, OpensearchProcessService [RUNNING]=3, PeriodicalsService [RUNNING]=110, OpensearchConfigurationService [RUNNING]=112}
> 2025-02-14T09:28:28.815+01:00 INFO [ServerBootstrap] Graylog DataNode datanode up and running.
> 2025-02-14T09:28:28.816+01:00 INFO [JerseyService] Starting Data node REST API
> 2025-02-14T09:28:29.118+01:00 INFO [Version] HV000001: Hibernate Validator 8.0.1.Final
> 2025-02-14T09:28:29.269+01:00 INFO [NetworkListener] Started listener bound to [0.0.0.0:8999]
> 2025-02-14T09:28:29.270+01:00 INFO [HttpServer] [HttpServer] Started.
> 2025-02-14T09:28:29.270+01:00 INFO [JerseyService] Started REST API at <0.0.0.0:8999>
> 
> ```

---

<div class="post-metadata">

**Author:** ![Tdvorak](https://avatars.discourse-cdn.com/v4/letter/t/a88e57/32.png) [@Tdvorak](https://community.graylog.org/u/Tdvorak)\
**Post date:** [February 14, 2025, 11:39am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/6 "2025-02-14T11:39:27Z")

</div>

Good, that’s fine and expected. If you now look into your graylog-server logs, you’ll see that it’s telling you that the preflight configuration interface is waiting for you, on which address and what are the credentials.

If you open the preflight website, the wizard there will guide you through the process of creating/uploading a certificate authority and provisioning certificates for your data node.

Only when the data node is fully provisioned, it will automatically start the opensearch process and will listen on port 9200.

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 14, 2025, 12:06pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/7 "2025-02-14T12:06:12Z")

</div>

Thanks, indeed after fulfilling the preflight settings, the port 9200 is opened and the opensearch process is running.

Might it be something worth adding to the datanode configuration ?

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 14, 2025, 2:01pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/8 "2025-02-14T14:01:52Z")

</div>

However, it’s not possible to configure a graylog-server with SSL certificates (following [this documentation](https://graylog.org/post/how-to-guide-securing-graylog-with-tls-2/)) **if the pre-flight is not performed**.

When the graylog-server is configure to use TLS, it fails to start if there is no datanode available at `127.0.0.1:9200`, which can only happen when the pre-flight is performed…

Here are the logs:

> **server.log**
>
> ```bash
> 2025-02-14T14:55:26.325+01:00 INFO [ImmutableFeatureFlagsCollector] Following feature flags are used: {default properties file=[cloud_inputs=on, investigation_report_by_ai=on, data_tiering_cloud=off, composable_index_templates=off, preflight_web=on, data_node_migration=on, remote_reindex_migration=off, instant_archiving=off, configurable_value_units=on, new_report_creation=on, threat_coverage=on]}
> 2025-02-14T14:55:26.974+01:00 INFO [CmdLineTool] Loaded plugin: AWS plugins 6.1.6+a644883 [org.graylog.aws.AWSPlugin]
> 2025-02-14T14:55:26.974+01:00 INFO [CmdLineTool] Loaded plugin: Integrations 6.1.6+a644883 [org.graylog.integrations.IntegrationsPlugin]
> 2025-02-14T14:55:26.974+01:00 INFO [CmdLineTool] Loaded plugin: Threat Intelligence Plugin 6.1.6+a644883 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
> 2025-02-14T14:55:26.974+01:00 INFO [CmdLineTool] Loaded plugin: Elasticsearch 7 Support 6.1.6+a644883 [org.graylog.storage.elasticsearch7.Elasticsearch7Plugin]
> 2025-02-14T14:55:26.975+01:00 INFO [CmdLineTool] Loaded plugin: OpenSearch 2 Support 6.1.6+a644883 [org.graylog.storage.opensearch2.OpenSearch2Plugin]
> 2025-02-14T14:55:26.990+01:00 INFO [CmdLineTool] Running with JVM arguments: -Xms1g -Xmx1g -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -Djdk.tls.acknowledgeCloseNotify=true -Dlog4j2.formatMsgNoLookups=true -Djavax.net.ssl.trustStore=/etc/graylog/graylog.jks -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Dgraylog2.installation_source=deb
> 2025-02-14T14:55:27.149+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-31-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@611a990b, com.mongodb.Jep395RecordCodecProvider@7cbeac65, com.mongodb.KotlinCodecProvider@7fe82967]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-14T14:55:27.167+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-31-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@611a990b, com.mongodb.Jep395RecordCodecProvider@7cbeac65, com.mongodb.KotlinCodecProvider@7fe82967]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-14T14:55:27.209+01:00 INFO [cluster] Waiting for server to become available for operation with ID 3. Remaining time: 29995 ms. Selector: ReadPreferenceServerSelector{readPreference=primary}, topology description: {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING}].
> 2025-02-14T14:55:27.210+01:00 INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, cryptd=false, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=30762701, minRoundTripTimeNanos=0}
> 2025-02-14T14:55:27.282+01:00 INFO [MongoDBPreflightCheck] Connected to MongoDB version 7.0.16
> 2025-02-14T14:55:27.733+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-31-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@611a990b, com.mongodb.Jep395RecordCodecProvider@7cbeac65, com.mongodb.KotlinCodecProvider@7fe82967]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-14T14:55:27.745+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-31-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@611a990b, com.mongodb.Jep395RecordCodecProvider@7cbeac65, com.mongodb.KotlinCodecProvider@7fe82967]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-14T14:55:27.746+01:00 INFO [cluster] Waiting for server to become available for operation with ID 11. Remaining time: 29999 ms. Selector: ReadPreferenceServerSelector{readPreference=primary}, topology description: {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING}].
> 2025-02-14T14:55:27.757+01:00 INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, cryptd=false, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=21, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=5099684, minRoundTripTimeNanos=0}
> 2025-02-14T14:55:28.235+01:00 INFO [ServerBootstrap] Running 2 migrations...
> 2025-02-14T14:55:28.454+01:00 ERROR [VersionProbe] Unable to retrieve version from indexer node: Failed to connect to /127.0.0.1:9200. - Connection refused.
> 2025-02-14T14:55:28.477+01:00 INFO [VersionProbe] Indexer is not available. Retry #1
> 2025-02-14T14:55:33.478+01:00 ERROR [VersionProbe] Unable to retrieve version from indexer node: Failed to connect to /127.0.0.1:9200. - Connection refused.
> 2025-02-14T14:55:33.479+01:00 INFO [VersionProbe] Indexer is not available. Retry #2
> 2025-02-14T14:55:38.480+01:00 ERROR [VersionProbe] Unable to retrieve version from indexer node: Failed to connect to /127.0.0.1:9200. - Connection refused.
> 
> ```

In my opinion this is a deadlock situation:

1. Configure SSL certificate for accessing Graylog pre-flight config
2. graylog-server fails to start because it expected a datanode on 127.0.0.1:9200
3. the datanode’s opensearch on 9200 is not started because the pre-flight has not been done…

Am I missing something ?

---

<div class="post-metadata">

**Author:** ![Sinan](https://avatars.discourse-cdn.com/v4/letter/s/f4b2a3/32.png) [@Sinan](https://community.graylog.org/u/Sinan)\
**Post date:** [February 14, 2025, 6:30pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/9 "2025-02-14T18:30:04Z")

</div>

just in case, in datanode.conf I would explicitly set: `opensearch_http_port = 9200`

---

<div class="post-metadata">

**Author:** ![Sinan](https://avatars.discourse-cdn.com/v4/letter/s/f4b2a3/32.png) [@Sinan](https://community.graylog.org/u/Sinan)\
**Post date:** [February 14, 2025, 6:32pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/10 "2025-02-14T18:32:03Z")

</div>

NVM you have server + datanode installed at the same machine it seems.  
~~Also. in `/etc/mongodb.conf` , you should set: `bindIp: 0.0.0.0`~~

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 17, 2025, 8:27am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/11 "2025-02-17T08:27:22Z")

</div>

> [@Sinan](#):
>
> just in case, in datanode.conf I would explicitly set: `opensearch_http_port = 9200`

I’ve set this, but this does not change anything, as OpenSearch is not starting.

I’ve tried specifying these options, hoping it would make opensearch start:

- `transport_certificate`
- `transport_certificate_password`
- `http_certificate`
- `http_certificate_password`

But it didn’t.  
That .p12 file is correctly parsed, but the datanode complains that opensearch is not started:

```auto
2025-02-17T09:49:01.304+01:00 ERROR [ServerBootstrap] Graylog DataNode startup failed. Exiting. Exception was:
java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[OpensearchConfigurationService [FAILED]]}

```

> **datanode.log**
>
> ```auto
> 2025-02-17T09:48:59.658+01:00 INFO [OpensearchDataDirCompatibilityCheck] Found 0 indices and all of them are valid with current opensearch version 2.15.0
> 2025-02-17T09:48:59.875+01:00 INFO [DatanodeDirectories] Opensearch of the node 74f09692-f70d-4c11-997d-669854d690c8 uses following directories as its storage: DatanodeDirectories{dataTargetDir='/var/lib/graylog-datanode/opensearch/data', logsTargetDir='/var/log/graylog-datanode/opensearch', configurationSourceDir='Optional[/etc/graylog/datanode]', configurationTargetDir='/var/lib/graylog-datanode/opensearch/config', opensearchProcessConfigurationDir='/var/lib/graylog-datanode/opensearch/config/opensearch'}
> 2025-02-17T09:49:00.927+01:00 INFO [DbEntitiesScanner] 16 entities have been scanned and added to DB Entity Catalog, it took 914.9 ms
> 2025-02-17T09:49:00.957+01:00 INFO [ServerBootstrap] Graylog datanode 6.1.6+a644883 starting up
> 2025-02-17T09:49:00.958+01:00 INFO [ServerBootstrap] JRE: Eclipse Adoptium 17.0.14 on Linux 6.1.0-31-cloud-amd64
> 2025-02-17T09:49:00.958+01:00 INFO [ServerBootstrap] Deployment: deb
> 2025-02-17T09:49:00.958+01:00 INFO [ServerBootstrap] OS: Debian GNU/Linux 12 (bookworm) (debian)
> 2025-02-17T09:49:00.958+01:00 INFO [ServerBootstrap] Arch: amd64
> 2025-02-17T09:49:00.979+01:00 INFO [PeriodicalsService] Starting 6 periodicals ...
> 2025-02-17T09:49:00.979+01:00 INFO [PeriodicalsService] Delaying start of 1 periodicals until this node becomes leader ...
> 2025-02-17T09:49:00.980+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.OpensearchNodeHeartbeat] periodical in [0s], polling every [10s].
> 2025-02-17T09:49:00.985+01:00 INFO [Periodicals] Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
> 2025-02-17T09:49:00.990+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeConfigurationPeriodical] periodical in [0s], polling every [2s].
> 2025-02-17T09:49:01.000+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeCertRenewalPeriodical] periodical in [0s], polling every [1800s].
> 2025-02-17T09:49:01.006+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.MetricsCollector] periodical in [0s], polling every [60s].
> 2025-02-17T09:49:01.077+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.NodePingPeriodical] periodical in [0s], polling every [1s].
> 2025-02-17T09:49:01.250+01:00 INFO [OpensearchSecurityConfiguration] Opensearch transport certificate has following alternative names: graylog.<FQDN>, 172.28.14.4
> 2025-02-17T09:49:01.250+01:00 INFO [OpensearchSecurityConfiguration] Opensearch transport certificate has following serial number: 622915052742269610821727451503118679977033641676
> 2025-02-17T09:49:01.251+01:00 INFO [OpensearchSecurityConfiguration] Opensearch transport certificate has following validity: 2025-02-13T14:55:17.000+0100 - 2035-02-12T14:55:17.000+0100
> 2025-02-17T09:49:01.267+01:00 INFO [OpensearchSecurityConfiguration] Opensearch HTTP certificate has following alternative names: graylog.<FQDN>, 172.28.14.4
> 2025-02-17T09:49:01.268+01:00 INFO [OpensearchSecurityConfiguration] Opensearch HTTP certificate has following serial number: 622915052742269610821727451503118679977033641676
> 2025-02-17T09:49:01.268+01:00 INFO [OpensearchSecurityConfiguration] Opensearch HTTP certificate has following validity: 2025-02-13T14:55:17.000+0100 - 2035-02-12T14:55:17.000+0100
> 2025-02-17T09:49:01.270+01:00 INFO [TruststoreUtils] Detected existing JVM truststore: /usr/share/graylog-datanode/jvm/lib/security/cacerts of type pkcs12
> 2025-02-17T09:49:01.302+01:00 INFO [Periodicals] Shutting down periodical [org.graylog2.events.ClusterEventPeriodical].
> 2025-02-17T09:49:01.302+01:00 INFO [Periodicals] Shutting down periodical [org.graylog.datanode.bootstrap.preflight.DataNodeConfigurationPeriodical].
> 2025-02-17T09:49:01.303+01:00 INFO [Periodicals] Shutting down periodical [org.graylog.datanode.bootstrap.preflight.DataNodeCertRenewalPeriodical].
> 2025-02-17T09:49:01.304+01:00 ERROR [ServerBootstrap] Graylog DataNode startup failed. Exiting. Exception was:
> java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[OpensearchConfigurationService [FAILED]]}
> at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:772) ~[guava-33.3.1-jre.jar:?]
> at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:584) ~[guava-33.3.1-jre.jar:?]
> at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:298) ~[guava-33.3.1-jre.jar:?]
> at org.graylog.datanode.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:228) [graylog-datanode.jar:?]
> at org.graylog.datanode.bootstrap.CmdLineTool.doRun(CmdLineTool.java:305) [graylog-datanode.jar:?]
> at org.graylog.datanode.bootstrap.CmdLineTool.run(CmdLineTool.java:246) [graylog-datanode.jar:?]
> at org.graylog.datanode.bootstrap.Main.main(Main.java:59) [graylog-datanode.jar:?]
> Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: OpensearchConfigurationService [FAILED]
> Caused by: java.lang.NullPointerException: Cannot read the array length because "<parameter1>" is null
> at java.base/java.util.Arrays.stream(Unknown Source) ~[?:?]
> at org.graylog.datanode.configuration.TruststoreCreator.findRootCert(TruststoreCreator.java:101) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.TruststoreCreator.addRootCert(TruststoreCreator.java:65) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.variants.OpensearchSecurityConfiguration.configure(OpensearchSecurityConfiguration.java:92) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.OpensearchConfigurationService.get(OpensearchConfigurationService.java:159) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.OpensearchConfigurationService.triggerConfigurationChangedEvent(OpensearchConfigurationService.java:218) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.OpensearchConfigurationService.startUp(OpensearchConfigurationService.java:95) ~[graylog-datanode.jar:?]
> at com.google.common.util.concurrent.AbstractIdleService$DelegateService.lambda$doStart$0(AbstractIdleService.java:64) ~[guava-33.3.1-jre.jar:?]
> at com.google.common.util.concurrent.Callables.lambda$threadRenaming$3(Callables.java:105) ~[guava-33.3.1-jre.jar:?]
> at java.base/java.lang.Thread.run(Unknown Source) ~[?:?]
> 2025-02-17T09:49:01.312+01:00 INFO [Server] SIGNAL received. Shutting down.
> 2025-02-17T09:49:01.314+01:00 INFO [GracefulShutdown] Graceful shutdown initiated.
> 2025-02-17T09:49:01.314+01:00 INFO [GracefulShutdown] Goodbye.
> 
> ```

---

<div class="post-metadata">

**Author:** ![janheise](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@janheise](https://community.graylog.org/u/janheise)\
**Post date:** [February 18, 2025, 3:31pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/12 "2025-02-18T15:31:49Z")

</div>

The temp credentials in the preflight were specifically added because of the understanding that you setup GL and the DataNode before you have secured GL itself via SSL. So that no prod passwords ever travel the network without TLS.  
Can you try setting up GL/DataNode first and then add the SSL config later?

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 25, 2025, 7:35am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/13 "2025-02-25T07:35:27Z")

</div>

Well, I’m trying to set the SSL certificate at the start, because I want to avoid the pre-flight configuration.

I have seen docs for securing the WebUI access with certificate, but nothing is mentionned for the datanode SSL settings.

---

<div class="post-metadata">

**Author:** ![Tdvorak](https://avatars.discourse-cdn.com/v4/letter/t/a88e57/32.png) [@Tdvorak](https://community.graylog.org/u/Tdvorak)\
**Post date:** [February 25, 2025, 9:08am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/14 "2025-02-25T09:08:20Z")

</div>

> [@3isenHeiM](#):
>
> ```auto
> Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: OpensearchConfigurationService [FAILED]
> Caused by: java.lang.NullPointerException: Cannot read the array length because "<parameter1>" is null
> at java.base/java.util.Arrays.stream(Unknown Source) ~[?:?]
> at org.graylog.datanode.configuration.TruststoreCreator.findRootCert(TruststoreCreator.java:101) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.TruststoreCreator.addRootCert(TruststoreCreator.java:65) ~[
> 
> ```

This error suggests that there is something wrong with the keystore used in the datanode. First of all I’d suggest updating your datanode package. Recently there were some changes implemented, which could help.

Could you please tell me what structure, keys and certificates does your .p12 keystore contain?

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 25, 2025, 9:44am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/15 "2025-02-25T09:44:58Z")

</div>

I’m setting this up with Ansible → packages are up to date.

Here is the keystore contents:

> **graylog.p12**
>
> ```bash
> keytool -list -v -keystore /etc/graylog/certs/graylog.p12 -storetype PKCS12
> 
> Enter keystore password:  
> Keystore type: PKCS12
> Keystore provider: SUN
> 
> Your keystore contains 1 entry
> 
> Alias name: graylog
> Creation date: Feb 25, 2025
> Entry type: PrivateKeyEntry
> Certificate chain length: 1
> Certificate[1]:
> Owner: CN=<GRAYLOG.FQDN>, O=Us, L=somewhere, C=US
> Issuer: CN=<GRAYLOG.FQDN>, O=Us, L=somewhere, C=US
> Serial number: <REDACTED>
> Valid from: Sun Feb 23 16:57:55 CET 2025 until: Thu Feb 22 16:57:55 CET 2035
> Certificate fingerprints:
> SHA1: <REDACTED> SHA256: <REDACTED>
> Signature algorithm name: SHA256withRSA
> Subject Public Key Algorithm: 4096-bit RSA key
> Version: 3
> 
> Extensions: 
> 
> #1: ObjectId: 2.5.29.19 Criticality=true
> BasicConstraints:[
> CA:true
> PathLen: no limit
> ]
> 
> #2: ObjectId: 2.5.29.15 Criticality=true
> KeyUsage [
> Key_CertSign
> ]
> 
> #3: ObjectId: 2.5.29.17 Criticality=false
> SubjectAlternativeName [
> DNSName: graylog.backend
> IPAddress: 172.28.14.4
> ]
> 
> #4: ObjectId: 2.5.29.14 Criticality=false
> SubjectKeyIdentifier [
> KeyIdentifier [
> 0000: <REDACTED>
> 0010: <REDACTED>
> ]
> ]
> 
> *******************************************
> *******************************************
> 
> ```

---

<div class="post-metadata">

**Author:** ![Tdvorak](https://avatars.discourse-cdn.com/v4/letter/t/a88e57/32.png) [@Tdvorak](https://community.graylog.org/u/Tdvorak)\
**Post date:** [February 25, 2025, 10:06am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/16 "2025-02-25T10:06:53Z")

</div>

Thank you!

I asked specifically about the update, because the method from the stacktrace:

```auto
TruststoreCreator.findRootCert

```

was removed in the latest [6.1.7 release (Released: 2025-02-17)](https://go2docs.graylog.org/current/changelogs/changelog.html#Graylog617) and replaced with better certificate extraction from keystores. Updating the datanode to 6.1.7 could fix your problem.

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 25, 2025, 12:11pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/17 "2025-02-25T12:11:06Z")

</div>

> [@Tdvorak](#):
>
> d in the latest [6.1.7 release (Released: 2025-02-17)](https://go2docs.graylog.org/current/changelogs/changelog.html#Graylog617) and replaced with

It’s already in that latest version 😉

> **package list**
>
> ```bash
> dpkg -l | grep -E ".(elasticsearch|graylog|mongo)"
> ii graylog-6.1-repository 1-1 all Package to install Graylog 6.1 GPG key and repository
> ii graylog-datanode 6.1.7-1 amd64 Graylog data node
> ii graylog-server 6.1.7-1 amd64 Graylog server
> ii mongodb-database-tools 100.11.0 amd64 mongodb-database-tools package provides tools for working with the MongoDB server: 
> ii mongodb-mongosh 2.4.0 amd64 MongoDB Shell CLI REPL Package
> hi mongodb-org 8.0.5 amd64 MongoDB open source document-oriented database system (metapackage)
> ii mongodb-org-database 8.0.5 amd64 MongoDB open source document-oriented database system (metapackage)
> ii mongodb-org-database-tools-extra 8.0.5 amd64 Extra MongoDB database tools
> ii mongodb-org-mongos 8.0.5 amd64 MongoDB sharded cluster query router
> ii mongodb-org-server 8.0.5 amd64 MongoDB database server
> ii mongodb-org-shell 8.0.5 amd64 MongoDB shell client
> ii mongodb-org-tools 8.0.5 amd64 MongoDB tools
> 
> ```

I think that the content of the truststore is badly “formatted” or something.

What are the requirement of the certificates to create ?

---

<div class="post-metadata">

**Author:** ![janheise](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@janheise](https://community.graylog.org/u/janheise)\
**Post date:** [February 25, 2025, 12:37pm UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/18 "2025-02-25T12:37:13Z")

</div>

Hi @3isenHeiM , you write that you already have the latest version (6.1.7) deployed. Can you please confirm that you still get the same Exception that @Tdvorak mentioned 3 posts above this one? Because your first mention of this exception is from Monday, 17.2. 9:27 in the morning while we released 6.1.7 the same day in the evening?

---

<div class="post-metadata">

**Author:** ![3isenHeiM](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/3isenheim/32/17630_2.png) [@3isenHeiM](https://community.graylog.org/u/3isenHeiM)\
**Post date:** [February 26, 2025, 8:45am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/19 "2025-02-26T08:45:10Z")

</div>

Well, _per se_ I don’t find the error `findRootCert` anymore in the log files. But the datanode process exits with the error:

```bash
java.lang.NullPointerException: Cannot read the array length because "<parameter1>" is null

```

Here is the full log:

> **datanode.log**
>
> ```bash
> 2025-02-26T09:39:02.040+01:00 INFO [ImmutableFeatureFlagsCollector] Following feature flags are used: {}
> 2025-02-26T09:39:02.278+01:00 INFO [CmdLineTool] Running with JVM arguments: -Dlog4j.configurationFile=file:///etc/graylog/datanode/log4j2.xml -Xms4g -Xmx4g -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+UnlockExperimentalVMOptions -Djdk.tls.acknowledgeCloseNotify=true
> 2025-02-26T09:39:02.406+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-30-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@7569ea63, com.mongodb.Jep395RecordCodecProvider@ebd06a9, com.mongodb.KotlinCodecProvider@16c587de]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-26T09:39:02.413+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-30-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@7569ea63, com.mongodb.Jep395RecordCodecProvider@ebd06a9, com.mongodb.KotlinCodecProvider@16c587de]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-26T09:39:02.454+01:00 INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, cryptd=false, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=25, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=17937429, minRoundTripTimeNanos=0}
> 2025-02-26T09:39:02.457+01:00 INFO [cluster] Waiting for server to become available for operation with ID 3. Remaining time: 29990 ms. Selector: ReadPreferenceServerSelector{readPreference=primary}, topology description: {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING}].
> 2025-02-26T09:39:02.504+01:00 INFO [MongoDBPreflightCheck] Connected to MongoDB version 8.0.5
> 2025-02-26T09:39:02.642+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-30-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@7569ea63, com.mongodb.Jep395RecordCodecProvider@ebd06a9, com.mongodb.KotlinCodecProvider@16c587de]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-26T09:39:02.644+01:00 INFO [client] MongoClient with metadata {"driver": {"name": "mongo-java-driver", "version": "5.2.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.0-30-cloud-amd64"}, "platform": "Java/Eclipse Adoptium/17.0.14+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, transportSettings=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@7569ea63, com.mongodb.Jep395RecordCodecProvider@ebd06a9, com.mongodb.KotlinCodecProvider@16c587de]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverMonitoringMode=AUTO, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null, timeoutMS=null}
> 2025-02-26T09:39:02.645+01:00 INFO [cluster] Waiting for server to become available for operation with ID 11. Remaining time: 29999 ms. Selector: ReadPreferenceServerSelector{readPreference=primary}, topology description: {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING}].
> 2025-02-26T09:39:02.647+01:00 INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, cryptd=false, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=25, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=3344403, minRoundTripTimeNanos=0}
> 2025-02-26T09:39:02.677+01:00 INFO [DatanodeDirectories] Opensearch of the node f34b798a-0a69-40ba-ba45-2b9980e564cf uses following directories as its storage: DatanodeDirectories{dataTargetDir='/var/lib/graylog-datanode/opensearch/data', logsTargetDir='/var/log/graylog-datanode/opensearch', configurationSourceDir='Optional[/etc/graylog/datanode]', configurationTargetDir='/var/lib/graylog-datanode/opensearch/config', opensearchProcessConfigurationDir='/var/lib/graylog-datanode/opensearch/config/opensearch'}
> 2025-02-26T09:39:02.703+01:00 INFO [OpensearchConfigSync] Directory used for Opensearch process configuration is /var/lib/graylog-datanode/opensearch/config/opensearch
> 2025-02-26T09:39:02.716+01:00 INFO [OpensearchConfigSync] Synchronizing Opensearch configuration
> 2025-02-26T09:39:02.719+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/log4j2.properties
> 2025-02-26T09:39:02.720+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/jvm.options
> 2025-02-26T09:39:02.720+01:00 INFO [FullDirSync] Deleting obsolete directory /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security
> 2025-02-26T09:39:02.731+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/roles_mapping.yml
> 2025-02-26T09:39:02.732+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/audit.yml
> 2025-02-26T09:39:02.732+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/nodes_dn.yml
> 2025-02-26T09:39:02.733+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/config.yml
> 2025-02-26T09:39:02.734+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/allowlist.yml
> 2025-02-26T09:39:02.734+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/action_groups.yml
> 2025-02-26T09:39:02.734+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/whitelist.yml
> 2025-02-26T09:39:02.734+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/opensearch.yml.example
> 2025-02-26T09:39:02.735+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/roles.yml
> 2025-02-26T09:39:02.735+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/internal_users.yml
> 2025-02-26T09:39:02.735+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/tenants.yml
> 2025-02-26T09:39:02.735+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/http-keystore.p12
> 2025-02-26T09:39:02.736+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/datanode-truststore.p12
> 2025-02-26T09:39:02.736+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/transport-keystore.p12
> 2025-02-26T09:39:02.737+01:00 INFO [FullDirSync] Deleting obsolete directory /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-observability
> 2025-02-26T09:39:02.737+01:00 INFO [FullDirSync] Deleting obsolete file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-observability/observability.yml
> 2025-02-26T09:39:02.739+01:00 INFO [FullDirSync] Synchronizing directory /var/lib/graylog-datanode/opensearch/config/opensearch
> 2025-02-26T09:39:02.741+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/log4j2.properties
> 2025-02-26T09:39:02.742+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/jvm.options
> 2025-02-26T09:39:02.742+01:00 INFO [FullDirSync] Synchronizing directory /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-observability
> 2025-02-26T09:39:02.743+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-observability/observability.yml
> 2025-02-26T09:39:02.744+01:00 INFO [FullDirSync] Synchronizing directory /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security
> 2025-02-26T09:39:02.744+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/whitelist.yml
> 2025-02-26T09:39:02.745+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/opensearch.yml.example
> 2025-02-26T09:39:02.745+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/allowlist.yml
> 2025-02-26T09:39:02.745+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/roles_mapping.yml
> 2025-02-26T09:39:02.746+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/internal_users.yml
> 2025-02-26T09:39:02.746+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/action_groups.yml
> 2025-02-26T09:39:02.746+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/audit.yml
> 2025-02-26T09:39:02.747+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/config.yml
> 2025-02-26T09:39:02.747+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/tenants.yml
> 2025-02-26T09:39:02.748+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/nodes_dn.yml
> 2025-02-26T09:39:02.748+01:00 INFO [FullDirSync] Synchronizing file /var/lib/graylog-datanode/opensearch/config/opensearch/opensearch-security/roles.yml
> 2025-02-26T09:39:02.750+01:00 INFO [OpensearchDistributionProvider] Found following opensearch distributions: [/usr/share/graylog-datanode/dist/opensearch-2.15.0-linux-x64]
> 2025-02-26T09:39:02.763+01:00 INFO [OpensearchDataDirCompatibilityCheck] Found 0 indices and all of them are valid with current opensearch version 2.15.0
> 2025-02-26T09:39:02.939+01:00 INFO [DatanodeDirectories] Opensearch of the node f34b798a-0a69-40ba-ba45-2b9980e564cf uses following directories as its storage: DatanodeDirectories{dataTargetDir='/var/lib/graylog-datanode/opensearch/data', logsTargetDir='/var/log/graylog-datanode/opensearch', configurationSourceDir='Optional[/etc/graylog/datanode]', configurationTargetDir='/var/lib/graylog-datanode/opensearch/config', opensearchProcessConfigurationDir='/var/lib/graylog-datanode/opensearch/config/opensearch'}
> 2025-02-26T09:39:03.877+01:00 INFO [DbEntitiesScanner] 16 entities have been scanned and added to DB Entity Catalog, it took 810.7 ms
> 2025-02-26T09:39:03.920+01:00 INFO [ServerBootstrap] Graylog datanode 6.1.7+6ec0bac starting up
> 2025-02-26T09:39:03.924+01:00 INFO [ServerBootstrap] JRE: Eclipse Adoptium 17.0.14 on Linux 6.1.0-30-cloud-amd64
> 2025-02-26T09:39:03.925+01:00 INFO [ServerBootstrap] Deployment: deb
> 2025-02-26T09:39:03.925+01:00 INFO [ServerBootstrap] OS: Debian GNU/Linux 12 (bookworm) (debian)
> 2025-02-26T09:39:03.925+01:00 INFO [ServerBootstrap] Arch: amd64
> 2025-02-26T09:39:03.949+01:00 INFO [PeriodicalsService] Starting 6 periodicals ...
> 2025-02-26T09:39:03.949+01:00 INFO [PeriodicalsService] Delaying start of 1 periodicals until this node becomes leader ...
> 2025-02-26T09:39:04.040+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.NodePingPeriodical] periodical in [0s], polling every [1s].
> 2025-02-26T09:39:04.048+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeConfigurationPeriodical] periodical in [0s], polling every [2s].
> 2025-02-26T09:39:04.064+01:00 INFO [Periodicals] Starting [org.graylog.datanode.bootstrap.preflight.DataNodeCertRenewalPeriodical] periodical in [0s], polling every [1800s].
> 2025-02-26T09:39:04.068+01:00 INFO [Periodicals] Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
> 2025-02-26T09:39:04.089+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.OpensearchNodeHeartbeat] periodical in [0s], polling every [10s].
> 2025-02-26T09:39:04.098+01:00 INFO [Periodicals] Starting [org.graylog.datanode.periodicals.MetricsCollector] periodical in [0s], polling every [60s].
> 2025-02-26T09:39:04.278+01:00 INFO [OpensearchSecurityConfiguration] Opensearch transport certificate has following alternative names: graylog.<redacted>, 172.28.14.4
> 2025-02-26T09:39:04.279+01:00 INFO [OpensearchSecurityConfiguration] Opensearch transport certificate has following serial number: 429255647533457603859532427351443426371580009640
> 2025-02-26T09:39:04.280+01:00 INFO [OpensearchSecurityConfiguration] Opensearch transport certificate has following validity: 2025-02-25T09:21:04.000+0100 - 2035-02-24T09:21:04.000+0100
> 2025-02-26T09:39:04.299+01:00 INFO [OpensearchSecurityConfiguration] Opensearch HTTP certificate has following alternative names: graylog.<redacted>, 172.28.14.4
> 2025-02-26T09:39:04.300+01:00 INFO [OpensearchSecurityConfiguration] Opensearch HTTP certificate has following serial number: 429255647533457603859532427351443426371580009640
> 2025-02-26T09:39:04.300+01:00 INFO [OpensearchSecurityConfiguration] Opensearch HTTP certificate has following validity: 2025-02-25T09:21:04.000+0100 - 2035-02-24T09:21:04.000+0100
> 2025-02-26T09:39:04.303+01:00 INFO [TruststoreUtils] Detected existing JVM truststore: /usr/share/graylog-datanode/jvm/lib/security/cacerts of type pkcs12
> 2025-02-26T09:39:04.324+01:00 INFO [Periodicals] Shutting down periodical [org.graylog.datanode.bootstrap.preflight.DataNodeConfigurationPeriodical].
> 2025-02-26T09:39:04.325+01:00 INFO [Periodicals] Shutting down periodical [org.graylog.datanode.bootstrap.preflight.DataNodeCertRenewalPeriodical].
> 2025-02-26T09:39:04.325+01:00 INFO [Periodicals] Shutting down periodical [org.graylog2.events.ClusterEventPeriodical].
> 2025-02-26T09:39:04.326+01:00 ERROR [ServerBootstrap] Graylog DataNode startup failed. Exiting. Exception was:
> java.lang.IllegalStateException: Expected to be healthy after starting. The following services are not running: {FAILED=[OpensearchConfigurationService [FAILED]]}
> at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.checkHealthy(ServiceManager.java:772) ~[guava-33.3.1-jre.jar:?]
> at com.google.common.util.concurrent.ServiceManager$ServiceManagerState.awaitHealthy(ServiceManager.java:584) ~[guava-33.3.1-jre.jar:?]
> at com.google.common.util.concurrent.ServiceManager.awaitHealthy(ServiceManager.java:298) ~[guava-33.3.1-jre.jar:?]
> at org.graylog.datanode.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:228) [graylog-datanode.jar:?]
> at org.graylog.datanode.bootstrap.CmdLineTool.doRun(CmdLineTool.java:305) [graylog-datanode.jar:?]
> at org.graylog.datanode.bootstrap.CmdLineTool.run(CmdLineTool.java:246) [graylog-datanode.jar:?]
> at org.graylog.datanode.bootstrap.Main.main(Main.java:59) [graylog-datanode.jar:?]
> Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: OpensearchConfigurationService [FAILED]
> Caused by: java.lang.NullPointerException: Cannot read the array length because "<parameter1>" is null
> at java.base/java.util.Arrays.stream(Unknown Source) ~[?:?]
> at org.graylog2.security.TruststoreCreator.toX509Certs(TruststoreCreator.java:80) ~[graylog2-server-6.1.7.jar:?]
> at org.graylog2.security.TruststoreCreator.addFromKeystore(TruststoreCreator.java:74) ~[graylog2-server-6.1.7.jar:?]
> at org.graylog.datanode.configuration.variants.OpensearchSecurityConfiguration.configure(OpensearchSecurityConfiguration.java:92) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.OpensearchConfigurationService.get(OpensearchConfigurationService.java:159) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.OpensearchConfigurationService.triggerConfigurationChangedEvent(OpensearchConfigurationService.java:218) ~[graylog-datanode.jar:?]
> at org.graylog.datanode.configuration.OpensearchConfigurationService.startUp(OpensearchConfigurationService.java:95) ~[graylog-datanode.jar:?]
> at com.google.common.util.concurrent.AbstractIdleService$DelegateService.lambda$doStart$0(AbstractIdleService.java:64) ~[guava-33.3.1-jre.jar:?]
> at com.google.common.util.concurrent.Callables.lambda$threadRenaming$3(Callables.java:105) ~[guava-33.3.1-jre.jar:?]
> at java.base/java.lang.Thread.run(Unknown Source) ~[?:?]
> 2025-02-26T09:39:04.333+01:00 INFO [Server] SIGNAL received. Shutting down.
> 2025-02-26T09:39:04.334+01:00 INFO [GracefulShutdown] Graceful shutdown initiated.
> 2025-02-26T09:39:04.334+01:00 INFO [GracefulShutdown] Goodbye.
> 
> ```

And here is the datanode config:

```auto
node_id_file = /etc/graylog/datanode/node-id
config_location = /etc/graylog/datanode
password_secret = REDACTED
root_password_sha2 = REDACTED
mongodb_uri = mongodb://localhost/graylog
bind_address = 127.0.0.1
opensearch_location = /usr/share/graylog-datanode/dist
opensearch_config_location = /var/lib/graylog-datanode/opensearch/config
opensearch_data_location = /var/lib/graylog-datanode/opensearch/data
opensearch_logs_location = /var/log/graylog-datanode/opensearch
transport_certificate = /etc/graylog/certs/graylog.p12
transport_certificate_password = changeit
http_certificate = /etc/graylog/certs/graylog.p12
http_certificate_password = changeit

```

The contents of the keystore are the same than posted [here](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/15).

---

<div class="post-metadata">

**Author:** ![janheise](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@janheise](https://community.graylog.org/u/janheise)\
**Post date:** [February 26, 2025, 8:53am UTC](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921/20 "2025-02-26T08:53:06Z")

</div>

can you try with a new/recreated keystore where you use “datanode” instead of “graylog” as the alias for the key?

[Next page](https://community.graylog.org/t/datanode-not-listening-on-9200-no-opensearch/34921.md?page=2)