Clean Install v5.2.3 Fails with Runtime Exception: retentionStrategyConfig

abrenner · February 7, 2024, 7:19am

1. Describe your incident:

During a fresh / clean install of Graylog version 5.2.3 on ArchLinux with MongoDB v6.0.13 and OpenSearch v2.11.1 the installation fails with a Java runtime exception when trying to create the open search indices (aka migration steps)

2. Describe your environment:

OS Information: ArchLinux
Package Version:
- OpenJDK/JRE: 17.0.10
- MongoDB: 6.0.13
- OpenSearch: 2.11.1
  - minimum_wire_compatibility_version: 7.10.0
  - minimum_wire_compatibility_version: 7.0.0
  - lucene_version: 9.7.0
Service logs, configurations, and environment variables:

Feb 06 22:58:01 syslog systemd[1]: Started Graylog management server.
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,144 INFO : org.graylog2.featureflag.ImmutableFeatureFlagsCollector - Following feature flags are used: {default properties file=[frontend_hotkeys=on, field_types_management=on, cloud_inputs=on, scripting_api_preview=on, composable_index_templates=off, search_filter=on, preflight_web=on, instant_archiving=off]}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,514 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: AWS plugins 5.2.3+9aee303 [org.graylog.aws.AWSPlugin]
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,515 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Integrations 5.2.3+9aee303 [org.graylog.integrations.IntegrationsPlugin]
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,515 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Threat Intelligence Plugin 5.2.3+9aee303 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,516 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Elasticsearch 7 Support 5.2.3+9aee303 [org.graylog.storage.elasticsearch7.Elasticsearch7Plugin]
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,516 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: OpenSearch 2 Support 5.2.3+9aee303 [org.graylog.storage.opensearch2.OpenSearch2Plugin]
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,535 INFO : org.graylog2.bootstrap.CmdLineTool - Running with JVM arguments: -Djava.net.preferIPv6Addresses=true -Djava.library.path=/usr/lib/graylog/lib/sigar
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,622 INFO : org.mongodb.driver.client - MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "4.8.1"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.7.3-arch1-2"}, "platform": "Java/N/A/17.0.10+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.Jep395RecordCodecProvider@2be95d31]}, clusterSettings={hosts=[127.0.0.1:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='30000 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, sendBufferSize=0}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, sendBufferSize=0}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, contextProvider=null}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,623 INFO : org.mongodb.driver.client - MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "4.8.1"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.7.3-arch1-2"}, "platform": "Java/N/A/17.0.10+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.Jep395RecordCodecProvider@2be95d31]}, clusterSettings={hosts=[127.0.0.1:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='30000 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, sendBufferSize=0}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, sendBufferSize=0}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, contextProvider=null}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,631 INFO : org.mongodb.driver.cluster - Cluster description not yet available. Waiting for 30000 ms before timing out
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,635 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=127.0.0.1:27017, type=STANDALONE, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=17, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=11228138}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,677 INFO : org.graylog2.bootstrap.preflight.MongoDBPreflightCheck - Connected to MongoDB version 6.0.13
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,945 INFO : org.mongodb.driver.client - MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "4.8.1"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.7.3-arch1-2"}, "platform": "Java/N/A/17.0.10+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.Jep395RecordCodecProvider@2be95d31]}, clusterSettings={hosts=[127.0.0.1:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='30000 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, sendBufferSize=0}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, sendBufferSize=0}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, contextProvider=null}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,946 INFO : org.mongodb.driver.client - MongoClient with metadata {"driver": {"name": "mongo-java-driver|legacy", "version": "4.8.1"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.7.3-arch1-2"}, "platform": "Java/N/A/17.0.10+7"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.Jep395RecordCodecProvider@2be95d31]}, clusterSettings={hosts=[127.0.0.1:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='30000 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, sendBufferSize=0}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, sendBufferSize=0}, connectionPoolSettings=ConnectionPoolSettings{maxSize=1000, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=UNSPECIFIED, serverApi=null, autoEncryptionSettings=null, contextProvider=null}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,946 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=127.0.0.1:27017, type=STANDALONE, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=17, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=996497}
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,946 INFO : org.mongodb.driver.cluster - Cluster description not yet available. Waiting for 30000 ms before timing out
Feb 06 22:58:02 syslog java[79262]: 2024-02-06 22:58:02,954 INFO : org.graylog2.configuration.IndexerDiscoveryProvider - No indexer hosts configured, using fallback http://127.0.0.1:9200
Feb 06 22:58:03 syslog java[79262]: 2024-02-06 22:58:03,137 INFO : org.graylog2.plugin.system.FilePersistedNodeIdProvider - Node ID: b5822540-f432-45d4-baea-fa4943b8c5a9
Feb 06 22:58:03 syslog java[79262]: 2024-02-06 22:58:03,358 INFO : org.graylog2.configuration.IndexerDiscoveryProvider - No indexer hosts configured, using fallback http://127.0.0.1:9200
Feb 06 22:58:03 syslog java[79262]: 2024-02-06 22:58:03,369 INFO : org.graylog2.plugin.system.FilePersistedNodeIdProvider - Node ID: b5822540-f432-45d4-baea-fa4943b8c5a9
Feb 06 22:58:03 syslog java[79262]: 2024-02-06 22:58:03,428 INFO : org.graylog2.bootstrap.preflight.SearchDbPreflightCheck - Connected to (Elastic/Open)Search version <OpenSearch:2.11.1>
Feb 06 22:58:03 syslog java[79262]: 2024-02-06 22:58:03,501 INFO : org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator null
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,672 INFO : org.graylog2.shared.buffers.InputBufferImpl - Message journal is enabled.
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,674 INFO : org.graylog2.plugin.system.FilePersistedNodeIdProvider - Node ID: b5822540-f432-45d4-baea-fa4943b8c5a9
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,754 INFO : org.graylog.shaded.kafka09.log.LogManager - Loading logs.
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,767 WARN : org.graylog.shaded.kafka09.log.Log - Found a corrupted index file, /var/lib/graylog/data/journal/messagejournal-0/00000000000000000000.index, deleting and rebuilding index...
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,781 INFO : org.graylog.shaded.kafka09.log.LogManager - Logs loading complete.
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,782 INFO : org.graylog2.shared.journal.LocalKafkaJournal - Initialized Kafka based journal at /var/lib/graylog/data/journal
Feb 06 22:58:04 syslog java[79262]: 2024-02-06 22:58:04,797 INFO : org.graylog2.shared.buffers.InputBufferImpl - Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
Feb 06 22:58:05 syslog java[79262]: 2024-02-06 22:58:05,126 INFO : org.graylog2.configuration.IndexerDiscoveryProvider - No indexer hosts configured, using fallback http://127.0.0.1:9200
Feb 06 22:58:05 syslog java[79262]: 2024-02-06 22:58:05,142 INFO : org.graylog2.storage.providers.ElasticsearchVersionProvider - Elasticsearch cluster is running OpenSearch:2.11.1
Feb 06 22:58:05 syslog java[79262]: 2024-02-06 22:58:05,233 INFO : org.graylog2.shared.buffers.ProcessBuffer - Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
Feb 06 22:58:05 syslog java[79262]: 2024-02-06 22:58:05,270 INFO : org.graylog2.buffers.OutputBuffer - Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,102 INFO : org.graylog2.database.dbcatalog.DbEntitiesScanner - 13 entities have been scanned and added to DB Entity Catalog, it took 588.5 ms
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,243 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server 5.2.3+9aee303 starting up
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,243 INFO : org.graylog2.bootstrap.ServerBootstrap - JRE: N/A 17.0.10 on Linux 6.7.3-arch1-2
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,244 INFO : org.graylog2.bootstrap.ServerBootstrap - Deployment: unknown
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,244 INFO : org.graylog2.bootstrap.ServerBootstrap - OS: Arch Linux (arch)
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,244 INFO : org.graylog2.bootstrap.ServerBootstrap - Arch: amd64
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,298 INFO : org.graylog2.bootstrap.ServerBootstrap - Running 67 migrations...
Feb 06 22:58:06 syslog java[79262]: 2024-02-06 22:58:06,451 ERROR: org.graylog2.bootstrap.ServerBootstrap - Exception while running migrations
Feb 06 22:58:06 syslog java[79262]: java.lang.RuntimeException: retentionStrategyConfig
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.migrations.V20190705071400_AddEventIndexSetsMigration.setupEventsIndexSet(V20190705071400_AddEventIndexSetsMigration.java:170) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.migrations.V20190705071400_AddEventIndexSetsMigration.ensureEventsStreamAndIndexSet(V20190705071400_AddEventIndexSetsMigration.java:122) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.migrations.V20190705071400_AddEventIndexSetsMigration.upgrade(V20190705071400_AddEventIndexSetsMigration.java:92) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.bootstrap.ServerBootstrap.lambda$runMigrations$3(ServerBootstrap.java:390) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at com.google.common.collect.CollectSpliterators$1WithCharacteristics.lambda$forEachRemaining$1(CollectSpliterators.java:72) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:104) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at com.google.common.collect.CollectSpliterators$1WithCharacteristics.forEachRemaining(CollectSpliterators.java:72) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.bootstrap.ServerBootstrap.runMigrations(ServerBootstrap.java:387) ~[server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.bootstrap.ServerBootstrap.startCommand(ServerBootstrap.java:313) [server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.bootstrap.CmdLineTool.doRun(CmdLineTool.java:328) [server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:260) [server.jar:?]
Feb 06 22:58:06 syslog java[79262]:         at org.graylog2.bootstrap.Main.main(Main.java:55) [server.jar:?]
Feb 06 22:58:06 syslog systemd[1]: graylog.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 22:58:06 syslog systemd[1]: graylog.service: Failed with result 'exit-code'.
Feb 06 22:58:06 syslog systemd[1]: graylog.service: Consumed 19.483s CPU time.

3. What steps have you already taken to try and solve the problem?

Verified Graylog supports OpenSearch 2.11.1 via docs website: Operating System Packages
Verified Graylog supports MongoDB 6.0.13 via docs website: Operating System Packages
MongoDB database is created by graylog installer – mongosh confirms entires
OpenSearch contains no indices created by Graylog
Able to manually create a test index via CURL to OpenSearch without issue
Verified OpenSearch is working via CURL commands
DataNode is not installed / enabled for Graylog 5.2
graylog server.conf contains follow ES/OpenSearch settings:

elasticsearch_hosts = http://127.0.0.1:9200
elasticsearch_index_prefix = graylog
elasticsearch_template_name = graylog-internal
default_events_index_prefix = gl-events
default_system_events_index_prefix = gl-system-events
elasticsearch_analyzer = standard
enabled_index_rotation_strategies = count,size,time,time-size-optimizing
rotation_strategy = time-size-optimizing
retention_strategy = delete
retention_strategy_config = delete
disabled_retention_strategies = delete

4. How can the community help?

How do I install Graylog 5.2.3 with OpenSearch and MongoDB so that the opensearch indices are created without getting a runtime error of retentionStrategyConfig

patrickmann · February 7, 2024, 10:12am

Not a very helpful error message
Since it’s a fresh install: try dropping the entire mongo DB and rebuilding it.

abrenner · February 7, 2024, 4:10pm

I agree, the runtime exception is not very detailed as to why its throwing an error. Based on the java stack trace and some spelunking I did on the open source code, the error is related to creating an index within OpenSearch. Graylog is performing a bunch of checks before issuing the create index API call to OpenSearch

Below is where the error message is thrown
V20190705071400_AddEventIndexSetsMigration.java#L170

Which is caused by this:

github.com

Graylog2/graylog2-server/blob/master/graylog2-server/src/main/java/org/graylog2/indexer/IndexSetValidator.java#L212


      
                  return Violation.create(
                          f("Index retention setting %s=%d would result in an effective index retention period of %s. This exceeds the configured maximum of %s=%s.",
                                  RetentionStrategyConfig.MAX_NUMBER_OF_INDEXES_FIELD, retentionStrategyConfig.maxNumberOfIndices(), effectiveRetentionPeriod,
                                  ElasticsearchConfiguration.MAX_INDEX_RETENTION_PERIOD, maxRetentionPeriod));
              }
          
              return null;
          }
          
          @Nullable
          public Violation validateRetentionConfig(RetentionStrategyConfig retentionStrategyConfig) {
              ValidationResult validationResult = retentionStrategyConfig.validate(elasticsearchConfiguration);
          
              if (validationResult.failed()) {
                  Optional<String> error = validationResult.getErrors().keySet().stream().findFirst();
                  return Violation.create(error.orElse("Unknown retention config validation error"));
              }
          
              return null;
          }

and ultimately decided by this class:

github.com

Graylog2/graylog2-server/blob/5.2/graylog2-server/src/main/java/org/graylog2/plugin/indexer/retention/RetentionStrategyConfig.java

/*
 * Copyright (C) 2020 Graylog, Inc.
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the Server Side Public License, version 1,
 * as published by MongoDB, Inc.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 * Server Side Public License for more details.
 *
 * You should have received a copy of the Server Side Public License
 * along with this program. If not, see
 * <http://www.mongodb.com/licensing/server-side-public-license>.
 */
package org.graylog2.plugin.indexer.retention;

import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonTypeInfo;

This file has been truncated. show original

I believe I need to enable: max_number_of_indices in my config. I will try this out and report back.

drewmiranda-gl · February 7, 2024, 4:38pm

My recommendation is to comment out all of those settings above except for elasticsearch_hosts = http://127.0.0.1:9200

I haven’t tested but i have a hunch these might be conflicting:

retention_strategy = delete
disabled_retention_strategies = delete

This is setting the retention strategy to delete and then disabling that as a usable retention strategy.

abrenner · February 8, 2024, 3:28am

drewmiranda-gl:

I haven’t tested but i have a hunch these might be conflicting:
retention_strategy = delete
disabled_retention_strategies = delete
This is setting the retention strategy to delete and then disabling that as a usable retention strategy.

Thanks! Removing these settings fixed it. We are up and running.

Thanks again!
-Adam

system · February 22, 2024, 3:29am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Graylog 5.0.2 will not start cleanly... many Java errors Graylog Central (peer support)	9	1017	February 4, 2023
Graylog INPUTS fail Graylog Central (peer support)	7	272	September 19, 2023
Graylog installation issue in Ubuntu server 18.04 Graylog Central (peer support)	2	508	April 29, 2021
Made the jump from Graylog 4 to 5 ? Please share your feedback! Graylog Central (peer support)	8	747	February 23, 2023
Graylog 3.3.X doesn't start. Graylog 3.2.X does Graylog Central (peer support)	20	1865	August 31, 2020

Clean Install v5.2.3 Fails with Runtime Exception: retentionStrategyConfig

Related Topics