1. Describe your incident:
Graylog is in the process of being deployed as a 2x nodes cluster and they share the same MongoDB instance running on node-1 for now, until I have the time to convert this single MongoDB instance into a replicaset.
I can connect with my LDAP/AD credentials on node-1 but on node-2 not always, as it uses to fail and this is shown on the login screen:
Error - the server returned: 503 - There was an error fetching a resource: . Additional information: Authentication service unavailable
The GL log shows this more extensive output:
2022-07-12T11:18:46.717+02:00 ERROR [LDAPAuthServiceBackend] LDAP error
com.unboundid.ldap.sdk.LDAPException: Simple bind operations are not allowed to contain a bind DN without a password.
[ ⊠trimmed for clarity ⊠]
2022-07-12T11:18:46.722+02:00 INFO [SessionCreator] Session creation failed due to authentication service being unavailable. Actor: âurn:graylog:user:foobar@domain.tldâ
2. Describe your environment:
-
OS Information: Ubuntu 20.04 LTS
-
Package Version: Graylog 4.3.2, OpenSearch 1.3.3
-
Service logs, configurations, and environment variables:
Graylog config file:
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = XXXXXXXXXXXXXXXXXXX
root_password_sha2 = XXXXXXXXXXXXXXXXXXX
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 0.0.0.0:9000
elasticsearch_version = 7
elasticsearch_hosts = http://admin:admin@node-1:9200,http://admin:admin@node-2:9200,http://admin:admin@node-3:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://node-1:27017/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32
prometheus_exporter_enabled = true
prometheus_exporter_bind_address = 127.0.0.1:9090
node-2 connects to both OpenSearch and MongoDB:
2022-07-12T11:18:22.839+02:00 INFO [MongoDBPreflightCheck] Connected to MongoDB version 5.0.9
2022-07-12T11:18:22.952+02:00 INFO [SearchDbPreflightCheck] Connected to (Elastic/Open)Search version <OpenSearch:1.3.3>
3. What steps have you already taken to try and solve the problem?
The fix is to log in as the admin user on node-2 and re-enter the LDAP bind password⊠which should be already set on node-1, stored on MongoDB and accessed from node-2âŠ
4. How can the community help?
I suppose that logging in from all nodes in a cluster should be possible and not only from the master.
If that is the case, could you please tell me if am I overseeing the obvious?
The final picture is to have both nodes behind an Azure LB but LDAP authentication must work on both nodes first.
Thanks a lot in advance!