Docker Graylog connect in single MongoDB but not in Replica set

gabriel.lindoso · April 4, 2019, 7:26pm

I’m having problems connecting a Graylog docker to a MongoDB replica set.
I’ve built a MongoDB replica set with 3 servers (1 primary - 2 secondary). But when I execute the command “docker run”, the installation did not connect to MongoDB correctly. I’ve made a test with a single MongoDB and the same “docker run” of Graylog and it worked.

sudo docker run --name mongo -v /opt/open/mongodb/data:/data/db -v /opt/open/mongodb/mongod.conf:/etc/mongod.conf.orig -p 27017-27019:27017-27019 -d mongo --smallfiles

sudo docker run -d -e TZ=America/Sao_Paulo --restart always --name graylog --link mongo -p 80:9000 -p 514:514 -p 514:514/udp -p 12201:12201 -p 12201:12201/udp -v /opt/open/graylog:/usr/share/graylog graylog/graylog:3.0 && sudo docker logs -f graylog &

2019-04-04 15:15:04,951 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server up and running.

But when I execute the same command with a replica set configured, I get this message:

2019-04-04 15:58:01,220 ERROR: org.graylog2.filters.ExtractorFilter - Unable to load extractors for all inputs
com.mongodb.MongoQueryException: Query failed with error code 13 and error message ‘command find requires authentication’ on server mongo:27017

I’ve configured the connection string with user and password created on admin database, but nothing happened.

mongodb_uri = mongodb://admin:admin@server-1:27017,server-2:27017,server-3:27017/admin?replicaSet=rs_graylog_sistemas

I don’t know what else I can do. I’m trying this for months. I’ve already read tha manual and site many times, but nothing helped.

jan · April 5, 2019, 8:21am

how did you configured in Graylog what MongoDB the Graylog should use?

Did you mount a configuration?
Did you use environment variables?

gabriel.lindoso · April 8, 2019, 6:10pm

This is the configuration of MongoDB in Graylog.conf

mongodb_uri = mongodb://admin:admin@server-3:27017,server-2:27017,server-3:27017/admin?replicaSet=rs_graylog_sistemas

I didnt mount a configuration. I use only “docker run” with the variables. It was mentioned above.

jan · April 9, 2019, 6:30am

how did you handover this configuration string to Graylog exactly?

macko003 · April 9, 2019, 12:49pm

Check
grep mongodb_uri /etc/graylog/server/server.conf

And
mongo -u admin -p admin admin
// Are you sure you use the user admin with password admin, and you store the graylog related information in admin database?

gabriel.lindoso · April 9, 2019, 4:29pm

This string is configured in /opt/open/graylog/graylog.conf.

mongodb connection string
#See https://docs.mongodb.com/manual/reference/connection-string/ for details
#mongodb_uri = mongodb://mongo/graylog
#mongodb_uri = mongodb://server-1 (used to test the connection - not successful)

#Authenticate against the MongoDB server
#mongodb_uri = mongodb://grayloguser:secret@mongo:27017/graylog
#mongodb_uri = mongodb://root:admin20@server-1:27017/admin (used to test the connection - not successful)

#Use a replica set instead of a single host
#mongodb_uri = mongodb://grayloguser:secret@mongo:27017,mongo:27018,mongo:27019/graylog
mongodb_uri = mongodb://admin:admin@server-1:27017,server-2:27017,server-3:27017/admin?replicaSet=rs_graylog_sistemas

Both users (root and admin) are configured with role “root” from ADMIN database.

gabriel.lindoso · April 9, 2019, 4:35pm

#From Check
The mongodb_uri string is above.

#From And

From Graylog Server

root@server-2:/# mongo -u admin -p admin admin
MongoDB shell version v4.0.5
connecting to: mongodb://127.0.0.1:27017/admin?gssapiServiceName=mongodb
Implicit session: session { “id” : UUID(“d3d01022-d638-4226-bfe4-8877b4c8ab8e”) }
MongoDB server version: 4.0.5
Welcome to the MongoDB shell.
For interactive help, type “help”.
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
Server has startup warnings:
2019-04-04T14:00:24.450-0300 I STORAGE [initandlisten]
2019-04-04T14:00:24.450-0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-04-04T14:00:24.450-0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten]
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is ‘always’.
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten] ** We suggest setting it to ‘never’
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten]
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is ‘always’.
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten] ** We suggest setting it to ‘never’
2019-04-04T14:00:25.888-0300 I CONTROL [initandlisten]

Enable MongoDB’s free cloud-based monitoring service, which will then receive and display metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()

rs_graylog_sistemas:SECONDARY>

From MongoDB Master Server

root@server-1:/# mongo -u admin -p admin admin
MongoDB shell version v4.0.5
connecting to: mongodb://127.0.0.1:27017/admin?gssapiServiceName=mongodb
Implicit session: session { “id” : UUID(“140e4d98-d485-44f7-b02a-25f8b9278654”) }
MongoDB server version: 4.0.5
Server has startup warnings:
2019-04-04T14:00:18.381-0300 I STORAGE [initandlisten]
2019-04-04T14:00:18.381-0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-04-04T14:00:18.381-0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten]
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is ‘always’.
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten] ** We suggest setting it to ‘never’
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten]
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is ‘always’.
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten] ** We suggest setting it to ‘never’
2019-04-04T14:00:19.540-0300 I CONTROL [initandlisten]

Enable MongoDB’s free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()

rs_graylog_sistemas:PRIMARY>

jan · April 10, 2019, 6:52am

How did you handover the configuration your wrote to Graylog?

Hw should the docker container take the configuration from /opt/open/graylog/graylog.conf?

Check the Documentation again on the confinguration files in docker:

http://docs.graylog.org/en/3.0/pages/installation/docker.html#custom-configuration-files

gabriel.lindoso · April 17, 2019, 7:53pm

I’ve made some modifications on the command and worked well. The MongoDB replica set is up and connected.

sudo docker run -d -e TZ=America/Sao_Paulo --restart always --name graylog --link mongo -p 80:9000 -p 514:514 -p 514:514/udp -p 12201:12201 -p 12201:12201/udp -v /opt/open/graylog/graylog.conf:/usr/share/graylog/data/config/graylog.conf graylog/graylog:3.0 && sudo docker logs -f graylog &

But it still doesn’t connect to the ELS.

2019-04-17 16:21:06,096 INFO : org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical - Cluster not connected yet, delaying index field type initialization until it is reachable.
2019-04-17 16:21:06,110 ERROR: org.graylog2.indexer.cluster.Cluster - Couldn’t read cluster health for indices [graylog_*] (n/a)
2019-04-17 16:21:06,115 INFO : org.graylog2.periodical.IndexerClusterCheckerThread - Indexer not fully initialized yet. Skipping periodic cluster check.
2019-04-17 16:21:06,115 INFO : org.graylog2.periodical.IndexRetentionThread - Elasticsearch cluster not available, skipping index retention checks.
2019-04-17 16:21:06,307 INFO : org.graylog2.migrations.V20161130141500_DefaultStreamRecalcIndexRanges - Cluster not connected yet, delaying migration until it is reachable.
2019-04-17 16:21:20,820 INFO : org.graylog2.periodical.IndexRangesCleanupPeriodical - Skipping index range cleanup because the Elasticsearch cluster is unreachable or unhealthy
2019-04-17 16:45:05,900 WARN : org.graylog2.initializers.BufferSynchronizerService - Elasticsearch is unavailable. Not waiting to clear buffers and caches, as we have no healthy cluster.

I can connect and check the settings on ELS cluster.

[user@server-2 graylog]$ curl ‘10.128.20.21:9200/_cat/health?v’
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1555530558 19:49:18 graylog_prd_1 green 3 3 0 0 0 0 0 0 - 100.0%
[user@server-2 graylog]$

[user@server-2 graylog]$ curl ‘10.128.20.21:9200/_cat/nodes?v’
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.128.20.23 12 57 0 0.14 0.08 0.06 mdi - els3
10.128.20.22 12 61 0 0.21 0.13 0.09 mdi - els2
10.128.20.21 13 60 0 0.01 0.04 0.05 mdi * els1
[user@server-2 graylog]$

[user@server-2 graylog]$ telnet 10.128.20.23 9300
Trying 10.128.20.23…
Connected to 10.128.20.23.
Escape character is ‘^]’.
^]
telnet> quit
Connection closed.
[user@server-2 graylog]$ telnet 10.128.20.23 9200
Trying 10.128.20.23…
Connected to 10.128.20.23.
Escape character is ‘^]’.
^L^]
telnet> quit
Connection closed.
[user@server-2 graylog]$

What am I missing?

jan · April 18, 2019, 6:22am

what setting do you have in your graylog.conf ? The default connects to localhost

gabriel.lindoso · April 18, 2019, 3:37pm

I’m using this line:

elasticsearch_hosts = http://els1:9300,http://els2:9300,http://els3:9300

It’s the same configuration I use on the other graylog structure, but without docker. I use RPM files for everything
1 Graylog installed with MongoDB local and 3 ELS in cluster mode in another network.

jan · April 18, 2019, 6:58pm

are the elasticsearch reachable with that name in your docker setup?

gabriel.lindoso · April 18, 2019, 7:27pm

Yes.

[user@server-2 ~]$ sudo docker exec -it graylog curl ‘els1:9200/_cat/nodes?v’
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.128.20.23 16 57 0 0.00 0.03 0.05 mdi - els3
10.128.20.22 20 62 0 0.01 0.03 0.05 mdi - els2
10.128.20.21 20 61 0 0.05 0.04 0.05 mdi * els1

[user@server-2 ~]$ sudo docker exec -it graylog curl ‘els1:9200/_cat/health?v’
epoch timestamp cluster status node.total node.data shards pri
1555615571 19:26:11 graylog_prd_1 green 3 3 0 0

system · May 2, 2019, 7:27pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Graylog docker did not connect to MongoDB docker Graylog Central (peer support)	22	8608	March 6, 2019
Graylog container cannot connect to MongoDB container Graylog Central (peer support)	2	6756	July 18, 2018
Having issues connecting to mongodb Graylog Central (peer support) mongodb	4	548	August 4, 2024
Unable to connect graylog cluster with mongodb replicaset Graylog Central (peer support) pipeline-rules , debuggingpl	9	11900	March 14, 2018
Can't get docker graylog up and running Graylog Central (peer support)	3	7310	January 2, 2019

Docker Graylog connect in single MongoDB but not in Replica set

From Graylog Server

From MongoDB Master Server

Related topics