Turning a 2 node graylog cluster in to only single node

How can I configure graylog in a single node? . Graylog was working as a 2 node cluster (was like that when I arrived this company) but now for reducing cloud cost I have to use only one server. Cluster had a public LB ip and the 2 nodes added. Now I remove one of the nodes and changed the configuration of graylog as follow: I just remove the other node IP form mongo and elastic search configuration and left the rest as it was.

############################
# GRAYLOG CONFIGURATION FILE
############################
is_master = True
node_id_file = /etc/graylog/server/node-id
password_secret = 2jueVqZpwLLjaWxV
root_username = 
root_password_sha2 = some pass here
root_timezone = EST
plugin_dir = /usr/share/graylog-server/plugin
message_journal_enabled = false
message_journal_dir = /var/lib/graylog-server/journal

###########################
# WEB & REST API
###########################
http_bind_address = 10.0.5.83:9000  (local private adress
http_publish_uri = http://45.87.x.x:9000  (public ip adress of LB)
http_enable_cors = True
http_enable_gzip = True
http_max_header_size = 8192
http_thread_pool_size = 16
http_enable_tls = False
http_tls_cert_file =
http_tls_key_file =
http_tls_key_password =

#############################
# Elasticsearch
#############################
elasticsearch_hosts = http://10.0.5.83:9200  *(removed the second ip of the other node)*
elasticsearch_connect_timeout = 10s
elasticsearch_socket_timeout = 60s
elasticsearch_max_total_connections = 20
elasticsearch_max_total_connections_per_route = 2
elasticsearch_max_retries = 2
elasticsearch_discovery_enabled = False
elasticsearch_index_prefix = graylog
elasticsearch_template_name = graylog-internal
elasticsearch_analyzer = standard
disable_index_optimization = False
elasticsearch_analyzer = standard
elasticsearch_request_timeout = 1m
index_ranges_cleanup_interval = 1h
output_batch_size = 25
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
outputbuffer_processor_keep_alive_time = 5000
outputbuffer_processor_threads_core_pool_size = 3
outputbuffer_processor_threads_max_pool_size = 30
udp_recvbuffer_sizes = 1048576
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
index_optimization_max_num_segments = 1
no_retention = False
async_eventbus_processors = 2
lb_recognition_period_seconds = 3
lb_throttle_threshold_percentage = 95
stream_processing_timeout = 2000
stream_processing_max_faults = 3
alert_check_interval = 60
output_module_timeout = 10000
stale_master_timeout = 2000
shutdown_timeout = 30000

#################################
# Message Journal
#################################
message_journal_enabled = True
message_journal_dir = /var/lib/graylog-server/journal
message_journal_max_age = 12h
message_journal_max_size = 5gb
message_journal_flush_age = 1m
message_journal_flush_interval = 1000000
message_journal_segment_age = 1h
message_journal_segment_size = 100mb

#############################
# MongoDB
#############################
mongodb_uri = mongodb://graylog:graylog@10.0.5.83:27017/graylog?replicaSet=rs0
mongodb_max_connections = 100
mongodb_threads_allowed_to_block_multiplier = 5

#############################
# Script Alerts
#############################
#integrations_web_interface_uri =
integrations_scripts_dir = /usr/share/graylog-server/script

#############################
# Miscellaneous
#############################
rules_file =
http_connect_timeout = 5s
http_read_timeout = 10s
http_write_timeout = 10s
disable_index_optimization = False
index_optimization_max_num_segments = 1
ldap_connection_timeout = 2000
disable_sigar = False
dashboard_widget_default_cache_time = 10s
content_packs_loader_enabled = True
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load =
proxied_requests_thread_pool_size = 32
gc_warning_threshold = 1s

Also remove the other cluster name in /etc/elasticsearch/elasticsearch.yml
and in /etc/mongod.conf.

Mongo file is now like this:

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  #dbPath: /data/db
  journal:
    enabled: true
  mmapv1:
    smallFiles: true
#  engine:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: localhost, 10.0.5.83

security:
  keyFile: /opt/mongodb/keyfile

#operationProfiling:

replication:
   replSetName: "rs0"

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

But I´m getting in both mongod.conf and graylog logs erros that is still searching for the other node even I already remove all the lines regarding the other server that was part of the cluster. What else should I have to configure to keep a single node graylog?

I´m getting from graylog log:

2019-09-06T11:55:03.478-04:00 INFO [cluster] No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=10.0.5.83:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 10]}, minWireVersion=0, maxWireVersion=7, maxDoc

and from mongo log:

root@Gray-2/var/log/mongodb# tail -f mongod.log
2019-09-06T11:58:05.264-0400 I ASIO     [Replication] Connecting to 10.0.5.82:27017
2019-09-06T11:58:06.234-0400 I REPL     [replexec-1] Not starting an election, since we are not electable due to: Not standing for election because I cannot see a majority (mask 0x1)
2019-09-06T11:58:08.336-0400 I ASIO     [Replication] Failed to connect to 10.0.5.82:27017 - HostUnreachable: Error connecting to 10.0.5.82:27017 :: caused by :: No route to host
2019-09-06T11:58:08.336-0400 I CONNPOOL [Replication] Dropping all pooled connections to 10.0.4.62:27017 due to HostUnreachable: Error connecting to 10.0.5.82:27017 :: caused by :: No route to host
2019-09-06T11:58:08.336-0400 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 398) to 10.0.4.62:27017, response status: HostUnreachable: Error connecting to 10.0.5.82:27017 :: caused by :: No route to

I checked in graylog configuration and nothing is related to replica configuration. How can I recofngiured all this to leave graylog running in one node?

Solved the issue removing replica set config in graylog like folow:

`mongodb_uri = mongodb://graylog:graylog@10.0.5.83:27017/graylog

and also commenting in mongo db:

replication:
   replSetName: "rs0"

but now need to fix the issue with elastic search, as I´m getting elastic search red message. I asume becuse is not able to access the shards in the node that is off?

It highly depends how you had configured elasticsearch, what are your sharding settings and replica settings before you have remove the server?

Did you drain out the server before you power it off? Did you make any attemp to move all data to the leftover server?

Check your elasticsearch logs they will give you guidance.

No I didn´t. However I entrer the graylog portal with new node and remove all indexes and inputs that were created when cluster was running. Now the index failure message is gone, but still getting the elasticsearch issue:

**Elasticsearch cluster is red.** Shards: 8 active, 0 initializing, 0 relocating, 4 unassigned,

I already restarted elasticsearch service in node B, and as I said I removed all old indices and inputs. What else can I do the wipe or remove this old shards that I won´t use in new configuration with this single node graylog?

what is the response of the following command against your elasticsearch server?

curl -X GET "localhost:9200/_cat/indices?v&pretty"
health status index     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
red    open   graylog_5 YTmhJMbSQqWnSV4SiMGpvw   4   0          0            0       522b           522b
green  open   graylog_8 w3NaOe7CS3aOCKTkYbVoLw   4   0    1876740            0    379.6mb        379.6mb
red    open   graylog_7 ArtpN0dOQliBKr_fySRcRw   4   0      38622            0      5.6mb          5.6mb

you might need to delete the red indices also to get it work again.

yeap, just did ant elastisearch turn green automatically. Thanks man

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.