Dear Graylog crowd, I would greatly appreciate your help!
Being completely new to Graylog I decided to deploy a simple “hello world” for Graylog on Windows 10 using docker. After 2 days of playing with it I don’t get messages through to the GL webui.
In a nutshell
- Installed Docker desktop for windows
- followed “Installing Graylog” guide (with and without some minor modifications (explained further))
- noted some apparent exceptions of graylog not being able to connect to elasticsearch (when running docker-compose.yml script)
- Able to connect to Graylog’s webui on 127.0.0.0:9000
- Inside webui, System->inputs, opened an input stream for raw text TCP on port 5555
- used ncat tool for Windows from zenmap to inject a message on localhost:5555
- realized that no messages arrive or being logged by Graylog.
My docker-compose.yml
version: '2'
services:
mongodb:
image: mongo:3
tty: true
volumes:
- mongo_data:/data/db
networks:
- esnet
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.6.2
tty: true
container_name: elasticsearch
volumes:
- es_data:/usr/share/elasticsearch/data
environment:
- bootstrap.memory_lock=true
- http.host=0.0.0.0
- transport.host=localhost
- network.host=0.0.0.0
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
mem_limit: 1g
ports:
- 9200:9200
networks:
- esnet
graylog:
image: graylog/graylog:3.0
tty: true
volumes:
- graylog_journal:/usr/share/graylog/data/journal
environment:
# CHANGE ME (must be at least 16 characters)!
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
# Password: admin
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
- GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000
- GRAYLOG_ELASTICSEARCH_HOSTS=http://elasticsearch:9200
links:
- mongodb:mongo
- elasticsearch
depends_on:
- mongodb
- elasticsearch
ports:
# Graylog web interface and REST API
- 9000:9000
# Syslog TCP
- 1514:1514
# Syslog UDP
- 1514:1514/udp
# GELF TCP
- 12201:12201
# GELF UDP
- 12201:12201/udp
# raw TCP
- 5555:5555
# raw UDP
- 5555:5555/udp
networks:
- esnet
volumes:
mongo_data:
driver: local
es_data:
driver: local
graylog_journal:
driver: local
networks:
esnet:
Running docker-compose up
…
Good status messages except following:
graylog_1 | 2019-08-19 16:29:49,783 ERROR: org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical - Couldn’t update field types for index set <Default index set/5d5a5f582ab79c000e55e047>
…
graylog_1 | Caused by: io.searchbox.client.config.exception.CouldNotConnectException: Could not connect to http://elasticsearch:9200
…
graylog_1 | Caused by: org.apache.http.conn.HttpHostConnectException: Connect to elasticsearch:9200 [elasticsearch/172.30.0.2] failed: Connection refused (Connection refused)
Test sending data
echo “test message\n” | ncat -v -u 127.0.0.1 5555
Ncat: Version 7.80 ( )
Ncat: Connected to 127.0.0.1:5555.
Ncat: 19 bytes sent, 0 bytes received in 0.38 seconds.
Result
Apparently Graylog starts OK even with above errors and warnings. I am able to create an input inside webui system->inputs
* bind_address:
127.0.0.1
* max_message_size:
2097152
* number_worker_threads:
2
* override_source:
*<empty>*
* port:
5555
* recv_buffer_size:
1048576
* tcp_keepalive:
false
* tls_cert_file:
*<empty>*
* tls_client_auth:
disabled
* tls_client_auth_cert_file:
*<empty>*
* tls_enable:
false
* tls_key_file:
*<empty>*
* tls_key_password:
********
* use_null_delimiter:
false
However no messages go through
Questions/Discussion
-
as the docker compose up loads up I am able to see java exceptions of graylog failing to connect to elasticsearch:
Connect to elasticsearch:9200 [elasticsearch/172.29.0.2] failed: Connection refused (Connection refused)
I have tried setting/removing elasticsearch ports inside the .yml
ports:- 9200:9200
- 9300:9300
-
I have tried setting the GRAYLOG_ELASTICSEARCH_HOSTS variable inside graylog environment section of the .yml like this:
- GRAYLOG_ELASTICSEARCH_HOSTS=0.0.0.0:9200
- GRAYLOG_ELASTICSEARCH_HOSTS=127.0.0.1:9200
- GRAYLOG_ELASTICSEARCH_HOSTS=elasticsearch:9200
“http” prefix omitted due to being a new user (cant have more than two links in my posts)
-
I have tried setting networks section (not sure if it is needed):
networks:- esnet
-
Regardless of all that I am still not getting a clean start-up of the docker-compose, always complaints about “connection refused” whether on elasticsearch:9200 [elasticsearch/172.29.0.2] or 127.0.0.1:9200 or 0.0.0.0:9200 depending on the setting in the .yml.
-
I have tried disabling firewall on my PC, none of this has moved me closer to success.
I ran out of ideas and kindly ask you folks to point me towards the light in the end of the tunnel. Do I need to play with Hyper V manager? is my elasticsearch version 6.6.2 not good?, any additional port forwarding between containers? or is there anything obvious I am missing?
Thanks
