Graylog fail to run on centos when transfering images to another machine

hel · November 10, 2022, 5:10am

Hi, I tried to run graylog on CentOS Linux release 7.6.1810 (Core) machine A and everything works well.
Then I copyied docker images and docker-compose.yml to another CentOS Linux release 7.6.1810 (Core) machine B, repeat same but failed

docker-compose.yml:

version: '3'
services:
    # MongoDB: https://hub.docker.com/_/mongo/
    mongo:
      image: mongo:4.2
      networks:
        - graylog
    # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docker.html
    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
      environment:
        - http.host=0.0.0.0
        - transport.host=localhost
        - network.host=0.0.0.0
        - "ES_JAVA_OPTS=-Dlog4j2.formatMsgNoLookups=true -Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
      deploy:
        resources:
          limits:
            memory: 1g
      networks:
        - graylog
     # Graylog: https://hub.docker.com/r/graylog/graylog/
    graylog:
      image: graylog/graylog:4.3
      environment:
        # CHANGE ME (must be at least 16 characters)!
        - GRAYLOG_PASSWORD_SECRET=somepasswordpepper
        # Password: admin
        - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
        - GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:9005
        - GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9005/
      entrypoint: /usr/bin/tini -- wait-for-it elasticsearch:9200 --  /docker-entrypoint.sh
      networks:
        - graylog
      restart: always
      depends_on:
        - mongo
        - elasticsearch
      ports:
        # Graylog web interface and REST API
        - 9005:9005
        # Syslog TCP
        - 1515:1515
        # Syslog UDP
        - 1515:1515/udp
        # GELF TCP
        - 12202:12202
        # GELF UDP
        - 12202:12202/udp
networks:
    graylog:
      driver: bridge

I tried to load saved image and run another centos machine with commands below:

docker load -i es.tar
docker load -i graylog.tar
docker load -i mongo.tar

docker-compose up

startup log shows:

hel · November 10, 2022, 5:12am

I saved images by command docker save -o.

docker ps shows:

gsmith · November 10, 2022, 5:47am

Hello @hel

Hard to read the screen shots over here. from what I can see it looks like you maybe missing
Plugin _dir and something called realpath: operand which I’m not sure of.

github.com/djmaze/docker-prosody

chown: missing operand

opened 10:39PM - 28 Dec 17 UTC

ncoder-1

I'm getting the following error when I docker run and it fails to start at all: … `chown: missing operand` `Try 'chown --help' for more information.` This is with docker 17.11.0-ce on Arch Here's a bit more detail: ``` [root prosody-docker]# docker build -t xmpp.example.com . Sending build context to Docker daemon 9.728kB Step 1/2 : FROM mazzolino/prosody ---> d06cb22ae177 Step 2/2 : CMD chown prosody /var/lib/prosody && prosodyctl start ---> Running in 8568f2bae49f 2017-12-28T23:06:29.544242259Z container create 8568f2bae49fb29ba0f41c2dd1bcf3a98edac29b8d6a3bb20594915c89ac3786 (image=sha256:d06cb22ae17792c643cc021fb973a5205126e8c0 c4ee3109c57402ba2b310356, name=suspicious_darwin) 2017-12-28T23:06:29.591906291Z container commit 8568f2bae49fb29ba0f41c2dd1bcf3a98edac29b8d6a3bb20594915c89ac3786 (comment=, image=sha256:d06cb22ae17792c643cc021fb973a5 205126e8c0c4ee3109c57402ba2b310356, imageID=sha256:0b46343272f5e4ad184722067d464652caf9a81ff94936d0f84da005b13e03b0, imageRef=, name=suspicious_darwin) Removing intermediate container 8568f2bae49f ---> 0b46343272f5 Successfully built 0b46343272f5 2017-12-28T23:06:29.602522632Z container destroy 8568f2bae49fb29ba0f41c2dd1bcf3a98edac29b8d6a3bb20594915c89ac3786 (image=sha256:d06cb22ae17792c643cc021fb973a5205126e8c 0c4ee3109c57402ba2b310356, name=suspicious_darwin) Successfully tagged xmpp.example.com:latest 2017-12-28T23:06:29.609080602Z image tag sha256:0b46343272f5e4ad184722067d464652caf9a81ff94936d0f84da005b13e03b0 (name=xmpp.exa mple.com:latest) ``` ``` [root prosody-docker]# docker run -d -p 5222:5222 -p 5269:5269 -p 5280:5280 -p 5347:5347 -v $(pwd)/.prosody:/var/lib/prosody -e "XMPP_DOMAIN=example.com" xmpp.example.com b1647363102d83563c44c205965a1761cf111326021863f66a11c6df73331044 2017-12-28T23:06:50.785355478Z container create b1647363102d83563c44c205965a1761cf111326021863f66a11c6df73331044 (image=xmpp.example.com, name=suspicious_mestorf) 2017-12-28T23:06:50.897701787Z network connect 70574a014e4a8b7c69abc35c4c3368fdbb44cddecd7eb253230705035e7aa20e (container=b1647363102d83563c44c205965a1761cf1113260218 63f66a11c6df73331044, name=bridge, type=bridge) 2017-12-28T23:06:51.111876566Z container start b1647363102d83563c44c205965a1761cf111326021863f66a11c6df73331044 (image=xmpp.exa mple.com, name=suspicious_mestorf) 2017-12-28T23:06:51.148692336Z container die b1647363102d83563c44c205965a1761cf111326021863f66a11c6df73331044 (exitCode=1, image=xmpp.example.com, name=suspicious_mest orf) 2017-12-28T23:06:51.265222756Z network disconnect 70574a014e4a8b7c69abc35c4c3368fdbb44cddecd7eb253230705035e7aa20e (container=b1647363102d83563c44c205965a1761cf1113260 21863f66a11c6df73331044, name=bridge, type=bridge) ``` ``` [root prosody-docker]# ls -lRah .: total 24K drwxr-xr-x 5 root root 4.0K Dec 28 23:06 . drwxr-x--- 5 root root 4.0K Dec 28 21:53 .. drwxr-xr-x 2 root root 4.0K Dec 28 21:53 certs drwxr-xr-x 2 root root 4.0K Dec 28 21:52 conf.d -rw-r--r-- 1 root root 78 Dec 28 21:52 Dockerfile drwxr-xr-x 2 105 root 4.0K Dec 28 23:06 .prosody ./certs: total 16K drwxr-xr-x 2 root root 4.0K Dec 28 21:53 . drwxr-xr-x 5 root root 4.0K Dec 28 23:06 .. -rw-r--r-- 1 root root 1.9K Dec 28 21:53 ssl.cert -rw------- 1 root root 3.2K Dec 28 21:53 ssl.key ./conf.d: total 8.0K drwxr-xr-x 2 root root 4.0K Dec 28 21:52 . drwxr-xr-x 5 root root 4.0K Dec 28 23:06 .. ./.prosody: total 8.0K drwxr-xr-x 2 105 root 4.0K Dec 28 23:06 . drwxr-xr-x 5 root root 4.0K Dec 28 23:06 .. ```

Is it possible to show full log for graylog?

hel · November 10, 2022, 6:07am

@gsmith
docker graylog logs:

graylog_1    | realpath: missing operand
graylog_1    | Try 'realpath --help' for more information.
graylog_1    | wait-for-it: waiting 15 seconds for elasticsearch:9200
graylog_1    | wait-for-it: elasticsearch:9200 is available after 11 seconds 
graylog_1    | find '/plugins-default/': No such file or directory
cp_graylog_1   exited with code 1

graylog_1    | realpath: missing operand
graylog_1    | Try 'realpath --help' for more information.
graylog_1    | wait-for-it: waiting 15 seconds for elasticsearch:9200
graylog_1    | wait-for-it: elasticsearch:9200 is available after 0 seconds 
graylog_1    | find '/plugins-default/': No such file or directory
cp_graylog_1   exited with code 1

process of elasticsearch and mongo seems works well

gsmith · November 11, 2022, 3:00am

hello,

I see these two, have you check the plugin directory?

hel · November 15, 2022, 5:26am

The container of graylog doesn’t startup successfully, so I can’t enter the container to check that directory. That container is always in restarting.

gsmith · November 15, 2022, 5:32am

Hey,

What does it show in the logs why Docker cant start your container?
I know I had to reference my journalctl -x. Are you able to check the rest of the logs to find out why Graylog container will not start?

gsmith · November 15, 2022, 5:36am

@hel

This is my compose file, not sure if it will help but You can see I have a plugin directory

graylog:
    #image: graylog/graylog-enterprise:4.3.3-jre11
    image: graylog/graylog-enterprise:4.3.8-jre11
    network_mode: bridge
    dns:
      - 8.8.8.8
      - 8.8.4.4
   # journal and config directories in local NFS share for persistence
    volumes:
       - graylog_journal:/usr/share/graylog/data/journal     
       - graylog_bin:/usr/share/graylog-server/bin/
       - graylog_data:/usr/share/graylog/data/config
       - graylog_log:/usr/share/graylog/data/log
       - graylog_plugin:/usr/share/graylog/data/plugin
       - graylog_content:/usr/share/graylog/data/contentpacks
      # Mount local configuration directory into Docker container
       - graylog_scripts:/usr/share/graylog/scripts      
    environment:
      # Container time Zone
      - TZ=America/Chicago
      # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_PASSWORD_SECRET=pJod1TRZAckHmqM2oQPqX1qnLV

hel · November 15, 2022, 5:44am

I have posted docker logs above. The log repeats
graylog_1 | realpath: missing operand graylog_1 | Try 'realpath --help' for more information. graylog_1 | wait-for-it: waiting 15 seconds for elasticsearch:9200 graylog_1 | wait-for-it: elasticsearch:9200 is available after 11 seconds graylog_1 | find '/plugins-default/': No such file or directory cp_graylog_1 exited with code 1

I run graylog with docker-compose. docker-compose.yml is the same with Docker and Example Version 3 section.

hel · November 15, 2022, 5:46am

version of docker is : 1.13.1

gsmith · November 15, 2022, 5:55am

hey,

I understand , Hence why I was suggesting and defining the logs for you.

EDIT:

Unfortunately those logs don’t help as much as other logs would have to troubleshoot this issue.
Posting them would or might have helped. Sorry I cant be further assistance.

aaronsachs · November 16, 2022, 6:40pm

@hel So…Docker 1.13.1 is 5 years old (see Docker Engine release notes | Docker Documentation). Is there a reason you’re running such an old version and not running something that’s more recent (FWIW, the current docker version is 20.10.1 Docker Engine release notes | Docker Documentation). I think you’d find that running a more recent version of Docker that’s supported would give you a better result.

hel · November 17, 2022, 4:39am

@aaronsachs Yes. I upgrade docker to v20.10.9 and graylog startup successfully. The Welcome page of graylog is shown. The /api/system/sessions returns 200 and session_id is returned./api/users/admin returns 401. /api/users/admin is sent without cookie by browser. There is no proxy.

Username and password is correct.

How can I find log file to solve this 401 problem. Thanks.

hel · November 17, 2022, 4:55am

And api/system/sessions also returns valid_util. Its value is very near to the time the reponse is received. It seems to affect Response’s Set-Cookie Expires value. I don’t whether it’s related to the 401 response.

hel · November 17, 2022, 8:52am

It’s my local network issue. After upgrading docker, graylog works well. Thanks.

aaronsachs · November 17, 2022, 8:54pm

Ah! Good to know. Thanks for the update!

Topic		Replies	Views
Graylog failed to run when transfering images to another centos machine Graylog Central (peer support)	2	546	November 25, 2022
Graylog can't start on cenos Graylog Central (peer support)	2	344	November 24, 2022
graylog docker container on rhel 7 New to Graylog Community? READ-ME FIRST Guides	12	1001	July 19, 2023
Problems with Graylog Docker image Graylog Central (peer support)	28	6640	August 11, 2017
Docker graylog server is up but accesing port 9000 just gives me a white screen Graylog Central (peer support) sidecar	10	4848	January 14, 2020

Graylog fail to run on centos when transfering images to another machine

Related topics