Web interface not working all of sudden

Graylog Central (peer support)
1

Description of your problem

Web Interface is not loading

Description of steps you’ve taken to attempt to solve the issue

image
image1459×33 3.68 KB

Environmental information

Operating system information

  • RHEL
  • CentOS
  • Ubuntu
  • Debian
  • Containers (e.g., Docker, Kubernetes, etc.)
  • FreeBSD
  • Windows
  • Other (e.g., using config management like Chef, Puppet, Ansible or Salt to deploy Graylog)

Package versions

  • Graylog
  • MongoDB
  • Elasticsearch
  • Service logs, configuration, and environment variables
  • See the docs site for all file locations

NOTE: For all container-based deployments, please include your full, redacted YAML configuration file

NOTE: When posting log output or code snippets (e.g., JSON, YAML, etc.), please surround your code with three backticks like so:

```
 Your code goes here
```

For longer code or configuration bits, please enclose your snippet in a summary block like this:

Summary of your code snippet or config here 
Your code goes inside the triple backticks
2

It looks to be diskspace on Database but when attempting to clean up a i get failed to connect

/dev/mapper/graylog–vg-root 19035388 18045400 0 100% /

ubuntu@graylog:~$ curl -XPUT -H “Content-Type: application/json” http://localhost:9200/_cluster/settings -d ‘{ “transient”: { “cluster.routing.allocation.disk.threshold_enabled”: false } }’
curl: (7) Failed to connect to localhost port 9200: Connection refused

3

Hello && Welcome
I might be able to help withyour issue.

Just noting that posting an error under this section doesnt really show what you have done.
From the screen shot the error looks like a connection issue with you database (MongDb) make sure Graylog can connect ot MongDb.
Things to check are:

  • Status of the service systemctl status mongod What does it show?
  • Was there any updates on this servers before this issue?
  • Was the Graylog service restart prior to this issue? If so, check you Graylog configuration file to make sure there were no changes, specially with your mongodb connection configuration.
  • Check with ps, netstat and netcat whether the things are up and running.
  • Do you have a firewall enabled?
    It would help us, help you to show your Graylog configuration file.
4

No updates no changes been working for over a year.

I think the main issue is diskspace as this location is 100%
/dev/mapper/graylog–vg-root 19035388 18045400 0 100% /

but when I try to clear space getting failed to connect

ubuntu@graylog:~$ curl -XPUT -H “Content-Type: application/json” http://localhost:9200/_cluster/settings -d ‘{ “transient”: { “cluster.routing.allocation.disk.threshold_enabled”: false } }’
curl: (7) Failed to connect to localhost port 9200: Connection refused

ubuntu@graylog:~$ systemctl status mongod
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: e
Active: failed (Result: exit-code) since Mon 2021-08-16 20:45:46 UTC; 1h 45mi
Docs: https://docs.mongodb.org/manual
Process: 1493 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited
Main PID: 1493 (code=exited, status=1/FAILURE)

Aug 16 20:45:46 graylog systemd[1]: Started MongoDB Database Server.
Aug 16 20:45:46 graylog mongod[1493]: 2021-08-16T20:45:46.054+0000 I STORAGE [m
Aug 16 20:45:46 graylog systemd[1]: mongod.service: Main process exited, code=ex
Aug 16 20:45:46 graylog systemd[1]: mongod.service: Failed with result 'exit-cod
lines 1-11/11 (END)…skipping…
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-08-16 20:45:46 UTC; 1h 45min ago
Docs: https://docs.mongodb.org/manual
Process: 1493 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=1/FAILURE)
Main PID: 1493 (code=exited, status=1/FAILURE)

Aug 16 20:45:46 graylog systemd[1]: Started MongoDB Database Server.
Aug 16 20:45:46 graylog mongod[1493]: 2021-08-16T20:45:46.054+0000 I STORAGE [main] Max cache overflow file size custom option: 0
Aug 16 20:45:46 graylog systemd[1]: mongod.service: Main process exited, code=exited, status=1/FAILURE
Aug 16 20:45:46 graylog systemd[1]: mongod.service: Failed with result ‘exit-code’.
~

5

Is this a virtual machine?

So mongo staus is failed.
As for disk space what do you get when you excute this?

root # df -h

image
image1213×546 8.08 KB

EDIT: for some reason I just seen

If you have no room on this volume, your probably having a hard time typing bash commands to either delete file or to add more disk space. I’ve done this once.

6

That’s correct. The curl cmds to localhost fail to connect

ubuntu@graylog:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 2.0G 0 2.0G 0% /dev
tmpfs 395M 740K 394M 1% /run
/dev/mapper/graylog–vg-root 19G 18G 0 100% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 395M 0 395M 0% /run/user/1000

7

Out of curiosity, are you running this on the OVA? I went back and looked over your initial post, and didn’t see much of anything in terms of config, versions, OS, etc.

8

It’s an OVA, I inherited the system from prior Admin. After doing some research I expanded the drive 5GB. Then ran the two curl cmds I had found in the forums.

I am back in web portal and will be looking to add some additional space. Thanks ever one that replied.

1 Like
9

Ah, good to know. As an aside, the OVA was only ever intended for testing Graylog–it’s not hardened and wouldn’t be suitable for production use. I’d suggest migrating off of it onto a more production-ready deployment.

1 Like
10

Glad you solved you issue :slight_smile:

1 Like
11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.