# Unable to log into web UI (Error 500) **URL:** \ **Category:** Graylog Central (peer support)\ **Tags:** mongodb\ **Created:** [November 29, 2022, 11:51am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704 "2022-11-29T11:51:15Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![rstellma](https://avatars.discourse-cdn.com/v4/letter/r/6a8cbe/32.png) [@rstellma](https://community.graylog.org/u/rstellma)\ **Post date:** [November 29, 2022, 11:51am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/1 "2022-11-29T11:51:15Z")
**1. Describe your incident:** A few days ago, we had to rebuild our MongoDB replica set. Since then we are no longer able to log into Graylog web interface, neither with ‘admin’ account, nor with personal accounts. When attempting to log in, an error is displayed: “Error - the server returned: 500 - There was an error fetching a resource: Internal Server Error. Additional information: Couldn’t load session” No changes to graylog configuration were made. Until changes were made to the replica set, graylog login worked. **2. Describe your environment:** - OS Information: ```auto # grep ^VERSION= /etc/os-release VERSION="20.04.5 LTS (Focal Fossa)" ``` - Package Version: ```auto # apt list --installed | grep -E 'graylog|elasticsearch' WARNING: apt does not have a stable CLI interface. Use with caution in scripts. elasticsearch-oss/stable,now 7.10.2 amd64 [installed] graylog-4.2-repository/stable,now 1-4 all [installed] graylog-server/stable,now 4.2.13-1 all [installed] ``` MongoDB is 4.4.13 (we use the MongoDB Cloud Manager to install and maintain the replica set). From mongod `host1`: ```auto $> mongod --version db version v4.4.13 Build Info: { "version": "4.4.13", "gitVersion": "df25c71b8674a78e17468f48bcda5285decb9246", "openSSLVersion": "OpenSSL 1.1.1 11 Sep 2018", "modules": [], "allocator": "tcmalloc", "environment": { "distmod": "ubuntu1804", "distarch": "x86_64", "target_arch": "x86_64" } } ``` - Service logs, configurations, and environment variables: - /etc/graylog/server/server.conf ```auto # grep ^http /etc/graylog/server/server.conf http_bind_address = 0.0.0.0:9000 ``` ```auto # grep ^mongodb_uri /etc/graylog/server/server.conf mongodb_uri = mongodb://host1:27017,host2:27017,host3:27017/graylog?replicaSet=replSet01 ``` - /var/log/graylog-server/server.log: ```auto 2022-11-29T11:02:44.704Z ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource java.lang.RuntimeException: Couldn't load session at org.graylog2.security.MongoDbSessionDAO.doUpdate(MongoDbSessionDAO.java:101) ~[graylog.jar:?] at org.apache.shiro.session.mgt.eis.CachingSessionDAO.update(CachingSessionDAO.java:277) ~[graylog.jar:?] at org.apache.shiro.session.mgt.DefaultSessionManager.onChange(DefaultSessionManager.java:212) ~[graylog.jar:?] at org.apache.shiro.session.mgt.AbstractNativeSessionManager.applyGlobalSessionTimeout(AbstractNativeSessionManager.java:125) ~[graylog.jar:?] at org.apache.shiro.session.mgt.AbstractNativeSessionManager.start(AbstractNativeSessionManager.java:100) ~[graylog.jar:?] at org.apache.shiro.mgt.SessionsSecurityManager.start(SessionsSecurityManager.java:152) ~[graylog.jar:?] at org.apache.shiro.subject.support.DelegatingSubject.getSession(DelegatingSubject.java:340) ~[graylog.jar:?] at org.apache.shiro.subject.support.DelegatingSubject.getSession(DelegatingSubject.java:316) ~[graylog.jar:?] at org.graylog2.shared.security.SessionCreator.create(SessionCreator.java:80) ~[graylog.jar:?] at org.graylog2.rest.resources.system.SessionsResource.newSession(SessionsResource.java:146) ~[graylog.jar:?] at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_352] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_352] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_352] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_352] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352] ``` **3. What steps have you already taken to try and solve the problem?** - verified, that no changes were made to `/etc/graylog/server/server.conf` - especially: `mongodb_uri` - verified, hostnames in `mongodb_uri` resolve to the correct IP addresses of mongod nodes - verified, replica set is set up properly and can be written to and read from (tested from various internal applications relying on MongoDB) - `host1` is primary, `host2` and `host3` are secondaries - verified, `graylog` database exists in replica set - verified, that no auth is required against replica set **4. How can the community help?** Any advise is appreciated - if deleting the replica set and creating a new one with same name is the cause of the server error 500 when attempting to log in - if there are any information about the replica set (i.e.: UUID, …) stored in graylog and if so, - how to change it or any other way to (re-)connect the graylog instance to the replica set, - helping to interpret the above error in graylog’s server.log (google did not help very much 🫤), especially: - what the exact cause is, - how and where to find further information and - how to solve the issue. Thanks in advance! ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [November 29, 2022, 11:32pm UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/2 "2022-11-29T23:32:26Z")
Hello @rstellma I see that the status & access of MongoDb was checked. > [@rstellma](#): > > if deleting the replica set and creating a new one with same name is the cause of the server error 500 when attempting to log in I need to ask a couple question about rebuild that was executed. Just to make sure , you are aware that all settings and configuration metadata are kept in MongoDb & Graylog is just the from end? Was the MongoDb nodes deleted and then rebuild from the ground up or did you save the databases from each node and installed them back? What does the rest of the log file look like? If you restart Graylog service and` tail -f` the log file is it possible to post that here? How about Firewall, SELinux/Apparmor are any of those enable? I know this will sound redundant but have you tried shutting down Graylog services, rebooting MongoDb services and starting Graylog service back up? Then tailing GL log file when it starts up. This kind of shows a clear picture on what is going on. Did you see anything in the MongoDb log’s? Was this rebuild to Upgrade MongoDb instance? > [@rstellma](#): > > `host1` is primary, `host2` and `host3` are secondaries Could you explain in greater detail how this environment is set up. From what I get is you have three nodes with GL/ES/ and MongoDb on them is this correct? EDIT: Out of curiosity what does the status of Graylog look like? `systemctl status graylog-server` EDIT:2 I noticed you have a `http_bind to 0.0.0.0`, are you using a load balancer in front of this cluster (i.e. nginx)? Just doing some research on this issue, I found this posts below, Notice the logs file that was posted? looks very familiar 🤔 but your log tends to be from Mongo. - [Newbie: Rule not saving](https://community.graylog.org/t/newbie-rule-not-saving/4655) And here - [After the graylog version was upgraded from 4.2 to 4.3, the cluster could not find the master node](https://community.graylog.org/t/after-the-graylog-version-was-upgraded-from-4-2-to-4-3-the-cluster-could-not-find-the-master-node/24580) ---
**Author:** ![rstellma](https://avatars.discourse-cdn.com/v4/letter/r/6a8cbe/32.png) [@rstellma](https://community.graylog.org/u/rstellma)\ **Post date:** [November 30, 2022, 9:02am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/3 "2022-11-30T09:02:28Z")
Hello @gsmith Thanks for your reply. I will try to answer your questions as clearly and comprehensively as I can, but i am not a graylog expert. Please excuse, if some points may remain unclear. > Just to make sure , you are aware that all settings and configuration metadata are kept in MongoDb & Graylog is just the from end? Yes, i am aware of that. I checked the `graylog` database in MongoDB: there are several tables with data. I hoped graylog would connect to the database and keep using that data. > Was the MongoDb nodes deleted and then rebuild from the ground up or did you save the databases from each node and installed them back? The nodes were not deleted and no databases were re-installed. We use the MongoDB automation agent in conjunction with the MongoDB Cloud Manager to maintain the nodes and the replica set. In Cloud Manager all configuration is done (i.e.: set configuration variables, add nodes to replica set, …), the Automation Agent pulls the changes to the nodes and/or replica set and deploys them on the nodes. We rebuilt the replica set, by disabling the automation in the Cloud Manager UI, deleted the replica set entry, added a new replica set with same name and added the nodes to the replica set. While we were taking these steps, the replica set has continued to work. We verified that with `rs.status()` in `mongos`. No node was reinstalled, no databases were deleted and reinstalled. We just disabled automation via Cloud Manager for a few minutes. > What does the rest of the log file look like? If you restart Graylog service and tail -f the log file is it possible to post that here? Log file would be too large for this forum, you may fetch it from [here](https://cloud.saar-storage.de/s/Q9fNmmZ9X7HEKWC). > How about Firewall, SELinux/Apparmor are any of those enable? No firewall, no Apparmor. > have you tried shutting down Graylog services, rebooting MongoDb services and starting Graylog service back up? Then tailing GL log file when it starts up. I did and log shows the same as above. > From what I get is you have three nodes with GL/ES/ and MongoDb on them is this correct? No. GS/EL is a single node, MongoDB runs on 3 different nodes independent of the graylog node > Out of curiosity what does the status of Graylog look like? > > systemctl status graylog-server ```auto ● graylog-server.service - Graylog server Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2022-11-30 08:24:09 UTC; 22min ago Docs: http://docs.graylog.org/ Main PID: 465917 (graylog-server) Tasks: 360 (limit: 28753) Memory: 2.6G CGroup: /system.slice/graylog-server.service ├─465917 /bin/sh /usr/share/graylog-server/bin/graylog-server └─465965 /usr/bin/java -Xms4g -Xmx4g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:-OmitStackTraceInFastThrow -Djdk.tls.acknowledgeCloseNotify=true -Dlog4j2.formatMsgNoLookups=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -jar -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar -Dgraylog2.installation_source=deb /usr/share/graylog-server/graylog.jar server -f /etc/graylog/server/server.conf -np Nov 30 08:24:09 graylog systemd[1]: Started Graylog server. ``` > I noticed you have a http\_bind to 0.0.0.0, are you using a load balancer in front of this cluster (i.e. nginx)? No loadbalancer in front of graylog. > Notice the logs file that was posted? looks very familiar 🤔 but your log tends to be from Mongo. No. The log is definitely from the graylog’s `server.log`. I hope I was able to answer your questions in sufficient detail? ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [November 30, 2022, 11:05pm UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/4 "2022-11-30T23:05:57Z")
Hello @rstellma Just from the log files you showed I can see a lot, much appreciated. To be honest it looks good I did find some WARN’ings in there. I see Graylog can see MongoDb. ```auto cluster] No server chosen by com.mongodb.client.internal.MongoClientDelegate$1@4f4d05b3 from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=host1:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=host3:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=host2:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2022-11-30T08:24:16.277Z INFO [connection] Opened connection [connectionId{localValue:1, serverValue:48467}] to host2:27017 2022-11-30T08:24:16.277Z INFO [connection] Opened connection [connectionId{localValue:2, serverValue:392412}] to host3:27017 2022-11-30T08:24:16.278Z INFO [connection] Opened connection [connectionId{localValue:3, serverValue:2440461}] to host1:27017 2022-11-30T08:24:16.290Z INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=host3:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 4, 13]}, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=8411832, setName='replicaSet', canonicalAddress=host3:27017, hosts=[host2:27017, host3:27017, host1:27017], passives=[], arbiters=[], primary='host1:27017', tagSet=TagSet{[]}, electionId=null, setVersion=28789, lastWriteDate=Wed Nov 30 08:24:16 UTC 2022, lastUpdateTimeNanos=695892175957107} 2022-11-30T08:24:16.290Z INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=host1:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 4, 13]}, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=8628150, setName='replicaSet', canonicalAddress=host1:27017, hosts=[host2:27017, host3:27017, host1:27017], passives=[], arbiters=[], primary='host1:27017', tagSet=TagSet{[]}, electionId=7fffffff0000000000000082, setVersion=28789, lastWriteDate=Wed Nov 30 08:24:16 UTC 2022, lastUpdateTimeNanos=695892176180898} 2022-11-30T08:24:16.292Z INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=host2:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 4, 13]}, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=4895870, setName='replicaSet', canonicalAddress=host2:27017, hosts=[host2:27017, host3:27017, host1:27017], passives=[], arbiters=[], primary='host1:27017', tagSet=TagSet{[]}, electionId=null, setVersion=28789, lastWriteDate=Wed Nov 30 08:24:16 UTC 2022, lastUpdateTimeNanos=695892172799732} 2022-11-30T08:24:16.304Z INFO [cluster] Setting max election id to 7fffffff0000000000000082 from replica set primary host1:27017 2022-11-30T08:24:16.304Z INFO [cluster] Setting max set version to 28789 from replica set primary host1:27017 2022-11-30T08:24:16.304Z INFO [cluster] Discovered replica set primary host1:27017 2022-11-30T08:24:16.314Z INFO [connection] Opened connection [connectionId{localValue:4, serverValue:2440462}] to host1:27017 ``` This mean you have GEO enabled but Graylog cant find this database **GeoLite2-City.mmdb**. > GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb This is an operating system configuration issue. Pretty easy to fix, just google it > [UdpTransport] receiveBufferSize (SO\_RCVBUF) For the name Rebuild/Rebuilt, I just assumed you recreate those MongoDb nodes. I personal do not see anything that would be and issue. As you stated above this worked prior until MongoDB automation agent was execute? Not sure all what that does but unfortunately I’m not familiar with that service. Have you check MongoDb log’s also? So to re-cap You can not login the Web UI at all? Some suggestion for check list: Permissions - Check Files and Directory’s make sure Graylog and/mongo has access to what they own. What I don’t get is that this worked before, I’m wondering what this MongoDB automation agent actually performed and Why. Question: if you only have a `http_bind 0.0.0.0.9000` on all three Graylog servers, which address are you using , gl\_node-01, gl\_node-02 or gl\_node-03, Also are you just using ip address for FQDN? Have you tried other Graylog addresses. Last question: Have you tried to reset the admin password. Which is found in this documentation below. [https://docs.graylog.org/v1/docs/ubuntu](https://docs.graylog.org/v1/docs/ubuntu) Please read the whole screen shot. ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/4/4ae8634346d24818889f4dfc1e5ec6e40d320c3e.png) EDIT I’m not sure if I read that correct, do you only have 1 Graylog-Server /w elasticsearch and 3 MongoDb node? ---
**Author:** ![rstellma](https://avatars.discourse-cdn.com/v4/letter/r/6a8cbe/32.png) [@rstellma](https://community.graylog.org/u/rstellma)\ **Post date:** [December 5, 2022, 1:35pm UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/5 "2022-12-05T13:35:51Z")
Hello @gsmith Thanks for your reply and please excuse my late repsonse. It’s been busy days and i have not had the time yet to further troubleshoot the issue. > [@gsmith](#): > > This mean you have GEO enabled but Graylog cant find this database **GeoLite2-City.mmdb**. > > > GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb > > This is an operating system configuration issue. Pretty easy to fix, just google it I will once i find the time. Are you positively sure, this has something to do with the issue in question? > [@](#): > > As you stated above this worked prior until MongoDB automation agent was execute? Not sure all what that does but unfortunately I’m not familiar with that service. It’s a management software to manage and maintain MongoDB clusters and replica sets > **[MongoDB Agent — MongoDB Cloud Manager](https://www.mongodb.com/docs/cloud-manager/tutorial/nav/mongodb-agent/)** > [@](#): > > Have you check MongoDb log’s also? I did and as far i can tell (i am not a MongoDB expert, either) it looks ok: - graylog connects to the replica set - graylog writes some information into the `graylog`database (collection: ‘sessions’) > So to re-cap You can not login the Web UI at all? True > Some suggestion for check list: > Permissions - Check Files and Directory’s make sure Graylog and/mongo has access to what they own. Checked and verified, that permissions are ok. We did not change anything there, anyway. > Question: if you only have a `http_bind 0.0.0.0.9000` on all three Graylog servers, which address are you using , gl\_node-01, gl\_node-02 or gl\_node-03, Also are you just using ip address for FQDN? Have you tried other Graylog addresses. As i said earlier: it’s only one standalone graylog node. > Last question: > Have you tried to reset the admin password. Which is found in this documentation below. I did not. If you are positively sure, resetting the password will help, i will try this step. > EDIT I’m not sure if I read that correct, do you only have 1 Graylog-Server /w elasticsearch and 3 MongoDb node? Correct. - 1 standalone node with graylog and elasticsearch on same node. - 3 mongod nodes ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [December 5, 2022, 10:19pm UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/6 "2022-12-05T22:19:37Z")
Hello, > [@rstellma](#): > > I did not. If you are positively sure, resetting the password will help, i will try this step. Troubleshooting I would, BUT copy the graylog configuration file before you do any configurations. `root # cp server.conf server.conf.backup` Then execute the following ```auto root # echo -n "Enter Password: " && head -1 **Author:** ![rstellma](https://avatars.discourse-cdn.com/v4/letter/r/6a8cbe/32.png) [@rstellma](https://community.graylog.org/u/rstellma)\ **Post date:** [December 14, 2022, 9:56am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/7 "2022-12-14T09:56:23Z") Hello @gsmith Resetting the admin password did not help. I get the same error 500 when logging in and the same Java runtime exception is logged to the server’s log. ---
**Author:** ![rstellma](https://avatars.discourse-cdn.com/v4/letter/r/6a8cbe/32.png) [@rstellma](https://community.graylog.org/u/rstellma)\ **Post date:** [December 14, 2022, 1:47pm UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/8 "2022-12-14T13:47:59Z")
Addendum: even using a newly created database produces the error. I created a new database in MongoDB (“graylog2”), stopped Graylog, modified the `mongodb_uri` string, started Graylog and tried to log in to the web UI. Same error 500, same entries in server.log. Graylog creates all the collections in Mongo database `graylog2`, so it does not seem to be a permission problem. We don’t have authentication enabled anyway. ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [December 14, 2022, 10:16pm UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/9 "2022-12-14T22:16:39Z")
Hello, Yeah the HTTP status code 500 is a generic error response meaning the server encountered an unexpected condition that prevented it from fulfilling the request. This I would look at Graylog and `Tail -f` since this is a front end issue, Graylog would be resposible for that (i.e. Mongo connection, Ports/Firewalls, file and directory persmissions, etc…) Showing your Graylog/Mongo configuration file would help or any logs when you `tail -f` Graylog log file after a restart would also help. Right now Im only guessing. EDIT: To help you further , here is my lab setup config. take note I use static ip address and FQDN in my DMZ > **GL-Config** > > ```auto > [root@graylog graylog]# cat /etc/graylog/server/server.conf | egrep -v "^\s*(#|$)" > is_master = true > node_id_file = /etc/graylog/server/node-id > password_secret = epOqmLi7r7CdZxl76QOQxr8b------ > root_password_sha2 = 5e884898da28047151d0e56f8dc6------ > root_email = "greg.smith@domain.com" > root_timezone = America/Chicago > bin_dir = /usr/share/graylog-server/bin > data_dir = /var/lib/graylog-server > plugin_dir = /usr/share/graylog-server/plugin > http_bind_address = graylog.enseva-labs.net:9000 > http_enable_cors = true > elasticsearch_hosts = http://8.8.8.8:9200 > rotation_strategy = count > elasticsearch_max_docs_per_index = 20000000 > elasticsearch_max_number_of_indices = 20 > retention_strategy = delete > elasticsearch_shards = 4 > elasticsearch_replicas = 0 > elasticsearch_index_prefix = graylog > allow_leading_wildcard_searches = true > allow_highlighting = false > elasticsearch_analyzer = standard > elasticsearch_index_optimization_timeout = 1h > output_batch_size = 5000 > output_flush_interval = 1 > output_fault_count_threshold = 5 > output_fault_penalty_seconds = 30 > processbuffer_processors = 7 > outputbuffer_processors = 3 > processor_wait_strategy = blocking > ring_size = 65536 > inputbuffer_ring_size = 65536 > inputbuffer_processors = 2 > inputbuffer_wait_strategy = blocking > message_journal_enabled = true > message_journal_dir = /var/lib/graylog-server/journal > message_journal_max_size = 12gb > lb_recognition_period_seconds = 3 > mongodb_uri = mongodb://mongo_admin:password@localhost:27017/graylog > mongodb_max_connections = 1000 > mongodb_threads_allowed_to_block_multiplier = 5 > transport_email_enabled = true > transport_email_hostname = localhost > tansport_email_port = 25 > transport_email_subject_prefix = [graylog] > transport_email_from_email = root@domain.com > transport_email_web_interface_url = https://8.8.8.8:9000 > http_connect_timeout = 10s > proxied_requests_thread_pool_size = 32 > prometheus_exporter_enabled =true > [root@graylog graylog]# > > ``` > **Mongo-Config** > > ```auto > [root@graylog graylog]# cat /etc/mongod.conf | egrep -v "^\s*(#|$)" > systemLog: > destination: file > logAppend: true > path: /var/log/mongodb/mongod.log > storage: > dbPath: /var/lib/mongo > journal: > enabled: true > processManagement: > fork: true # fork and run in background > pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile > net: > port: 27017 > bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces. > security: > authorization: enabled > operationProfiling: > slowOpThresholdMs: 5 > mode: all > [root@graylog graylog]# > > ``` ---
**Author:** ![rstellma](https://avatars.discourse-cdn.com/v4/letter/r/6a8cbe/32.png) [@rstellma](https://community.graylog.org/u/rstellma)\ **Post date:** [December 16, 2022, 8:16am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/10 "2022-12-16T08:16:31Z")
Hello @gsmith > **GL-config** > > is\_master = true > node\_id\_file = /etc/graylog/server/node-id > password\_secret = > root\_username = admin > root\_password\_sha2 = > bin\_dir = /usr/share/graylog-server/bin > data\_dir = /var/lib/graylog-server > plugin\_dir = /usr/share/graylog-server/plugin > http\_bind\_address = 0.0.0.0:9000 > rotation\_strategy = count > elasticsearch\_max\_docs\_per\_index = 20000000 > elasticsearch\_max\_number\_of\_indices = 4 > retention\_strategy = delete > elasticsearch\_shards = 4 > elasticsearch\_replicas = 0 > elasticsearch\_index\_prefix = graylog > allow\_leading\_wildcard\_searches = false > allow\_highlighting = false > elasticsearch\_analyzer = standard > output\_batch\_size = 500 > output\_flush\_interval = 1 > output\_fault\_count\_threshold = 5 > output\_fault\_penalty\_seconds = 30 > processbuffer\_processors = 5 > outputbuffer\_processors = 3 > processor\_wait\_strategy = blocking > ring\_size = 65536 > inputbuffer\_ring\_size = 65536 > inputbuffer\_processors = 2 > inputbuffer\_wait\_strategy = blocking > message\_journal\_enabled = true > message\_journal\_dir = /var/lib/graylog-server/journal > lb\_recognition\_period\_seconds = 3 > mongodb\_uri = mongodb://host1:27017,host2:27017,host3:27017/graylog?replicaSet=rs01 > mongodb\_max\_connections = 1000 > mongodb\_threads\_allowed\_to\_block\_multiplier = 5 > proxied\_requests\_thread\_pool\_size = 32 > **Mongo-Config** > > # THIS FILE IS MAINTAINED BY [https://api-agents.mongodb.com](https://api-agents.mongodb.com) . DO NOT MODIFY AS IT WILL BE OVERWRITTEN. > > # To make changes to your MongoDB deployment, please visit [https://cloud.mongodb.com](https://cloud.mongodb.com) . Your Group ID is . > > net: > bindIp: 0.0.0.0 > port: 27017 > processManagement: > fork: “true” > replication: > oplogSizeMB: 190000 > replSetName: rs01 > setParameter: > enableFlowControl: “false” > logicalSessionRefreshMillis: “1200000” > maxSessions: “2000000” > storage: > dbPath: /data > engine: wiredTiger > systemLog: > destination: file > logAppend: true > path: /data/mongodb.log `tail -f` on the server.log shows the same error, i posted in my initial post > **server.log** > > 2022-12-16T07:58:11.851Z ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource > java.lang.RuntimeException: Couldn’t load session > at org.graylog2.security.MongoDbSessionDAO.doUpdate(MongoDbSessionDAO.java:101) ~[graylog.jar:?] > at org.apache.shiro.session.mgt.eis.CachingSessionDAO.update(CachingSessionDAO.java:277) ~[graylog.jar:?] > at org.apache.shiro.session.mgt.DefaultSessionManager.onChange(DefaultSessionManager.java:212) ~[graylog.jar:?] > at org.apache.shiro.session.mgt.AbstractNativeSessionManager.applyGlobalSessionTimeout(AbstractNativeSessionManager.java:125) ~[graylog.jar:?] > at org.apache.shiro.session.mgt.AbstractNativeSessionManager.start(AbstractNativeSessionManager.java:100) ~[graylog.jar:?] > at org.apache.shiro.mgt.SessionsSecurityManager.start(SessionsSecurityManager.java:152) ~[graylog.jar:?] > at org.apache.shiro.subject.support.DelegatingSubject.getSession(DelegatingSubject.java:340) ~[graylog.jar:?] > at org.apache.shiro.subject.support.DelegatingSubject.getSession(DelegatingSubject.java:316) ~[graylog.jar:?] > at org.graylog2.shared.security.SessionCreator.create(SessionCreator.java:80) ~[graylog.jar:?] > at org.graylog2.rest.resources.system.SessionsResource.newSession(SessionsResource.java:146) ~[graylog.jar:?] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0\_352] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0\_352] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0\_352] > at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0\_352] > at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[graylog.jar:?] > at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[graylog.jar:?] > at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) [graylog.jar:?] > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] > at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] > at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] > at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] > at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] > at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?] > at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] > at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] > at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0\_352] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0\_352] > at java.lang.Thread.run(Thread.java:750) [?:1.8.0\_352] I found another MongoDB-related error in the server.log: [com-mongodb-basicdbobject-cannot-be-cast-to-b](https://community.graylog.org/t/com-mongodb-basicdbobject-cannot-be-cast-to-b/21234) Unfortunately, deleting the ‘session’ collection did not help. Creating a new empty database (“graylog2”) and using it in graylog did not help. either. The collections are created in the new database, but login to web UI still fails. ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [December 17, 2022, 1:48am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/11 "2022-12-17T01:48:18Z")
Hey, Couple things I see different then mine, please take note we do not bind any addresses `0.0.0.0` to our servers. This is all done by static IPaddres so Ip address will not change on any host and it easier routing to remote destinations. Observations: > [@rstellma](#): > > Creating a new empty database (“graylog2”) So you have a database called garylog2 , but you configuration file shows this… > [@rstellma](#): > > elasticsearch\_index\_prefix = graylog And this… > [@rstellma](#): > > mongodb\_uri = mongodb://host1:27017,host2:27017,host3:27017/graylog?replicaSet=rs01 Can I ask how did you go about change Graylog Default database? By default graylog should of created a database called “graylog”. Next, I assume the configuration on each mongoDb are unique. I have links to this documention below. Example of mine. ```auto [root@graylog tools]# cat /etc/mongod.conf | egrep -v "^\s*(#|$)" systemLog: destination: file logAppend: true path: /var/log/mongodb/mongod.log storage: dbPath: /var/lib/mongo journal: enabled: true processManagement: fork: true # fork and run in background pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile net: port: 27017 bindIp: 192.168.1.100 # Listen to local interface only, comment to listen on all interfaces. security: authorization: enabled replication: replSetName: "replica01" operationProfiling: slowOpThresholdMs: 5 mode: all [root@graylog tools]# ``` You Graylog configuration look good, I just never used a http\_bind 0.0.0.0 for the front end before its been static i.e., 192.168.1.100:9000 as shown above same with Mongodb configuration file/s. Next troubleshooting. I understand from the statment above you did not adjust the configuration file for mongDb but the upgrade/update that was executed prior did failed now there seams to be an issue with MongoDb , which inturn unable to login. Not sure if you done this ,but to view your replica set found [here](https://www.mongodb.com/docs/v4.4/tutorial/deploy-replica-set/#view-the-replica-set-configuration) I was looking at these commands. Run [`rs.initiate()`](https://www.mongodb.com/docs/v4.4/reference/method/rs.initiate/#mongodb-method-rs.initiate) on _just one and only one_ [`mongod`](https://www.mongodb.com/docs/v4.4/reference/program/mongod/#mongodb-binary-bin.mongod) instance for the replica set. ```auto rs.initiate( { _id : "rs0", members: [ { _id: 0, host: "mongodb0.example.net:27017" }, { _id: 1, host: "mongodb1.example.net:27017" }, { _id: 2, host: "mongodb2.example.net:27017" } ] }) ``` - Use rs.conf() to display the replica set configuration object: - Use rs.status() to identify the primary in the replica set. Also note, sometime editing Mongodb if its done manually, may create more issue. What I was referring to in the way of Graylog logs is a _stack-trace_. When graylog startes up it performs checks, and I was/wanted to see logs that stating about a connected to MongoDb. This would show us that Graylog was/is able to connect to the database. Trying to do elimination on this issue because it did work before but now it doesnt. It would indicated that something did change. Really not sure and since it seams you wiped Graylog database, just a suggestion for troubleshooting perhaps try 1 mongodb node and see if that works first while using maybe a static IP address instead of _0.0.0.0_. you can always add more nodes. EDIT: I also was looking at this documentations on HowTo with mongDb replica-set Connections [here](https://www.mongodb.com/docs/v4.4/tutorial/deploy-replica-set/#connectivity) ---
**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\ **Post date:** [December 31, 2022, 1:48am UTC](https://community.graylog.org/t/unable-to-log-into-web-ui-error-500/26704/12 "2022-12-31T01:48:32Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.