LdapNetworkConnection Exception

[nnelson]

I keep getting this exception over and over in my Graylog log file. It happens quite often and makes it hard to filter though the logs to find real problems.

2020-05-11T10:27:02.985-04:00 WARN [LdapNetworkConnection] null
java.lang.NullPointerException: null
at org.apache.directory.ldap.client.api.LdapNetworkConnection.messageReceived(LdapNetworkConnection.java:2040) ~[graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:997) ~[graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:641) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1114) [graylog.jar:?]
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:236) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:641) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1114) [graylog.jar:?]
at org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:326) [graylog.jar:?]
at org.apache.mina.filter.ssl.SslFilter.filterClose(SslFilter.java:712) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterClose(DefaultIoFilterChain.java:767) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1600(DefaultIoFilterChain.java:48) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterClose(DefaultIoFilterChain.java:1141) [graylog.jar:?]
at org.apache.mina.core.filterchain.IoFilterAdapter.filterClose(IoFilterAdapter.java:145) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterClose(DefaultIoFilterChain.java:767) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1600(DefaultIoFilterChain.java:48) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterClose(DefaultIoFilterChain.java:1141) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.filterClose(DefaultIoFilterChain.java:1025) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterClose(DefaultIoFilterChain.java:767) [graylog.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireFilterClose(DefaultIoFilterChain.java:760) [graylog.jar:?]
at org.apache.mina.core.session.AbstractIoSession.closeNow(AbstractIoSession.java:353) [graylog.jar:?]
at org.apache.directory.ldap.client.api.LdapNetworkConnection.close(LdapNetworkConnection.java:883) [graylog.jar:?]
at org.graylog2.security.realm.LdapUserAuthenticator.$closeResource(LdapUserAuthenticator.java:133) [graylog.jar:?]
at org.graylog2.security.realm.LdapUserAuthenticator.doGetAuthenticationInfo(LdapUserAuthenticator.java:133) [graylog.jar:?]
at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:571) [graylog.jar:?]
at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doMultiRealmAuthentication(ModularRealmAuthenticator.java:219) [graylog.jar:?]
at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:269) [graylog.jar:?]
at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198) [graylog.jar:?]
at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106) [graylog.jar:?]
at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:274) [graylog.jar:?]
at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:260) [graylog.jar:?]
at org.graylog2.shared.security.ShiroSecurityContext.loginSubject(ShiroSecurityContext.java:107) [graylog.jar:?]
at org.graylog2.shared.security.ShiroAuthenticationFilter.filter(ShiroAuthenticationFilter.java:48) [graylog.jar:?]
at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:132) [graylog.jar:?]
at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:68) [graylog.jar:?]
at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:318) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]

I have found a couple of other posts where at least one user was able to determine that there was a bad LDAP search filter; however, I don’t know that is the case in my instance. I have ran ldapquery from the command line and it returned expected results. Additionally, when I go to map the LDAP groups to roles in Graylog it only returns the single LDAP group “Graylog Admins” as expected.

My relevant (I think) settings are as follows:

LDAP Settings:
Server Type: Active Directory
Server Address: ldap://dc1.mydomain.com:389 (StartTLS checked)
Search Base DN: dc=mydomain,dc=com
User Search Pattern: (&(objectClass=user)(sAMAccountName={0}))
Display Name Attribute: displayName
Group Search Base DN: dc=mydomain,dc=com
Group Search Pattern: (&(objectClass=group)(cn=Graylog*))
Group Name Attribute: cn
Default User Role: Reader
Additional Default Roles: Allow Searches (this is a custom role I created to allow non-admins to search all streams).

LDAP Group Mapping:
Graylog Admins: Admin

For what it’s worth, “Test Server Connection” and “Test login” both work. Also, all users seem to have no issues with actual product functionality (both admins and non-admins).

I am running on Graylog 3.2.4, though, this problem has been around as long as I can remember.

[nnelson]

OK so I can’t believe that 15 minutes after posting to the help forum I think I was able to fix the issue myself. In an effort to help future users that may encounter this issue, here are my troubleshooting steps and ultimately my fix.

I disabled StartTLS and the error went away. Encryption is a must in our environment so this wasn’t a good long term solution; however, it let me see another error that I either didn’t get before or didn’t see before:

2020-05-11T10:56:06.201-04:00 WARN [LdapConnector] Unable to iterate over user’s groups, unable to perform group mapping. Graylog does not support LDAP referrals at the moment. Please see http://docs.graylog.org/en/3.2/pages/users_and_roles/external_auth.html#troubleshooting for more information.

I visited the link from the warning and it basically said you have two options to fix this. The first is to not use LDAP group mappings and manage them locally in Graylog (not ideal because we love groups for granting access). The second was to use global catalog, which was the option I picked.

I changed the LDAP port to 3269/TCP and checked the “SSL” box. After testing and then saving the LDAP settings, I no longer get the error message in my logs.

I hope this helps anyone that encounters this in the future.