LDAP auth broken with recent patch? FIPS mode breaks openjdk crypto

Your LDAP must allow anonymous lookup.

Yes, in my lab our environment. Might want to check your LADP log files. Perhaps you can see Graylog trying to connect.

The original error you posted suggests a null field associated with system_user_password. For the User DN, you can try using the full username such as Auth-account@mylocaldomain.tld

Also double check that your enterprise plugins are updated - you can use this command to check (I think)

yum list installed | grep -E ".*(elasticsearch|graylog|mongo).*"

Full user name made no difference.

[root@graylog server]# dnf list installed | grep -E “.(elasticsearch|graylog|mongo).
elasticsearch-oss.x86_64 7.10.2-1 @elasticsearch-7.x
graylog-4.2-repository.noarch 1-4 @graylog
graylog-enterprise-integrations-plugins.noarch 4.2.3-1 @graylog
graylog-enterprise-plugins.noarch 4.2.3-1 @graylog
graylog-integrations-plugins.noarch 4.2.3-1 @graylog
graylog-server.noarch 4.2.3-1 @graylog
graylog-sidecar-repository.noarch 1-2 @System
mongodb-org.x86_64 4.2.17-1.el8 @mongodb-org-4.2
mongodb-org-mongos.x86_64 4.2.17-1.el8 @mongodb-org-4.2
mongodb-org-server.x86_64 4.2.17-1.el8 @mongodb-org-4.2
mongodb-org-shell.x86_64 4.2.17-1.el8 @mongodb-org-4.2
mongodb-org-tools.x86_64 4.2.17-1.el8 @mongodb-org-4.2

Rather than full user name, I should have put userPrincipalName

Test to see if your RHEL machine can see the LDAP server - maybe use ldapwhoami … Serverfault has a good example test command

I don’t use redhat, should I be suggesting dnf… rather than yum… ?

The pam/sssd login uses LDAP, so LDAP works for pam.

Here is the full log for a failed login:

2021-12-16T13:18:24.508-05:00 ERROR [AESTools] Could not encrypt value.
java.security.NoSuchProviderException: No such provider: SunJCE
	at javax.crypto.Cipher.getInstance(Cipher.java:596) ~[?:1.8.0_312]
	at org.graylog2.security.AESTools.encrypt(AESTools.java:57) ~[graylog.jar:?]
	at org.graylog2.security.encryption.EncryptedValueService.encrypt(EncryptedValueService.java:45) ~[graylog.jar:?]
	at org.graylog2.security.realm.UsernamePasswordRealm.doGetAuthenticationInfo(UsernamePasswordRealm.java:92) ~[graylog.jar:?]
	at org.graylog2.security.realm.UsernamePasswordRealm.doGetAuthenticationInfo(UsernamePasswordRealm.java:71) ~[graylog.jar:?]
	at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:571) ~[graylog.jar:?]
	at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doMultiRealmAuthentication(ModularRealmAuthenticator.java:225) ~[graylog.jar:?]
	at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:275) ~[graylog.jar:?]
	at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198) ~[graylog.jar:?]
	at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106) ~[graylog.jar:?]
	at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:275) ~[graylog.jar:?]
	at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:260) ~[graylog.jar:?]
	at org.graylog2.shared.security.SessionCreator.create(SessionCreator.java:82) ~[graylog.jar:?]
	at org.graylog2.rest.resources.system.SessionsResource.newSession(SessionsResource.java:142) ~[graylog.jar:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_312]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_312]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_312]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_312]
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) [graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) [graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) [graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) [graylog.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) [graylog.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) [graylog.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) [graylog.jar:?]
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?]
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?]
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?]

	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?]
	at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?]
	at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?]

	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_312]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_312]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
2021-12-16T13:18:24.509-05:00 ERROR [UsernamePasswordRealm] Unhandled authentication error
java.lang.NullPointerException: Null value
	at org.graylog2.security.encryption.AutoValue_EncryptedValue$Builder.value(AutoValue_EncryptedValue.java:96) ~[graylog.jar:?]
	at org.graylog2.security.encryption.EncryptedValueService.encrypt(EncryptedValueService.java:45) ~[graylog.jar:?]
	at org.graylog2.security.realm.UsernamePasswordRealm.doGetAuthenticationInfo(UsernamePasswordRealm.java:92) ~[graylog.jar:?]
	at org.graylog2.security.realm.UsernamePasswordRealm.doGetAuthenticationInfo(UsernamePasswordRealm.java:71) ~[graylog.jar:?]
	at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:571) ~[graylog.jar:?]
	at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doMultiRealmAuthentication(ModularRealmAuthenticator.java:225) ~[graylog.jar:?]
	at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:275) ~[graylog.jar:?]
	at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198) ~[graylog.jar:?]
	at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106) ~[graylog.jar:?]
	at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:275) ~[graylog.jar:?]
	at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:260) ~[graylog.jar:?]
	at org.graylog2.shared.security.SessionCreator.create(SessionCreator.java:82) ~[graylog.jar:?]
	at org.graylog2.rest.resources.system.SessionsResource.newSession(SessionsResource.java:142) ~[graylog.jar:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_312]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_312]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_312]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_312]
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) [graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) [graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) [graylog.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) [graylog.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) [graylog.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) [graylog.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) [graylog.jar:?]
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?]
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?]
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) [graylog.jar:?]
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?]
	at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?]
	at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_312]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_312]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
2021-12-16T13:18:24.510-05:00 INFO  [SessionCreator] Invalid credentials in session create request. Actor: "urn:graylog:user:joe_user"

Hmmm I have seen a few things on how password_secret in server.conf is used for salting passwords… is that something that might have changed in your test configuration or maybe wonked in translation?

No, the hash has not changed. Non-LDAP logins work just fine.

Hello

I found this. Different version looks like same issue.

I don’t have illegal key size, so not the same issue. Seems like the root cause is No such provider: SunJCE. I did more testing with a different AD account, same issue. Tried with TLS, same issue.

OK, so I’m a nitwit. We had an audit last week and enabled FIPS mode on all Linux systems. I’m REALLY confident this is what killed LDAP login. Probably SunJCE can’t function in that cryptographic mode. I have not verified, but I’m pretty sure that is the cause. Of course there is no reference to FIPS in Graylog documentation.

Gotcha,

Since Mongo holds all the metadata, have you check your ldap_settings in MongoDb?
By chance did JAVA get update also during your recent update?

Read my latest updated post

VERIFIED…

fips-mode-setup --disable
reboot

Fixed

Now what, if you need FIPS to be federally compliant? I guess possibly you need a paid java solution with FIPS support?

1 Like

:+1: Thanks for posting the resolution.

Here is a relevant reference on the topic. May take the Graylog devs to implement something, or it may require disabling FIPS in java if FIPS is enabled in the OS.

I’m actually curious about this situation. Correct me if I’m wrong, but did you have FIPS enable on the OS level then enabled it also on the software level? or was this just enable on the software level then it fail with LDAP?

Enabled FIPS at boot time with a kernel arg. Read the RHEL document, it seems OpenJDK picks up the fips boot mode and acts accordingly. This broke the Graylog LDAP. It seems you can force OpenJDK not to follow the boot env and disable it. But that sort of defeats the purpose.

This broke my Gitlab setup as well as it turns out. It also uses java. :face_vomiting:

1 Like

Thank you for your reply, This is good to know.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.