GELF TCP Input Suddenly Broken

A couple of weeks ago my Graylog GELF TCP Input suddenly stopped working. I have tried several suggestions I have found on postings of similar issues with no luck, as well as a few other things. All other inputs on the server are working fine, and the HTTPS is valid and cert confirmed OK. This seemed to happen out of the blue, with no manual updates recently performed and not close to certificate expiration.

Things that have been tried:

  • Creating and implementing a brand new certificate & private key from scratch
  • Rolling back the last few Graylog patches
  • Downgrading Java OpenJDK
  • Adding enabled_tls_protocols = TLSv1.2 to the server config file


2021-07-07T08:53:22.762-04:00 WARN  [ChannelInitializer] Failed to initialize a channel. Closing: [id: 0x4631075c, L:/SERVER:12202 - R:/CLIENT:50608]
org.bouncycastle.pkcs.PKCSException: Encountered unexpected object type: org.bouncycastle.cert.X509CertificateHolder
        at org.graylog2.plugin.inputs.transports.util.KeyUtil.privateKeyFromFile( ~[graylog.jar:?]
        at org.graylog2.plugin.inputs.transports.AbstractTcpTransport$3.createSslEngine( ~[graylog.jar:?]
        at org.graylog2.plugin.inputs.transports.AbstractTcpTransport$ ~[graylog.jar:?]
        at org.graylog2.plugin.inputs.transports.AbstractTcpTransport$ ~[graylog.jar:?]
        at org.graylog2.plugin.inputs.transports.NettyTransport$1.initChannel( ~[graylog.jar:?]
        at [graylog.jar:?]
        at [graylog.jar:?]
        at [graylog.jar:?]
        at [graylog.jar:?]
        at$100( [graylog.jar:?]
        at$PendingHandlerAddedTask.execute( [graylog.jar:?]
        at [graylog.jar:?]
        at [graylog.jar:?]
        at$AbstractUnsafe.register0( [graylog.jar:?]
        at$AbstractUnsafe.access$200( [graylog.jar:?]
        at$AbstractUnsafe$ [graylog.jar:?]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute( [graylog.jar:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks( [graylog.jar:?]
        at [graylog.jar:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor$ [graylog.jar:?]
        at io.netty.util.internal.ThreadExecutorMap$ [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$ [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker( [?:1.8.0_292]
        at java.util.concurrent.ThreadPoolExecutor$ [?:1.8.0_292]
        at com.codahale.metrics.InstrumentedThreadFactory$ [graylog.jar:?]
        at [?:1.8.0_292]

Input config (note that the cert & private key are not encrypted therefore not needing a tls key password):


OS: Debian 9
Graylog Version: 4.1.0
Java: OpenJDK 1.8.0_292


It seams that you tried a lot a different things to resolve your issue. When you stated

Are you refering to the INPUT is in a stopped state? or it stopped receiving message? If so have you tried using a different port then 12202?

You using the same certificates for your INPUTs as for your HTTPS?

1 Like

I have tried a new input with no luck. Yes, it is the same as the HTTPS.

Long story short,
Graylog did not like that there was a cert with a 4year expiration. When I remade the cert following my instructions, missed the step converting the private key from pkcs5 to pkcs8…

1 Like

:laughing: Yeah that would do it. Glad you found the issue.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.