Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!
1. Describe your incident:
While tracking down an apparent memory leak in graylog, I found a lot of these errors in the logs:
2025-03-31 09:13:24,752 WARN o.g.g.n.t.TCPNIOTransport [grizzly-nio-kernel(1) SelectorRunner] GRIZZLY0004: Can not set TCP_NODELAY to true
java.net.SocketException: Connection reset by peer
at java.base/sun.nio.ch.Net.setIntOption0(Native Method) ~[?:?]
at java.base/sun.nio.ch.Net.setSocketOption(Net.java:455) ~[?:?]
at java.base/sun.nio.ch.Net.setSocketOption(Net.java:393) ~[?:?]
at java.base/sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:280) ~[?:?]
at java.base/sun.nio.ch.SocketAdaptor.setBooleanOption(SocketAdaptor.java:237) ~[?:?]
at java.base/sun.nio.ch.SocketAdaptor.setTcpNoDelay(SocketAdaptor.java:273) ~[?:?]
at org.glassfish.grizzly.nio.transport.TCPNIOTransport$DefaultChannelConfigurator.postConfigure(TCPNIOTransport.java:756) [graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOServerConnection.configureAcceptedChannel(TCPNIOServerConnection.java:156) [graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOServerConnection.onAccept(TCPNIOServerConnection.java:211) [graylog.jar:?]
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:485) [graylog.jar:?]
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82) [graylog.jar:?]
at org.glassfish.grizzly.strategies.SameThreadIOStrategy.executeIoEvent(SameThreadIOStrategy.java:69) [graylog.jar:?]
at org.glassfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:66) [graylog.jar:?]
at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:381) [graylog.jar:?]
at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:353) [graylog.jar:?]
at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:319) [graylog.jar:?]
at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:248) [graylog.jar:?]
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535) [graylog.jar:?]
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515) [graylog.jar:?]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
How can I track down which TCP connections are causing the errors?
I have three inputs enabled
- syslog (UDP)
- Filebeat (TCP)
- Netflow (UDP)
Since it’s TCP errors, it’s either Filebeat or the connections to OpenSearch.
I’m not very concerned about not being able to set TCP_NODELAY (although I would like to silence the warnings), but connection reset by peer isn’t good.
2. Describe your environment:
- OS Information:
FreeBSD 14.2-RELEASE
- Package Version:
6.1.8
- Service logs, configurations, and environment variables:
3. What steps have you already taken to try and solve the problem?
4. How can the community help?
I’d like help to track down which connections are causing the errors, and a way to silence the TCP_NODELAY warnings (or convince me I need it enabled in the OS).
Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]