# UDP buffer size problem despite system configuration

**URL:** <https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726>\
**Category:** Graylog Central (peer support)\
**Created:** [June 14, 2024, 11:49am UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726 "2024-06-14T11:49:56Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![borjam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/borjam/32/11022_2.png) [@borjam](https://community.graylog.org/u/borjam)\
**Post date:** [June 14, 2024, 11:49am UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726/1 "2024-06-14T11:49:56Z")

</div>

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.  
**Don’t forget to select tags to help index your topic!**

**1. Describe your incident:**  
Despite system configuration Graylog is unable to set a UDP buffer size bigger than 65536.

The system is configured for a UDP buffer size of 1048576 bytes.

Checked with a silly C program that does a socket(), setsockopt(), getsockopt().

```auto
% sysctl net.inet.udp.recvspace
net.inet.udp.recvspace: 1048576

```

```auto
./checkbuf 262144
SO_RCVBUF: requested: 262144, actual: 262144

```

```auto
 ./checkbuf 1048576
SO_RCVBUF: requested: 1048576, actual: 1048576

```

Fails as expected then exceeding UDP buffer size.  
.

```auto
/checkbuf 2048576
setsockopt(): No buffer space available

```

I am not sure whether this is a limitation on OpenJDK 17, OpenJDK 17 on FreeBSD, netty or Graylog.

**2. Describe your environment:**

- OS Information: FreeBSD 14.1 (amd64)

- Package Version: Graylog 6.0.1

- Service logs, configurations, and environment variables:  
2024-06-14 10:50:21,533 WARN o.g.i.t.UdpTransport [netty-transport-0] receiveBufferSize (SO\_RCVBUF) for input SyslogUDPInput{title=DNS syslog, type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=XXXXXXX} (channel [id: 0x1bc943a9, L:/10.0.0.1:1312]) should be \>= 1048576 but is 65536.

**3. What steps have you already taken to try and solve the problem?**  
I have made sure the system is properly configured.

**4. How can the community help?**

If you are seeing these warnings on the logfile, check that your buffer size is properly configured using the attached program.

---- Check program, checkbuf.c

```auto
/*
 * checkbuf.c
 * Verifies UDP buffer size
 */

#include <stdio.h>
#include <stdlib.h>
#include <stddef.h>
#include <sys/types.h>
#include <sys/socket.h>

int main(int argc, char **argv)
{
    int s, r;
    unsigned int bufsiz, actual_bufsiz, len;

    if ( argc < 2 ) {
        fprintf(stderr, "Uso: %s buffer\n", argv[0]);
        exit(1);
    }
    bufsiz = atoi(argv[1]);
    
    s = socket(PF_INET, SOCK_DGRAM, 0);
    if ( s < 0 ) {
        perror("socket()");
        exit(1);
    }
    r = setsockopt(s, SOL_SOCKET, SO_RCVBUF, &bufsiz, sizeof(bufsiz));
    if ( r < 0 ) {
        perror("setsockopt()");
        exit(1);
    }
    /* Now verify */
    len = sizeof(actual_bufsiz);
    r = getsockopt(s, SOL_SOCKET, SO_RCVBUF, &actual_bufsiz, &len);
    if ( r < 0 ) {
        perror("getsockopt()");
        exit(1);
    }
    printf("SO_RCVBUF: requested: %d, actual: %d\n", bufsiz, actual_bufsiz);
    exit(0);
}

```

---

<div class="post-metadata">

**Author:** ![borjam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/borjam/32/11022_2.png) [@borjam](https://community.graylog.org/u/borjam)\
**Post date:** [June 14, 2024, 3:19pm UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726/2 "2024-06-14T15:19:38Z")

</div>

There is a workaround to prevent this issue:

When setting the buffer size to 0 on graylog.conf and not setting a buffer size for the input configuration, the setsockopt() system call fails and the socket keeps the default buffer size.

1048576 is the buffer size configured on my system.

```auto
Proto Recv-Q Send-Q Local Address Foreign Address R-HIWA S-HIWA R-LOWA S-LOWA R-BCNT S-BCNT R-BMAX S-BMAX rexmt persist keep 2msl delack rcvtime
udp4 0 0 10.0.0.1.515 *.* **1048576** 9216 1 2048 0 0 2097152 73728

```

---

<div class="post-metadata">

**Author:** ![drewmiranda-gl](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/drewmiranda-gl/32/13376_2.png) [@drewmiranda-gl](https://community.graylog.org/u/drewmiranda-gl)\
**Post date:** [June 14, 2024, 4:55pm UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726/3 "2024-06-14T16:55:40Z")

</div>

I believe this behavior has always existed. I’m out of my depth understanding the specifics but does seem to be related to how the operating system is configured.

I did find [receiveBufferSize (SO\_RCVBUF) should be 262144 but is 212992. · Issue #1350 · Graylog2/graylog2-server · GitHub](https://github.com/Graylog2/graylog2-server/issues/1350) which says that the following command resolves the error/warning:

```auto
sysctl -w net.core.rmem_max=1048576

```

I just tested this in my lab where i was receiving:

```auto
[AbstractTcpTransport] receiveBufferSize (SO_RCVBUF) for input PaloAlto9xInput{title=Log Replay - Palo 9+, type=org.graylog.integrations.inputs.paloalto9.PaloAlto9xInput, nodeId=null} (channel [id: 0x88f847b7, L:/[0:0:0:0:0:0:0:0%0]:2516]) should be >= 1048576 but is 425984.
[AbstractTcpTransport] receiveBufferSize (SO_RCVBUF) for input SyslogTCPInput{title=Log Replay - Syslog TCP, type=org.graylog2.inputs.syslog.tcp.SyslogTCPInput, nodeId=null} (channel [id: 0x760a0c8a, L:/[0:0:0:0:0:0:0:0%0]:2515]) should be >= 1048576 but is 425984.

```

And after running the above command and restarting graylog-server i no longer receive the warnings.

---

<div class="post-metadata">

**Author:** ![borjam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/borjam/32/11022_2.png) [@borjam](https://community.graylog.org/u/borjam)\
**Post date:** [June 15, 2024, 7:51am UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726/4 "2024-06-15T07:51:16Z")

</div>

There seems to be some inconsistency/limitation on how buffer size is treated.

Maybe it depends on the operating system. You are mentioning Linux, I am mentioning FreeBSD.

On FreeBSD I have the right UDP buffer size configuration. However, if I specify a buffer size (either on graylog.conf or an UDP input definition) it is capped to 65536.

However, if I specify 0 for both (configuration file and input definition) the default buffer size (as configured on the OS) is used because Graylog tries a setsockopt() with a buffer size of 0, which fails but does not invalidate the socket 🙂

I think I will open a Github issue.

If you want to check whether your problem is due to Graylog or the operating system try compiling and running the small program I have pasted here.

---

<div class="post-metadata">

**Author:** ![borjam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/borjam/32/11022_2.png) [@borjam](https://community.graylog.org/u/borjam)\
**Post date:** [June 26, 2024, 8:22am UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726/5 "2024-06-26T08:22:48Z")

</div>

Just for the record in case someone lands here, there is a workaround on FreeBSD.

FIrst, you need to set up the default UDP buffer size with:

(Example for 1 MB, 1048576 bytes)  
`# sysctl net.inet.udp.recvsize=1048576`

Second, make sure to **both** :

- Set `udp_recvbuffer_sizes = 0` on graylog.conf
- Leave the input buffer sizes empty for the UDP inputs on Graylog

You can verify that the desired buffer size is used with:

```auto
% sysctl net.inet.udp.recvspace
net.inet.udp.recvspace: 524288

% netstat -p udp -x -n -a
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address R-HIWA S-HIWA R-LOWA S-LOWA R-BCNT S-BCNT R-BMAX S-BMAX rexmt persist keep 2msl delack rcvtime
udp4 0 0 *.53917 *.* 524288 9216 1 2048 0 0 2097152 73728
udp4 0 0 *.17230 *.* 524288 9216 1 2048 0 0 2097152 73728
udp4 0 0 *.18359 *.* 524288 9216 1 2048 0 0 2097152 73728
udp4 0 0 *.50582 *.* 524288 9216 1 2048 0 0 2097152 73728
udp4 0 0 *.50040 *.* 524288 9216 1 2048 0 0 2097152 73728
udp4 0 0 *.27975 *.* 524288 9216 1 2048 0 0 2097152 73728

```

The R-HIWA shows the actual receiver buffer size for the UDP sockets. The value should match `net.inet.udp.recvspace`.

Editing graylog.conf requires a Graylog restart. When you set that bogus value on `graylog.conf` the setsockopt(SO\_RCVBUF) call will fail but Graylog will issue a warning and move forward, effectively leaving the default buffer size intact.

After graylog.conf is properly configured you can adjust the UDP buffer size by changing `net.inet.udp.recvspace` and as long as the input buffer size field is empty restarting the input will adjust the buffer size.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [July 10, 2024, 8:23am UTC](https://community.graylog.org/t/udp-buffer-size-problem-despite-system-configuration/32726/6 "2024-07-10T08:23:21Z")

</div>

Closed on July 10, 2024, 8:23am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.