What are the units of UDP buffers and where is the documentation for the sysctl parameter?
I’m running x86_64 RedHat 5.3 (kernel 2.6.18) and specifically looking at the net.core.rmem_max from sysctl -a in the context of trying to set the UDP buffer. The receiver application sometimes loses packets, but I think the buffer is large enough depending on what it means :
What is the unit of this setting – bits, bytes, packets, or pages? If it’s bits or bytes, is it from the datagram/payload (e.g. 100 bytes) or the network MTU size (~1500 bytes)? If it’s a page, how many bytes is the page size?
Is this the maximum per system, per physical device (NIC), per virtual device (VLAN), per process, per thread, per socket/per multicast group?
For example, let’s say my data is 100 bytes per message, each network packet contains 2 messages, I want each socket to buffer 50,000 messages, and I open 3 sockets for each thread on each of the 4 threads. net.core.rmem_max How big should it be? Similarly, when I set the socket option in my application, the unit payload bytes, in this case the unit is 5000000 on each socket?
Finally, in general, how do I find the details of the parameter units I see with sysctl -a? I have similar units and X questions about other parameters such as net.core.netdev_max_backlog and net.ipv4.igmp_max_memberships.
Thank you.
Solution
You’ll look at these documents. That said, many of these parameters are really poorly documented, so expect a Google search to dig up bloody details from blogs and mailing lists.
rmem_max is the maximum buffer, in bytes, per socket. On closer inspection, this seems to be the memory that receives the entire packet, so the size must also include the size of the whatever/ip/udp header – although this area is vague to me.
Keep in mind that UDP is not reliable. There are a lot of sources of loss, especially between switches and routers – they also have buffers.