ioctl fails when attempting to get more than 8 queue file descriptors from the tun interface
After I heard that kernel 3.8 linux added multi-queue functionality to tune tap devices with flag IFF_MULTI_QUEUE, I upgraded the kernel to 3.10 and put its header in /usr/src
Then I changed my C code to have a thread open a new queue file descriptor every time it is needed.
But the thread could only open 8 queues (with cores earlier than 3.8, it couldn’t open queues at all), after which I got this “parameter too long” error from ioctl
ioctl(fd, TUNSETIFF, (void *)&ifr)
Then I wrote another program to test how many queue fd could be opened in my ubuntu 12.10 linux box
uname -r
3.10.0-031000-generic
A simpler version of the kernel in a program.
//essential
#include <net/if.h>
#include <linux/if_tun.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <arpa/inet.h>
//
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/select.h>
#include <sys/time.h>
#include <errno.h>
#include <stdarg.h>
#include <netdb.h>
#define IFF_MULTI_QUEUE 0x0100
int tun_alloc_mq(char *dev, int queues, int *fds)
{
struct ifreq ifr;
int fd, err=-1, i;
char *clonedev = "/dev/net/tun";
if (!dev){
printf("dev");
return -1;
}
memset(&ifr, 0, sizeof(ifr));
ifr.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_MULTI_QUEUE;
strcpy(ifr.ifr_name, dev);
int error=0;
for (i = 0; i < queues; i++) {
printf("loop %d\n",i);
if( (fd = open(clonedev , O_RDWR)) < 0 ) {
perror("Opening /dev/net/tun");
error=1;
break;
}
if(ioctl(fd, TUNSETIFF, (void *)&ifr) < 0 ) {
printf("first error\n");
error=1;
close(fd);
break;
}
}
if(error==1)
return -1;
return 0;
}
int main(int argc, char *argv[])
{
int* fdsx;
if(tun_alloc_mq("testmqtun0",20,fdsx)<0)
{
perror("tun");
exit(1);
}
return 0;
}
It turns out that this is also limited to 8. Here is the output:
loop 0
loop 1
loop 2
loop 3
loop 4
loop 5
loop 6
loop 7
loop 8
first error
tun: Argument list too long
I tested it on another linux machine and it has the same output.
So is there a limit to opening more than 8 queues from tun devices in the kernel?
If so, how to fix it?
Please help me with this
Solution
If you want to get file descriptors for more than 8 queues from a tun interface, you can upgrade your Linux kernel to 4.0+, which supports 256 queues.