C – ioctl fails when attempting to get more than 8 queue file descriptors from the tun interface

ioctl fails when attempting to get more than 8 queue file descriptors from the tun interface… here is a solution to the problem.

ioctl fails when attempting to get more than 8 queue file descriptors from the tun interface

After I heard that kernel 3.8 linux added multi-queue functionality to tune tap devices with flag IFF_MULTI_QUEUE, I upgraded the kernel to 3.10 and put its header in /usr/src
Then I changed my C code to have a thread open a new queue file descriptor every time it is needed.
But the thread could only open 8 queues (with cores earlier than 3.8, it couldn’t open queues at all), after which I got this “parameter too long” error from ioctl

ioctl(fd, TUNSETIFF, (void *)&ifr)

Then I wrote another program to test how many queue fd could be opened in my ubuntu 12.10 linux box

uname -r
3.10.0-031000-generic 

A simpler version of the kernel in a program.

//essential
#include <net/if.h>
#include <linux/if_tun.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <arpa/inet.h>
//
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/socket.h> 
#include <sys/select.h>
#include <sys/time.h>
#include <errno.h>
#include <stdarg.h>

#include <netdb.h>

#define IFF_MULTI_QUEUE 0x0100
int tun_alloc_mq(char *dev, int queues, int *fds)
{
    struct ifreq ifr;
    int fd, err=-1, i;
    char *clonedev = "/dev/net/tun";

if (!dev){
        printf("dev");
        return -1;
    }

memset(&ifr, 0, sizeof(ifr));
    ifr.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_MULTI_QUEUE;
    strcpy(ifr.ifr_name, dev);
    int error=0;

for (i = 0; i < queues; i++) {

printf("loop %d\n",i);

if( (fd = open(clonedev , O_RDWR)) < 0 ) {
            perror("Opening /dev/net/tun");
            error=1;
            break;
          }

if(ioctl(fd, TUNSETIFF, (void *)&ifr) < 0 ) {
            printf("first error\n");
            error=1;
            close(fd);
            break;
          } 
    }
    if(error==1)
        return -1;
    return 0;
}
int main(int argc, char *argv[])
{

int* fdsx;
    if(tun_alloc_mq("testmqtun0",20,fdsx)<0)
    {
        perror("tun");
        exit(1);
    }
    return 0;
}

It turns out that this is also limited to 8. Here is the output:

loop 0
loop 1
loop 2
loop 3
loop 4
loop 5
loop 6
loop 7
loop 8
first error
tun: Argument list too long

I tested it on another linux machine and it has the same output.
So is there a limit to opening more than 8 queues from tun devices in the kernel?
If so, how to fix it?
Please help me with this

Solution

If you want to get file descriptors for more than 8 queues from a tun interface, you can upgrade your Linux kernel to 4.0+, which supports 256 queues.

Related Problems and Solutions