C – System calls on Linux without stacking

On Linux i386, int $0x80 system call ABI makes it easy to execute system calls without a valid userspace stack. The VDSO/VSSsysCall interface, on the other hand, requires access to the stack. How do other Linux ports perform in this regard, especially x86_64? Do they have a way to make system calls without a stack? Are there references to available system call methods for each schema?

In general: I don’t know. Even on i386, if there is a 6th parameter, it must be passed on the stack (e.g. mmap).

Specifically for x86_64: Put the system call number into %rax (note: the system call number is assigned completely differently than 32 bits), up to 6 parameters in %rdi<, %rsi, %rdx, %r10, %r8, and %r9 (which is almost but not exactly the same as the usual ABI for passing parameters in registers – note the use of %r10 instead of %rcx), and the use of the syscall instruction. The results are returned in %rax and %rcx and %r11 are corrupted.

x86_64 ABI information can be >http://www.x86-64.org/documentation/abi.pdf Found it; The Linux ABI is documented in the appendix. (If you’re looking elsewhere for x86_64 ABI information, note that 64-bit Windows uses its own different ABI.) )

I don’t think syscall has any requirements for the user stack framework to work properly. In the case of being interrupted by a signal, the handler obviously needs a sound stack; But the experiment below, which uses an alternate signal stack and deliberately drops %rsp around syscall, works fine for me:

$ cat syscall_sig.c
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <signal.h>
#include <time.h>
#include <unistd.h>

#define __NR_nanosleep 35

static sig_atomic_t alrm = 0;

void handler(int sig)
{
    if (sig == SIGALRM)
        alrm = 1;
}

int main(void)
{
    stack_t ss;
    struct sigaction sa;
    struct timespec req, rem;
    long ret;

ss.ss_flags = 0;
    ss.ss_size = SIGSTKSZ;
    ss.ss_sp = malloc(ss.ss_size);
    sigaltstack(&ss, NULL);

memset(&sa, 0, sizeof(sa));
    sa.sa_handler = handler;
    sa.sa_flags = SA_ONSTACK;
    sigaction(SIGALRM, &sa, NULL);

alarm(1);

req.tv_sec = 5;
    req.tv_nsec = 0;
    asm("xorq $0x12345678, %%rsp ; syscall ; xorq $0x12345678, %%rsp"
        : "=a" (ret)
        : "0" (__NR_nanosleep), "D" (&req), "S" (&rem)
        : "rcx", "r11", "memory");

printf("syscall return code %ld, alarm flag %d\n", ret, alrm);

return 0;
}

$ gcc -Wall -o syscall_sig syscall_sig.c
$ ./syscall_sig
syscall return code -4, alarm flag 1
$