Linux System Calls

The interface between an application program and the Operating System is through system calls.

What is a system call?

The operating system is responsible for

Process Management (starting, running, stopping processes)
File Management(creating, opening, closing, reading, writing, renaming files)
Memory Management (allocating, deallocating memory)
Other stuff (timing, scheduling, network management)

An application program makes a system call to get the operating system to perform a service for it, like reading from a file.

One nice thing about syscalls is that you don't have to link with a C library, so your executables can be much smaller.

System Calls in 32-bit Linux

This program makes two system calls: one to write the message "Hello, World" to standard output, and one to terminate the process. Note a process can ask for itself to be terminated, but it doesn't ask for itself to be created. How could it?

hello.s

# -----------------------------------------------------------------------------
# A 32-bit Linux standalone program that writes "Hello, World" to the console
# using system calls only.  The program does not need to link with any external
# libraries at all.
#
# System calls used:
#   4: write(fileid, bufferAddress, numberOfBytes)
#   1: exit(returnCode)
#
# Assemble:
#     gcc -c hello.s
# Link:
#     ld hello.o (to produce a.out)
#     (or) ld -o hello hello.o (to produce hello)
#
# Or, you can assemble and link in one step:
#     gcc -nostdlib hello.s
#
# The symbol _start is the default entry point for ld.
# -----------------------------------------------------------------------------

	.global	_start

	.text
_start:
	# write(1, message, 13)
	mov 	$4, %eax		# system call 4 is write
	mov 	$1, %ebx		# file handle 1 is stdout
	mov 	$message, %ecx		# address of string to output
	mov 	$13, %edx		# number of bytes to write
	int	$0x80			# invoke operating system code
	
	# exit(0)
	mov 	$1, %eax		# system call 1 is exit
	xor	%ebx, %ebx		# we want return code 0
	int	$0x80                   # invoke operating system code
message:
	.ascii	"Hello, World\n"

To make a system call in 32-bit Linux, place the system call number in eax, then its arguments, in order, in ebx, ecx, edx, esi, edi, and ebp, then invoke int 80h.
Some system calls return information, usually in eax.
All registers are saved across the system call.

System Calls in 64-bit Linux

hello.s

# -----------------------------------------------------------------------------
# A 64-bit Linux standalone program that writes "Hello, World" to the console
# using system calls only.  The program does not need to link with any external
# libraries at all.
#
# System calls used:
#   1: write(fileid, bufferAddress, numberOfBytes)
#   60: exit(returnCode)
#
# Assemble:
#     gcc -c hello.s
# Link:
#     ld hello.o (to produce a.out)
#     (or) ld -o hello hello.o (to produce hello)
#
# Or, you can assemble and link in one step:
#     gcc -nostdlib hello.s
#
# The symbol _start is the default entry point for ld.
# -----------------------------------------------------------------------------

        .global _start

        .text
_start:
        # write(1, message, 13)
        mov     $1, %rax                # system call 1 is write
        mov     $1, %rdi                # file handle 1 is stdout
        mov     $message, %rsi          # address of string to output
        mov     $13, %rdx               # number of bytes
        syscall                         # invoke operating system to do the write

        # exit(0)
        mov     $60, %rax               # system call 60 is exit
        xor     %rdi, %rdi              # we want return code 0
        syscall				# invoke operating system to exit
message:
        .ascii  "Hello, World\n"

To make a system call in 64-bit Linux, place the system call number in rax, then its arguments, in order, in rdi, rsi, rdx, r10, r8, and r9, then invoke syscall.
Some system calls return information, usually in rax. A value in the range between -4095 and -1 indicates an error, it is -errno.
The system call destroys rcx and r11 but others registers are saved across the system call.
Full details are in Section A.2.1 of the The AMD64 ABI.

Lists of Linux System Calls

There are hundreds of system calls in Linux. A good online source for 32-bit Linux is http://syscalls.kernelgrok.com/ . For 64-bit Linux, check out http://www.acsu.buffalo.edu/~charngda/linux_syscalls_64bit.html

Check those pages, and of course, the Linux source. To get a feel of what system calls look like, I've listed a tiny few of the calls here, specified for 32-bit Linux:

exit - terminate current process
In	eax	1
In	ebx	return code
Out	(This call does not return)
fork - create child process
In	eax	2
Out	eax	0 in the clone; process id of clone or EAGAIN or ENOMEM in caller
read - read from file or device
In	eax	3
	ebx	file descriptor
	ecx	address of the buffer to read into
	edx	maximum number of bytes to read
Out	eax	number of bytes actually read \| EAGAIN \| EBADF \| EFAULT \| EINTR \| EINVAL \| EIO \| EISDIR
write - write to file or device
In	eax	4
	ebx	file descriptor
	ecx	address of the buffer to write from
	edx	maximum number of bytes to write
Out	eax	number of bytes actually sent \| EAGAIN \| EBADF \| EFAULT \| EINTR \| EINVAL \| EIO \| ENOSPC \| EPIPE
open - open, create, or truncate a file or device
In	eax	5
	ebx	address of zero-terminated pathname
	ecx	file access bits
	edx	file permission mode
Out	eax	file descriptor of opened file \| EACCESS \| EEXIST \| EFAULT \| EISDIR \| ELOOP \| EMFILE, \| ENAMETOOLONG \| ENFILE \| ENOENT \| ENODEV \| ENODIR \| ENOMEM \| ENOSPC \| ENXIO \| EROFS \| ETXTBSY
close - close a file or device
In	eax	6
In	ebx	file descriptor
Out	eax	zero for success \| EBADF
waitpid - wait for a processes to terminate
In	eax	7
	ebx	process id of the process to wait for
	ecx	0, or address of buffer to hold exit state
	edx	option flags \| 0 \| WNOHANG \| WUNTRACED
Out	eax	pid of finished process \| ECHILD \| EINVAL \| ERESTART
Out	ecx	exit state of finished process, if non-zero value was input in ecx
create - create a file
In	eax	8
	ebx	address of zero-terminated pathname
	ecx	file permission mode
Out	eax	file descriptor of opened file \| EACCESS \| EEXIST \| EFAULT \| EISDIR \| ELOOP \| EMFILE, \| ENAMETOOLONG \| ENFILE \| ENOENT \| ENODEV \| ENODIR \| ENOMEM \| ENOSPC \| ENXIO \| EROFS \| ETXTBSY
Note	This call is identical to calling open with access bits O_CREATE\|O_WRONLY\|O_TRUNC
link - create a hard link to a file
In	eax	9
	ebx	address of zero-terminated pathname of existing file name
	ecx	address of zero-terminated pathname of new name
Out	eax	0 \| EACCESS \| EIO \| EPERM \| EEXIST \| EFAULT \| ELOOP \| EMLINK \| ENAMETOOLONG \| ENOENT \| ENOMEM \| ENOSPC \| ENOTDIR \| EPERM \| EROFS \| EXDEV
unlink - delete a name and remove file when not busy
In	eax	10
In	ebx	address of zero-terminated pathname of existing file name
Out	eax	0 \| EACCES \| EFAULT \| EIO \| EISDIR \| ELOOP \| ENAMETOOLONG \| ENOENT \| ENOMEM \| ENOTDIR \| EPERM \| EROFS
execve - execute a program
In	eax	11
	ebx	address of zero-terminated pathname of program
	ecx	address of zero-terminated list of addresses of zero-terminated argument strings
	edx	address of zero-terminated list of addresses of zero-terminated environment strings
Out	eax	If success, no return because the new program inherits resources and overwrites caller; otherwise: E2BIG \| EACCES \| EINVAL \| EIO \| EISDIR \| ELIBBAD \| ELOOP \| ENFILE \| ENOEXEC \| ENOENT \| ENOMEM \| ENOTDIR \| EFAULT \| ENAMETOOLONG \| EPERM \| ETXTBUSY
chdir - change working directory
In	eax	12
In	ebx	address of zero-terminated pathname of existing directory
Out	eax	0 \| EACCES \| EBADF \| EFAULT \| EIO \| ELOOP \| ENAMETOOLONG \| ENOENT \| ENOMEM \| ENOTDIR

File Access Bits

O_ACCMODE  0003
O_RDONLY 00
O_WRONLY 01
O_RDWR 02
O_CREAT 0100
O_EXCL 0200
O_NOCTTY 0400
O_TRUNC 01000
O_APPEND 02000
O_NONBLOCK 04000
O_NDELAY O_NONBLOCK
O_SYNC 010000 specific to ext2 fs and block devices
FASYNC 020000 fcntl, for BSD compatibility
O_DIRECT 040000 direct disk access hint - currently ignored
O_LARGEFILE 0100000
O_DIRECTORY 0200000 must be a directory
O_NOFOLLOW 0400000 don't follow links

File Permission Bits

S_ISUID 04000 set user ID on execution
S_ISGID 02000 set group ID on execution
S_ISVTX 01000 sticky bit
S_IRUSR 00400 read by owner (S_IREAD)
S_IWUSR 00200 write by owner (S_IWRITE)
S_IXUSR 00100 execute/search by owner (S_IEXEC)
S_IRGRP 00040 read by group
S_IWGRP 00020 write by group
S_IXGRP 00010 execute/search by group
S_IROTH 00004 read by others (R_OK)
S_IWOTH 00002 write by others (W_OK)
S_IXOTH 00001 execute/search by others (X_OK)
S_IRWXUGO (S_IRWXU|S_IRWXG|S_IRWXO) -re- umask
S_IALLUGO (S_ISUID|S_ISGID|S_ISVTX|S_IRWXUGO)
S_IRUGO (S_IRUSR|S_IRGRP|S_IROTH)
S_IWUGO (S_IWUSR|S_IWGRP|S_IWOTH)
S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)

Errors

#define	EPERM		 1	/* Operation not permitted */
#define	ENOENT		 2	/* No such file or directory */
#define	ESRCH		 3	/* No such process */
#define	EINTR		 4	/* Interrupted system call */
#define	EIO		 5	/* I/O error */
#define	ENXIO		 6	/* No such device or address */
#define	E2BIG		 7	/* Argument list too long */
#define	ENOEXEC		 8	/* Exec format error */
#define	EBADF		 9	/* Bad file number */
#define	ECHILD		10	/* No child processes */
#define	EAGAIN		11	/* Try again */
#define	ENOMEM		12	/* Out of memory */
#define	EACCES		13	/* Permission denied */
#define	EFAULT		14	/* Bad address */
#define	ENOTBLK		15	/* Block device required */
#define	EBUSY		16	/* Device or resource busy */
#define	EEXIST		17	/* File exists */
#define	EXDEV		18	/* Cross-device link */
#define	ENODEV		19	/* No such device */
#define	ENOTDIR		20	/* Not a directory */
#define	EISDIR		21	/* Is a directory */
#define	EINVAL		22	/* Invalid argument */
#define	ENFILE		23	/* File table overflow */
#define	EMFILE		24	/* Too many open files */
#define	ENOTTY		25	/* Not a typewriter */
#define	ETXTBSY		26	/* Text file busy */
#define	EFBIG		27	/* File too large */
#define	ENOSPC		28	/* No space left on device */
#define	ESPIPE		29	/* Illegal seek */
#define	EROFS		30	/* Read-only file system */
#define	EMLINK		31	/* Too many links */
#define	EPIPE		32	/* Broken pipe */
#define	EDOM		33	/* Math argument out of domain of func */
#define	ERANGE		34	/* Math result not representable */
#define	EDEADLK		35	/* Resource deadlock would occur */
#define	ENAMETOOLONG	36	/* File name too long */
#define	ENOLCK		37	/* No record locks available */
#define	ENOSYS		38	/* Function not implemented */
#define	ENOTEMPTY	39	/* Directory not empty */
#define	ELOOP		40	/* Too many symbolic links encountered */
#define	EWOULDBLOCK	EAGAIN	/* Operation would block */
#define	ENOMSG		42	/* No message of desired type */
#define	EIDRM		43	/* Identifier removed */
#define	ECHRNG		44	/* Channel number out of range */
#define	EL2NSYNC	45	/* Level 2 not synchronized */
#define	EL3HLT		46	/* Level 3 halted */
#define	EL3RST		47	/* Level 3 reset */
#define	ELNRNG		48	/* Link number out of range */
#define	EUNATCH		49	/* Protocol driver not attached */
#define	ENOCSI		50	/* No CSI structure available */
#define	EL2HLT		51	/* Level 2 halted */
#define	EBADE		52	/* Invalid exchange */
#define	EBADR		53	/* Invalid request descriptor */
#define	EXFULL		54	/* Exchange full */
#define	ENOANO		55	/* No anode */
#define	EBADRQC		56	/* Invalid request code */
#define	EBADSLT		57	/* Invalid slot */
#define	EDEADLOCK	EDEADLK
#define	EBFONT		59	/* Bad font file format */
#define	ENOSTR		60	/* Device not a stream */
#define	ENODATA		61	/* No data available */
#define	ETIME		62	/* Timer expired */
#define	ENOSR		63	/* Out of streams resources */
#define	ENONET		64	/* Machine is not on the network */
#define	ENOPKG		65	/* Package not installed */
#define	EREMOTE		66	/* Object is remote */
#define	ENOLINK		67	/* Link has been severed */
#define	EADV		68	/* Advertise error */
#define	ESRMNT		69	/* Srmount error */
#define	ECOMM		70	/* Communication error on send */
#define	EPROTO		71	/* Protocol error */
#define	EMULTIHOP	72	/* Multihop attempted */
#define	EDOTDOT		73	/* RFS specific error */
#define	EBADMSG		74	/* Not a data message */
#define	EOVERFLOW	75	/* Value too large for defined data type */
#define	ENOTUNIQ	76	/* Name not unique on network */
#define	EBADFD		77	/* File descriptor in bad state */
#define	EREMCHG		78	/* Remote address changed */
#define	ELIBACC		79	/* Can not access a needed shared library */
#define	ELIBBAD		80	/* Accessing a corrupted shared library */
#define	ELIBSCN		81	/* .lib section in a.out corrupted */
#define	ELIBMAX		82	/* Attempting to link in too many shared libraries */
#define	ELIBEXEC	83	/* Cannot exec a shared library directly */
#define	EILSEQ		84	/* Illegal byte sequence */
#define	ERESTART	85	/* Interrupted system call should be restarted */
#define	ESTRPIPE	86	/* Streams pipe error */
#define	EUSERS		87	/* Too many users */
#define	ENOTSOCK	88	/* Socket operation on non-socket */
#define	EDESTADDRREQ	89	/* Destination address required */
#define	EMSGSIZE	90	/* Message too long */
#define	EPROTOTYPE	91	/* Protocol wrong type for socket */
#define	ENOPROTOOPT	92	/* Protocol not available */
#define	EPROTONOSUPPORT	93	/* Protocol not supported */
#define	ESOCKTNOSUPPORT	94	/* Socket type not supported */
#define	EOPNOTSUPP	95	/* Operation not supported on transport endpoint */
#define	EPFNOSUPPORT	96	/* Protocol family not supported */
#define	EAFNOSUPPORT	97	/* Address family not supported by protocol */
#define	EADDRINUSE	98	/* Address already in use */
#define	EADDRNOTAVAIL	99	/* Cannot assign requested address */
#define	ENETDOWN	100	/* Network is down */
#define	ENETUNREACH	101	/* Network is unreachable */
#define	ENETRESET	102	/* Network dropped connection because of reset */
#define	ECONNABORTED	103	/* Software caused connection abort */
#define	ECONNRESET	104	/* Connection reset by peer */
#define	ENOBUFS		105	/* No buffer space available */
#define	EISCONN		106	/* Transport endpoint is already connected */
#define	ENOTCONN	107	/* Transport endpoint is not connected */
#define	ESHUTDOWN	108	/* Cannot send after transport endpoint shutdown */
#define	ETOOMANYREFS	109	/* Too many references: cannot splice */
#define	ETIMEDOUT	110	/* Connection timed out */
#define	ECONNREFUSED	111	/* Connection refused */
#define	EHOSTDOWN	112	/* Host is down */
#define	EHOSTUNREACH	113	/* No route to host */
#define	EALREADY	114	/* Operation already in progress */
#define	EINPROGRESS	115	/* Operation now in progress */
#define	ESTALE		116	/* Stale NFS file handle */
#define	EUCLEAN		117	/* Structure needs cleaning */
#define	ENOTNAM		118	/* Not a XENIX named type file */
#define	ENAVAIL		119	/* No XENIX semaphores available */
#define	EISNAM		120	/* Is a named type file */
#define	EREMOTEIO	121	/* Remote I/O error */
#define	EDQUOT		122	/* Quota exceeded */
#define	ENOMEDIUM	123	/* No medium found */
#define	EMEDIUMTYPE	124	/* Wrong medium type */