Pipes

Tags: comp-arch

Is a POSIX kernel construct that simply takes a stream of bytes as input and writes a stream of bytes as output. See the pipe system-call.

Pipes are designed such that they can be chained. A common example of this is your shell. Each command in a shell pipeline has its own dedicated process and has its standard-output linked to the next commands input and its standard-input connected to the previous commands output. The first command in a pipeline reads stdin from your shell and the last command writes stdout to your shell.

ls -1 | cut -d'.' -f1 | sort | uniq | tee dirents

Code Snippet 1: An example of a shell pipeline with 5 processes.

int filedes[2]; // { 0 = read, 1 = write }
pipe(filedes);
pid_t child = fork();
if (child > 0) {
    char buffer[80];
    int bytesread = read(filedes[0], buffer, sizeof(buffer));
    // do something with the bytes read
} else {
    write(filedes[1], "done", 4);
}

Code Snippet 2: A form of inter-process communication using pipes. The parent process creates a pipe and shares it among itself and its child. The child writes to the pipe and the parent reads from the pipe.

Warn: A pipe is buffered meaning you can't write to it endlessly. A process that repeatedly calls write on a pipe but doesn't have anything reading from pipe will fill up the pipe and all the writes following this will fail (until a read occurs and frees up some space on the pipe).

Pipe writes are atomic up to the size of the pipe. Meaning that if two processes try to write to the same pipe, the kernel has internal mutexes with the pipe that it will lock, do the write, and return. The only gotcha is when the pipe is about to become full. If two processes are trying to write and the pipe can only satisfy a partial write, that pipe write is not atomic – be careful about that!

Note: You can create named pipes that can be opened from the disk using mkfifo. Note these pipes don't actually read or write from disk. The disk entry is purely an abstraction for use by other programs who you can't direct a pipe to directly.

Links to this note

Inter-Process Communication