Lecture 16: Inter-Process Communication, Threads

» Lecture video (Brown ID required)
» Lecture code
» Post-Lecture Quiz (due 11:59pm Monday, April 6)

Inter-Process Communication

Processes, though operating in isolated virtual address spaces, often need to communicate with each other in controlled ways. This is what Inter-Process Communication (IPC) is about. We already saw simple examples of IPC in the form of process exit statuses last time. In addition, Linux provides other mechanisms for processes to communicate: they can read and write files on disk, or they can rely on a more efficient, in-memory streaming transport in the form of pipes.

The notion of multiple processes working together to achieve a goal shows up in many ways in your everday computing. Two key examples are:

• Combining multiple shell commands via the | ("pipe") character: you may have seen chains of commands such as wc -l *.cc | sort -n, which counts the lines in all C++ source files in the current directory (wc -l *.cc) and then sorts the result by the number of lines using another command (sort -n). The pipe symbol here tells the shell to set up the processes with a pipe between them and for the first command to send its output to the pipe, from where the second command reads it.
• Speeding up operations that interact with a large data set or many requests by splitting the work across processes that communicate results back to a parent process. This works especially well if you're using a computer with multiple processors, where in the best case N processes achieve N× the speed or throughput of a single process.

Pipes

A pipe provides a unidirectional, in-order transport mechanism. A process can use a pipe to send data to itself, but a pipe is most powerful when combined with fork(), which allows it to be used for communication across parent and child processes.

The signature of the pipe() system call looks like this:

int pipe(int pfd[2]);

It is the responsibility of the user-space process to allocate memory for the pfd array. The memory can be in a global variable, on the stack, or on the heap. This follows a general principle with Linux system calls: the memory that the kernel writes data or information to must be allocated by user-space and a valid pointer passed into the kernel.

A successful call to pipe() creates 2 file descriptors, placed in array pfd:

• pfd[0]: read end of the pipe
• pfd[1]: write end of the pipe

How do I remember which end is which?

The default file descriptors available to each process (stdin, stdout, and stderr) provide a useful mnemonic to remember which end of a pipe is the read end:

• 0 is the FD number for stdin, 1 is the FD number of stdout
• Programs read from stdin and write to stdout
• pfd[0] is the read end (input end), pfd[1] is the write end (output end)

Data written to pfd[1] can be read from pfd[0].

Can a pipe be used bi-directionally?

The read end of the pipe can't be written, and the write end of the pipe can't be read. Attempting to read/write to the wrong end of the pipe will result in a system call error (the read() or write() call will return -1).

Let's look at a concrete example in selfpipe.cc:

int main() {
    int pfd[2];
    int r = pipe(pfd);
    assert(r == 0);

    char wbuf[BUFSIZ];
    sprintf(wbuf, "Hello from pid %d\n", getpid());

    ssize_t n = write(pfd[1], wbuf, strlen(wbuf));
    assert(n == (ssize_t) strlen(wbuf));

    char rbuf[BUFSIZ];
    n = read(pfd[0], rbuf, BUFSIZ);
    assert(n >= 0);
    rbuf[n] = 0;

    assert(strcmp(wbuf, rbuf) == 0);
    printf("Wrote %s", wbuf);
    printf("Read %s", rbuf);
}

This code (which doesn't contain fork()) creates a pipe, writes to it, and reads from it within a single process. We create a pipe, write to the pipe, and then read from the pipe. Finally, we assert that the string we get out of the pipe is the same string we wrote into the pipe.

You might wonder where the data goes after the write() system call completes, but before the read() from the pipe is invoked. The data doesn't live in the process's address space! It actually goes into a memory buffer located in the kernel address space.

The read() system call blocks when reading from a stream file descriptor that doesn't have any data to be read. Pipe file descriptors are stream file descriptors, so reading from an empty pipe will block (meaning the kernel won't schedule the reading process until there is data available in the pipe). write() calls to a pipe block when the buffer is full (because reader the not consuming quickly enough). A read() from a pipe returns EOF if all write ends of a pipe is closed.

So far we've only seen pipe functioning within the same process. Since the pipe lives in the kernel, it can also be used to pass data between processes. Let's take a look at childpipe.cc as an example:

int main() {
    int pipefd[2];
    int r = pipe(pipefd);
    assert(r == 0);

    pid_t p1 = fork();
    assert(p1 >= 0);

    if (p1 == 0) {
        // child process
        const char* message = "Hello, mama!";
        printf("[child: %d] sending message\n", getpid());
        ssize_t nw = write(pipefd[1], message, strlen(message));
        assert(nw == (ssize_t) strlen(message));
        exit(0);
    }

    char buf[BUFSIZ];
    if (read(pipefd[0], buf, BUFSIZ) > 0) {
        printf("[parent: %d] I got a message! It was “%s”\n", getpid(), buf);
    }

    close(pipefd[0]);
    close(pipefd[1]);
}

Here, we use fork() to create a child process, but note that before forking we created a pipe first! The fork() duplicates the two pipe file descriptors in the child, but note that the pipe itself is not duplicated (because the pipe doesn't live in the process's address space). The child then writes a message to the pipe, and the same message can be read from the parent. Interprocess communication!

Note that in the scenario above we have four file descriptors associated with the pipe, because fork() duplicates the file descriptors corresponding to two ends of a pipe. The pipe in this case has two read ends and two write ends. The animated picture below (courtesy of Eddie Kohler) illustrates the situation in terms of the FD table maintained for both processes:

Note that there continues to exist a write end of the pipe in the parent, and a read end in the child that never get closed! This can lead to weird hangs when using pipes, so a common idiom is to close the pipe ends that a process does not need immediately after forking. In the example, we need to close the write end in the parent and the read end in the child:

    ...
    pid_t p1 = fork();
    assert(p1 >= 0);

    if (p1 == 0) {
       ... // child code
    }

    close(pipefd[1]); // close the write end in the parent
    char buf[BUFSIZ];
    if (read(pipefd[0], buf, BUFSIZ) > 0) {
    ...

Are pipes always 1:1 between a parent and a child process?

Pipes are quite powerful! You can also create a pipe between two child processes (think about how you would use fork(), pipe(), and close() to achieve this!), and a pipe can actually have multiple active read ends and write ends. A use case for a pipe with multiple active write ends is a multiple-producer, single-consumer (MPSC) setup – this is useful, e.g., when a process listens for messages from a set of other processes. Likewise, a pipe with multiple active read ends supports a single-producer, multiple consumer (SPMC) use case, where multiple readers compete for messages (e.g., work units) sent by a writer. Multiple producer, multiple consumer (MPMC) is also possible, but rarely used in practice because it becomes difficult to coordinate the readers and writers.

Concurrency vs. Parallelism

Modern day computers usually have multiple processors (CPUs) built in. This allows them to run instructions from two processes simultaneously: each processor has its own set of registers and L1 cache, and all processors can access memory (RAM).

But before ca. 2005, few computers had multiple processors. Yet, computers could still run multiple programs using the idea of "time-sharing", where the processor will switch between running processes in turn, entering the kernel in between different user-space processes.

This notion, which involves multiple processes being active on the computer at the same time is called multiprocessing, and constitutes a form of concurrency. Concurrency means that multiple processes are active concurrently. (As we will see shortly, this also generalizes to running multiple concurrent threads on the same processor.)

The notion of parallelism is subtly different from concurrency: it refers to the actual execution of instructions from multiple processes (or threads) truly at the same time on different processors. In order to achieve parallelism, you need multiple processors that are active at the same time. But a parallel execution is also concurrent, since multiple processes (or threads) are still active concurrently.

Threads

Pipes and IPC more generally allow processes in isolated address spaces to communicate in restricted and kernel-mediated ways. But sometimes you want multiple concurrent execution units to work on truly shared memory (e.g., because the data is large and forking would be expensive; or because you're implementing an application with truly shared data across parallel requests). For this purpose, operating systems provide threads, which execute concurrently like processes, but share the single virtual address space of their parent process.

Here is a comparison between what a parent and child process shared, and what two threads within the same process share:

A process can contain multiple threads. All threads within the same process share the same virtual address space and file descriptor table. However, each thread must have its own set of registers and stack. (Think about all the things that would go horribly wrong if two threads used the same stack and ran in parallel on differnt processors!). The processes we have looked at so far all have a single thread running.

It's possible to create threads directly using the clone() system call on Linux, but doing so is rather cumbersome (e.g., you need to allocate the stack for the new thread explicitly before calling clone()). Furthermore, the details of the clone() system call are very specific to Linux, but multithreaded programs should work across different operating systems. In practice, most programs that use threads therefore rely on a library to provide a more intuitive interface to threads. Commonly used libraries include pthreads and the the std::thread implementation in the C++ standard library. We will use the latter in CS 131.

Summary

Today, we dove deeper into how processes can interact via pipes, which are kernel-mediated shared-memory buffers that processes access via file descriptors. We saw that combining pipes with forking allows processes to establish communication channels with (and even between) their children. This is a very powerful abstraction that allows us to chain shell commands, and to achieve parallelism using multiple processes on multi-processor computers.

We also covered the difference in definition between the terms "concurrency" and "parallelism", which refer to the notion of multiple active processes or threads (concurrency) and truly parallel-in-time execution on different processors (parallelism).

Finally, we started looking at threads, which are similar to processes in that they execute independently and on their own stack, but differ in that they share a virtual address space (and therefore all memory) with other threads in the same process. We'll dig more into threads and how programming with threads can lead to weird behavior next time!