⚠️ This is not the current iteration of the course! Head here for the current offering.

Lecture 17: Process Creation

» Lecture video (Brown ID required)
» Lecture code
» Post-Lecture Quiz (due 11:59pm Sunday, April 10)

Process Lifecycle

Processes are how we run programs on our computers, and our computers often use several processes to get things done. For example, a simple terminal command such as ls or grep each run a new process that produces some output and then exits.

In WeensyOS, the kernel starts four processes at startup, but (at least until step 5 of Project 4), there is no way for a user-space process to start another user-space process. A realistic operating system clearly needs to be able to do so.

Process Creation

Many Unix-based operating systems – which include Linux, the BSD line of operating systems, and Mac OS X – use a system call named fork() for process creation. fork elicits controversy even after nearly 50 years of use, and it's not the only way to create processes (Windows, for example, has a different approach). But it is how millions of computers and devices do it!

The fork() system call

fork() has the effect of cloning a user-space process. For example, this program (fork1.cc) calls fork() ("forks"), prints a message, and exits: 

#include "helpers.hh"

int main() {
    pid_t p1 = fork();
    assert(p1 >= 0);

    printf("Hello from pid %d\n", getpid());
}
How many times will the message be printed when we run it? It is printed twice:

$ ./fork1
Hello from pid 19244
Hello from pid 19245
This happens because the call to fork() enters the kernel, which clones the process, and then continues user-space execution in both clones. Both processes execute the rest of the program, and thus both execute the printf function call. Note that the processes have different process IDs, as evidenced by the fact that the getpid() system call returns different values.

The return value from fork() depends on whether it is returning into the parent or into the child – every successful call to fork() returns twice:

  1. In the parent process, the return value is the new process's PID.
  2. In the child process, the return value is 0.
A negative return value indicates that fork() failed and no child process was created.

A forked child process shares many of its parent process's resources, and consequently the OS kernel needs to copy various pieces of information from the parent. The information that needs copying includes:

You will need to implement these copies as part of your fork implementation in WeensyOS.

Remember that execution continues in the same program for both the parent and child process (although their execution can diverge). If the child forks again, it can create further processes (see fork2.cc, which ends up with a total of four processes).

Since the child process receives a full copy of the parent process's address space, any virtual address that was mapped and valid in the parent is also valid in the child process. However, the same virtual address is backed by a different physical address in the child. In other words, parent memory and child memory are entirely independent.

Let's do a quick exercise to remind us of what fork() does. Take a look at this program:

int main() {
    printf("Hello from initial pid %d\n", getpid());

    pid_t p1 = fork();
    assert(p1 >= 0);

    pid_t p2  = fork();
    assert(p2 >= 0);

    printf("Hello from final pid %d\n", getpid());
}

Question: How many lines of output would you expect to see when you run the program?

Running a different program

If we just had fork(), we would only be able to execute copies of a single user-space process. But in reality, we want to be able to start other programs from a user-space process. One key example of a program that does this is your shell: when you type a command like ./myprogram into the terminal, the shell executes myprogram.

There are different ways to achieve this goal, some involving fork(). The debate over which way is best still rages today.

The UNIX way: fork-and-exec style

There is a family of system calls in UNIX that executes a new program. The system call we will discuss here is execv(). At some point you may want to use other system calls in the exec syscall family – you can use man exec to find more information about them. They differ primarily in how they except their arguments to be passed.

The execv system call (and all system calls in the exec family) performs the following:

Note that execv does not "spawn" a process. It destroys the current process and replaces it. Therefore, it's very common to use execv in conjunction with fork: we first call fork() to create a child process, and then call execv() to run a new program inside the child process, replacing the "process image" that fork() copied.

Let's look at the program in myecho.cc:

int main(int argc, char* argv[]) {
    fprintf(stderr, "Myecho running in pid %d\n", getpid());
    for (int i = 0; i != argc; ++i) {
        fprintf(stderr, "Arg %d: \"%s\"\n", i, argv[i]);
    }
}

It's a simple program that prints out its pid and content in its argv[].

We will now run this program using the execv() system call. The "launcher" program where we call execv is in forkmyecho.cc:

int main() {
    const char* args[] = {
        "./myecho", // argv[0] is the string used to execute the program
        "Hello!",
        "Myecho should print these",
        "arguments.",
        nullptr
    };

    pid_t p = fork();

    if (p == 0) {
        fprintf(stderr, "About to exec myecho from pid %d\n", getpid());

        int r = execv("./myecho", (char**) args);

        fprintf(stderr, "Finished execing myecho from pid %d; status %d\n",
                getpid(), r);
    } else {
        fprintf(stderr, "Child pid %d should exec myecho\n", p);
    }
}

The goal of the launcher program is to run myecho with the arguments shown in the args[] array. We need to pass these arguments to the execv system call. In the child process created by fork() we call execv to run the myecho program.

Terminating the argument array correctly

execv and execvp system calls take an array of C strings as the second parameter, which are arguments to run the specified program with. Note that everything here is in C: the array is a C array, and the strings are C strings. The array must be terminated by a nullptr (or NULL) as a C array contains no length information.

Running forkecho gives us outputs like the following:

Child pid 1440 should exec myecho
About to exec myecho from pid 1440
$ Myecho running in pid 1440
Arg 0: "./myecho"
Arg 1: "Hello!"
Arg 2: "Myecho should print these"
Arg 3: "arguments."

Notice that the line "Finished execing myecho from pid..." never gets printed! This is the case because the fprintf call printing this message comes after the execv system call. If the execv call is successful, the process's address space at the time of the call gets blown away (including the stack), so anything after execv won't execute at all. Another way to think about it is that if the execv system call succeeds, then the system call never returns. (Note though, that exec does return if it fails – it's not correct to write code that assumes that it never returns!)

The picture below summarizes what happened here, with the forkmyecho child process in green and the myecho child process in blue. (The red waitpid() part is explained further down.)

Note that there are three processes in total involved here: P1 is the original shell process running in your terminal, P2 is a child it forks, which then gets replaced by forkmyecho, and P3 is the process that ultimately runs myecho.

Alternative interface: posix_spawn

Calling fork() and execv() in succession to run a process may appear counter-intuitive and even inefficient. Imagine a complex program with gigabytes of virtual address space mapped and it wants to creates a new process. What's the point of copying the big virtual address space of the current program if all we are going to do is just to throw everything away and start anew?

These are valid concerns regarding the UNIX style of process management. Modern Linux systems provide an alternative system call, called posix_spawn(), which creates a new process without copying the address space or destroying the current process. A new program gets "spawned" in a new process and the pid of the new process is returned via one of the pointer arguments. Non-UNIX operating systems like Windows also uses this style of process creation.

The program in spawnmyecho.cc shows how to use the alternative interface to run a new program:

int main() {
    const char* args[] = {
        "./myecho", // argv[0] is the string used to execute the program
        "Hello!",
        "Myecho should print these",
        "arguments.",
        nullptr
    };

    fprintf(stderr, "About to spawn myecho from pid %d\n", getpid());

    pid_t p;
    int r = posix_spawn(&p, "./myecho", nullptr, nullptr,
                        (char**) args, nullptr);

    assert(r == 0);
    fprintf(stderr, "Child pid %d should run myecho\n", p);
}

Note that posix_spawn() takes many more arguments than execv(). This has something to do with the managing the environment within which the new process to be run.

In the fork-and-exec style of process creation, fork() copies the current process's environment, and execv() preserves the environment. The explicit gap between fork() and execv() provides us a natural window where we can set up and tweak the environment for the child process as needed, using the parent process's environment as a starting point.

With an interface like posix_spawn(), however, we need to supply more information directly to the system call itself. Take a look at posix_spawn's manual page to find out what these extra nullptr arguments are about – they are quite complicated. This teaches an interesting lesson in API design: performance and usability of an API, in many cases, are often a trade-off.

Why do we still have fork()?

The debate of which style of process creation is better has never settled. Modern UNIX operating systems inherited the fork-and-exec style from the original 1970s UNIX, where fork() turned out extremely easy to implement. Modern UNIX systems can execute fork() very efficiently without actually performing any substantial copying (using copy-on-write optimization) until necessary. For these reasons, in practice, the performance of the fork-and-exec style is not a common concern.

Running execv() without fork()

You might wonder what happens if we don't fork and just run execv. Let's take a look at runmyecho.cc:

int main() {
    const char* args[] = {
        "./myecho", // argv[0] is the string used to execute the program
        "Hello!",
        "Myecho should print these",
        "arguments.",
        nullptr
    };
    fprintf(stderr, "About to exec myecho from pid %d\n", getpid());

    int r = execv("./myecho", (char**) args);

    fprintf(stderr, "Finished execing myecho from pid %d; status %d\n",
            getpid(), r);
}

This program now invokes execv() directly, without fork-ing a child first. The new program (myecho) will print out the same pid as the original process. execv() blows away the old process's image (including code, global variables, heap, and stack), but it does not change the pid, because no new processes gets created. The new program runs inside the same process after the old program gets destroyed.

The picture below contrasts execution with fork() (left side) and with just execv() (right side):

Observe that if your shell was to just call execv(), it could only ever run a single command that would never return!

Summary

Today, we talked about how the fork() system call allows a user-space process to start another process by essentially cloning itself. The two processes, called "parent" and "child" continue executing from the same place in the code, and they start with identical memory mappings (though these mappings are backed by different physical memory pages, for the most part). But the processes can evolve independently after the fork() system call returns. In Project 3, you will implement handling of the fork() system call in the WeensyOS kernel!

We also discussed the exec() family of system calls, which replace a running process's program with another program. Composing fork() and execv() allows for a process to start another program, and gives us the basic building blocks to make, e.g., a shell.