Having spent a good portion of yesterday anthropomorphizing a CPU, I'm all ready to anthropomorphize the role of an operating system in, say, launching an application or interacting with a shell or a Scheme interpreter. Not really. I was looking over some of the early entries and noted that I'd promised to alter the names in database example and insert it into the July 6, 2002 entry. The easiest way of doing this it seemed to me was to just alter database commands using fake names and make the database do the work rather than doctoring the earlier text which include the students names. This turned out to be a royal pain due to the careless way in which I'd set up the files and permissions when I originally installed the MySQL database. Basically I forgot everything that I'd done earlier including the passwords for special administrator accounts and in the end it was easier just to delete everything and rebuild the database from start from scratch. I wasted several hours doing this however and it reminded me yet again about how literal-minded computers are and how tricky it can be when you deal with relatively low-level systems hacking.

Anyway, I'm tired and a little short tempered but I'm obstinate and I wanted to say a few things about operating systems.

Each time that you start (launch) an application you create a new process that may spawn off several threads of control. The operating system allocates space both to run processes (applications are typically stored on disk until they're needed at which point the operating system has to load the corresponding application code into memory in order for it to be run) and to use as temporary working memory say to keep track of text and formatting commands in a word-processing application. The application code itself relies on the operating system in a number of ways. Application programmers don't have to figure out how to create files or open and modify existing ones; they don't have to think about whatever disks might be attached to a computer to store files. The OS provides primitives to find, open, close, read from, write to and create new files. The OS does a lot more however - it basically provides a high-level abstraction that allows the programmer to think in terms of files, processes, input and output without having to worry about exactly how these things are implemented in low-level code. Some programmers look at the task of writing "drivers" - the pieces of code deep in the operating system that enable you to use special hardware on a particular machine - as one of the most onerous and mind-numbing programming tasks they can imagine. The operating system provides services that many applications need thereby freeing up the programmer to think about other things.

Here are the basic operations that a programmer typically expects of a modern operating system (adapted from [Silberschatz et al., 2001])

• process management
• creating, suspending, resuming, deleting, scheduling processes; providing mechanisms for synchronization, communication, deadlock handling so that it is easy to create and manage multiple threads of control.
• memory management
• allocating and deallocating memory; keeping track of which parts of memory are being used by what and making sure that different applications and different users don't get in each others way - for most applications the programmer can just behave like there's only one application running on the computer.
• file management
• creating and deleting files and directories; I don't want think about disk drives and directory hierarchies, I just want to be able to think about files like I think about the books on my shelves calling them by name and leafing through them in any way that feels natural.
• I/O management
• buffering, caching, spooling; the device-driver interface - please make my digital camera behave like a directory full of images, my keyboard like a file full of characters, and my new optical mouse like generator of screen coordinates; and don't bother me with the details.
• networking
• support for communication in distributed systems; allow me to access files and processes running on other machines as easily as I manage files and processes running on my own machine; let me think about programs with clients and servers running all over the world as easily as I think about building bridges out of Lego bricks.
• protection system
• controlling access to system resources; don't let some cracker or hacker break into my machine but also protect me from myself so I won't crash my machine or delete files that I can't do without.
• command-interpreter system
• interface between the user and the OS; make it easy for me call up the spirit of the machine and make it do my bidding, not just by invoking big applications but searching through the file system, performing routine sorts of maintenance and arranging things to suit my way of thinking.

Think back to the Unix shell - the most basic way of interacting with the Unix operating system and thus indirectly with the CPU. Remember ls for listing files, chdir for climbing up and down the file hierarchy, pipes | and various forms of redirection > and >> for creating and adding to files, rm for deleting (removing) files, and all sorts of useful little programs like grep, awk and sed for searching and manipulating files. Invoking a shell with sh creates a new process and any time that you end a command with & you're telling that you want the operating system to run your command in a separate (child) process. When you work in a shell typing commands and looking at the computer's responses, you're talk directly to the operating system in much the same way that an applications programmer writes code. This doesn't mean that you escape all of the hairy little details of thinking about, say, memory - allocating it when you need it, deallocating it when you're done, organizing it to keep track of whatever it is you need to remember - but the operating system makes it considerably easier and provides abstractions that are simple and uniform across all sorts of variations in hardware. Some programmers think the OS should do more - make things even easier, but others don't want to relinquish their control over and access to the machine. The design of operating systems like the design of programming languages is as much a question of psychology and sociology as it is one of technology - for now people program computers and people have to communicate with computers. Here's an annotated conversation with my computer that I had just a minute ago.

What day is it?

/u/tld/ % date
Tue Jul 30 16:37:28 EDT 2002

Connect me to the directory in which I'm keeping this month's diary entries. (The entry in my home directory, diary, is a shortcut (a "soft link") to the directory /u/tld/journal/2002/07/, and the change-directory program, chdir, allows me to traverse the link to land in another directory.)

/u/tld/ % chdir diary

What's in the directory? (The list-contents-of-directory program, ls lists the names of the files and directories in /u/tld/diary/ - there are only directories, one for each day in July that I wrote an entry.)

/u/tld/diary % ls
03  04  05  06  07  09  11  12  14  15  17  20  21  26  27  28  29  30

I'm lazy. How many files is that? (The the pipe | makes the list generated by ls look like a file or list of files which the word-count program, wc, is expecting, and the -l tells wc to count lines and not words.)

/u/tld/diary % ls | wc -l 
      18

I've created eighteen entries since I started. I wonder how many word I've written this month. (The * is a wildcard used in regular expressions to match zero or more characters, wc handles a list of files as easily as it does one and conveniently sums the results of all files in the list, and -w tells wc that I want to count words and not lines.)

/u/tld/diary % wc -w */day.txt
    1554 03/day.txt
    2954 04/day.txt
    5778 05/day.txt
    7117 06/day.txt
    2131 07/day.txt
    3634 09/day.txt
    2946 11/day.txt
    3052 12/day.txt
    3139 14/day.txt
    6392 15/day.txt
    3540 17/day.txt
    1018 20/day.txt
    4664 21/day.txt
    3512 26/day.txt
    3514 27/day.txt
    2063 28/day.txt
    1589 29/day.txt
    1132 30/day.txt
   59729 total

That makes me feel tired; I think I'll take a walk while there's still time to enjoy the sun on the beach. (Use the tape-archive program, tar - the "tape" part is a bit anachronistic - to create an archive, -c, in the form of a file, -f, (and not a tape) named Jul30.tar which contains everything I've written in the last month and then exit the shell.)

/u/tld/email/diary tar -c -f Jul30.tar .
/u/tld/email/diary % ls Jul*
Jul30.tar 
/u/tld/email/diary % exit
Process shell finished