# Lecture 1: OS Overview

## I. Overview

### 6.828 Goals

* Understand operating system design and implementation
* Hands-on experience by building small OS

### What is the purpose of an OS?

* Support applications
* Abstract the hardware for convenience and portability
* Multiplex the hardware among multiple applications
* Isolate applications in order to contain bugs
* Allow sharing among applications
* Provide high performance

### What is the OS design approach?

* Small view: a H/W management library
* Big view: physical machine -> abstract one w/ better properties

### Organization - Layered Picture

| Layers |
|----------|
| User applications: vi, gcc, &c |
| Kernel services |
| H/W: CPU, mem, disk, &c |

* we care a lot about the interfaces and internal kernel structure in this course

### What services does an OS kernel typically provide?

* processes
* memory allocation
* file contents
* directories and file names
* security
* many others: users, IPC, network, time, terminals

### What does an OS abstraction look like?

* Applications see them only via system calls
* For example, from UNIX (e.g. Linux, OSX, FreeBSD):

```c
fd = open("out", 1);
write(fd, "hello\n", 6);
pid = fork();
```

### Why is OS design/implementation hard/interesting?

* the environment is unforgiving: quirky h/w, weak debugger
* it must be efficient (thus low-level?)... but abstract/portable (thus high-level?)
* powerful (thus many features?)... but simple (thus a few composable building blocks?)
* features interact: `fd = open(); ...; fork()`
* behaviors interact: CPU priority vs memory allocator
* open problems: security; performance; exploiting new hardware

### You'll be glad you learned about operating systems if you...

* want to work on the above problems
* care about what's going on under the hood
* have to build high-performance systems
* need to diagnose bugs or security problems

## II. Class structure

### Lectures

* O/S ideas
* detailed inspection of xv6, a traditional O/S
* xv6 programming homework to motivate lectures
* papers on some recent topics

### Labs: JOS, a small O/S for x86 in an exokernel style

* you build it, 5 labs + final lab of your choice
* kernel interface: expose hardware, but protect -- few abstractions!
* unprivileged user-level library: fork, exec, pipe, ...
* applications: file system, shell, ..
* development environment: gcc, qemu
* lab 1 is out

(demo: build JOS and run ls)

### Xv6 and JOS

* Two different designs
* Hopefully will force you understand issues better, but will be confusing too
* Neither has the complexity of a real OS

(demo: build Linux)

## III. Introduction to system calls

6.828 is largely about design and implementation of system call interface. Let's look at how programs use that interface. We'll focus on UNIX (Linux, Mac, POSIX, &c).

### A simple example: what system calls does "ls" call?

* Trace system calls:
    - On OSX: `$ sudo dtruss /bin/ls`
    - On Linux: `$ strace /bin/ls`
So many system calls!

### An interesting program: the Unix shell

* it's the Unix command-line user interface
* it's a good illustration of the UNIX system call API
* some example commands:

```sh
$ ls > junk
$ ls | wc -l
$ ls | wc -l > junk
```

* the shell is also a programming/scripting language

```sh
$ cat > script
    echo one
    echo two
$ sh < script
```

* the shell uses system calls to set up redirection, pipes, waiting
programs like wc are ignorant of input/output setup

#### Let's look at source for a simple shell (sh.c, [xv6 source](../resources/xv6-source-rev11.pdf))

* main()
    - basic organization: parse into tree, then run main process: getcmd, fork, wait
    - child process: parsecmd, runcmd 
    - why the fork()? we need a new process for the command
    - what does fork() do?
        + copies user memory and copies kernel state, e.g. file descriptors, so "child" is almost identical to "parent"
        + child has different "process ID"
        + both processes now run, in parallel
        + fork() returns twice, once in parent, once in child. fork() returns child PID to parent and returns 0 to child
    - so `sh` calls runcmd() in the child process
    - why the wait()? what if child exits before parent calls wait()?

* runcmd()
    - executes parse tree generated by parsecmd()
    - distinct cmd types for simple command, redirection, pipe

* runcmd() for simple command with arguments
    - `execvp(cmd, args)`
    - ls command &c exist as executable files, e.g. /bin/ls
    - execvp() loads executable file over memory of current process
    - jumps to start of executable - `main()`
    > Note: execvp doesn't return if all goes well
    > execvp() only returns if it can't find the executable file
    > it's the shell child that's replaced with execvp()
    > the main shell process is still wait()ing for the child

* how does runcmd() handle I/O redirection?

    - `echo hello > junk`
    - parsecmd() produces tree with two nodes: `cmd->type='>', cmd->file="junk", cmd->cmd=...` and `cmd->type=' ', cmd->argv=["echo", "hello"]`
    - the open(); dup2() causes FD 1 to be replaced with FD to output file
    - it's the shell child process that changes its FD 1
    - execvp() preserves the FD setup
    - so `echo` runs with FD 1 connected to file junk. again, very nice that echo is oblivious, just writes FD 1

* why are fork() and exec() separate?
    - perhaps wasteful that fork copies shell memory, only to have it thrown away by exec()
    - the point: the child gets a chance to change FD setup before calling exec()
    and the parent's FD set is not disturbed

* how does the shell implement pipelines? `$ ls | wc -l`

* the kernel provides a pipe abstraction
    - int fds[2]
    - pipe(fds)
    - a pair of file descriptors: a write FD, and a read FD
    - data written to the write FD appears on the read FD

* example: pipe1.c
    - read() blocks until data is available
    - write() blocks if pipe buffer is full

* pipe file descriptors are inherited across fork(), so pipes can be used to communicate between processes. for many programs, just like file I/O, so pipes work for stdin/stdout

* for `ls | wc -l`, shell must:
    - create a pipe
    - fork()
    - set up fd 1 to be the pipe write FD
    - exec() `ls`
    - set up wc's fd 0 to be pipe read FD
    - exec() `wc`
    - wait() for `wc`
    - note: sh close()es unused FDs, so exit of writer produces EOF at reader

* you'll implement pieces of a shell in an upcoming homework