Recall: Key Unix I/O Design Concepts
• Uniformity – Everything Is a File!
– file operations, device I/O, and interprocess communication through open, read/write, close – Allows simple composition of programs
» find | grep | wc …
Copyright By PowCoder代写 加微信 powcoder
• Open before use
– Provides opportunity for access control and arbitration
– Sets up the underlying machinery, i.e., data structures
• Byte-oriented
– Even if blocks are transferred, addressing is in bytes
• Kernel buffered reads
– Streaming and block devices looks the same, read blocks yielding processor to other task
• Kernel buffered writes
– Completion of out-going transfer decoupled from the application, allowing it to continue
• Explicit close
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Recall: Low-Level vs High-Level file API
• Low-level direct use of syscall interface:
open(), read(), write(), close()
• Opening of file returns file descriptor:
int myfile = open(…);
• File descriptor only meaningful to kernel
– Index into process (PDB) which holds pointers to kernel-level structure (“file description”) describing file.
• Every read() or write() causes syscall no matter how small (could read a single byte)
• Consider loop to get 4 bytes at a time using read():
– Each iteration enters kernel for 4 bytes.
• High-level buffered access:
fopen(), fread(), fwrite(), fclose()
• Opening of file returns ptr to FILE:
FILE *myfile = fopen(…);
• FILE structure is user space contains:
– a chunk of memory for a buffer
– the file descriptor for the file (fopen() will call open() automatically)
• Every fread() or fwrite() filters through buffer and may not call read() or write() on every call.
• Consider loop to get 4 bytes at a time using fread():
– First call to fread() calls read() for block of bytes (say 1024). Puts in buffer and returns first 4 to user.
– Subsequent fread() grab bytes from buffer
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.2
Recall: Low-Level vs. High-Level File API
Low-Level Operation: ssize_t read(…) {
High-Level Operation: ssize_t fread(…) {
asm code … syscall # into %eax put args into registers %ebx, … special trap instruction
Check buffer for contents Return data to caller if available
asm code … syscall # into %eax put args into registers %ebx, … special trap instruction
get args from regs
dispatch to system func
Do the work to read from the file Store return value in %eax
get args from regs
dispatch to system func
Do the work to read from the file Store return value in %eax
get return values from regs
Return data to caller
get return values from regs
Update buffer with excess data Return data to caller
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Recall: Sockets:An Endpoint for Communication
• Key Idea: Communication across the world looks like File I/O write(wfd, wbuf, wlen);
Process Socket
• Sockets: Endpoint for Communication – Queues to temporarily hold results
• Connection:Two Sockets Connected Over the network ⇒ IPC over network! – How to open()?
– What is the namespace?
– How are they connected in time?
n = read(rfd, rbuf, rmax);
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.4
Recall: Connection Setup over TCP/IP
Client Side
Connection request:
1. Client IP addr
2. Client Port
3. Protocol (TCP/IP)
Server Side
Server Socket
Server Listening:
1. Server IP addr
2. well-known port,
3. Protocol (TCP/IP)
new socket
connection
• Special kind of socket: server socket – Has file descriptor
– Can’t read or write
• Two operations:
listen(): Start allowing clients to connect
accept(): Create a new socket for a particular client
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Request Connection
Recall: Connection Setup over TCP/IP
Client Side
Connection request:
1. Client IP addr
2. Client Port
3. Protocol (TCP/IP)
Server Socket
Server Side
Server Listening:
1. Server IP addr
2. well-known port,
3. Protocol (TCP/IP)
connection
new socket
• 5-Tuple identifies each connection:
1. Source IP Address
2. Destination IP Address
3. Source Port Number
4. Destination Port Number
5. Protocol (always TCP here)
• Often, Client Port “randomly” assigned – Done by OS during client socket setup
• Server Port often “well known”
– 80 (web), 443 (secure web), 25 (sendmail), etc
– Well-known ports from 0—1023
Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.6
Request Connection
Web Server
Web Server
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Client-Server Models
• File servers, web, FTP, Databases, …
• Many clients accessing a common server
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Sockets in concept
Create Client Socket
Connect it to server (host:port)
Create Server Socket
Bind it to an Address (host:port)
Listen for Connection
Connection Socket write request
read response
Close Client Socket
Accept syscall() Connection Socket
read request write response
Close Connection Socket
Close Server Socket
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Client Protocol
char *host_name, *port_name; // Create a socket
struct addrinfo *server = lookup_host(host_name, port_name);
int sock_fd = socket(server->ai_family, server->ai_socktype, server->ai_protocol);
// Connect to specified host and port connect(sock_fd, server->ai_addr, server->ai_addrlen);
// Carry out Client-Server protocol run_client(sock_fd);
/* Clean up on termination */ close(sock_fd);
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.10
Server Protocol (v1)
// Create socket to listen for client connections char *port_name;
// Bind socket to specific port
// Start listening for new client connections
while (1) {
// Accept a new client connection, obtaining a new socket int conn_socket = accept(server_socket, NULL, NULL); serve_client(conn_socket);
close(conn_socket);
} close(server_socket);
struct addrinfo *server = setup_address(port_name);
int server_socket = socket(server->ai_family, server->ai_socktype, server->ai_protocol);
bind(server_socket, server->ai_addr, server->ai_addrlen);
listen(server_socket, MAX_QUEUE);
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.11
How Could the Server Protect Itself?
• Handle each connection in a separate process
– This will mean that the logic serving each request will be “sandboxed” away from the main server process
Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.12
Sockets With Protection (each connection has own process)
Create Client Socket
Connect it to server (host:port)
Connection Socket
write request read response
Close Client Socket
Create Server Socket
Bind it to an Address (host:port)
Listen for Connection
Accept syscall() Connection Socket
Close Connection Socket
Wait for child
Close Listen Socket read request
write response
Close Connection Socket
Close Server Socket
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Server Protocol (v2)
// Socket setup code elided…
while (1) {
// Accept a new client connection, obtaining a new socket int conn_socket = accept(server_socket, NULL, NULL);
if (pid == 0) { close(server_socket); serve_client(conn_socket); close(conn_socket); exit(0);
} else { close(conn_socket); wait(NULL);
close(server_socket);
listen(server_socket, MAX_QUEUE);
pid_t pid = fork();
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.14
Concurrent Server
• So far, in the server:
– Listen will queue requests
– Buffering present elsewhere
– But server waits for each connection to terminate before servicing the next
• A concurrent server can handle and service a new connection before the previous client disconnects
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Sockets With Protection and Concurrency
Create Client Socket
Connect it to server (host:port)
Connection Socket
write request read response
Close Client Socket
Create Server Socket
Bind it to an Address (host:port)
Listen for Connection
Accept syscall() Connection Socket
Close Connection Socket
Close Server Socket
Close Listen Socket read request
write response
Close Connection Socket
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Server Protocol (v3)
// Socket setup code elided… listen(server_socket, MAX_QUEUE); while (1) {
// Accept a new client connection, obtaining a new socket int conn_socket = accept(server_socket, NULL, NULL); pid_t pid = fork();
if (pid == 0) {
close(server_socket); serve_client(conn_socket); close(conn_socket); exit(0);
} else { close(conn_socket); //wait(NULL);
close(server_socket);
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.17
Server Address: Itself
struct addrinfo *setup_address(char *port) {
struct addrinfo *server;
struct addrinfo hints;
memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE;
getaddrinfo(NULL, port, &hints, &server);
return server;
Accepts any connections on the specified port
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Client: Getting the Server Address
struct addrinfo *lookup_host(char *host_name, char *port) {
struct addrinfo *server;
struct addrinfo hints;
memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
int rv = getaddrinfo(host_name, port_name,
&hints, &server);
if (rv != 0) {
printf(“getaddrinfo failed: %s\n”, gai_strerror(rv));
return NULL;
return server;
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Concurrent Server without Protection
• Spawn a new thread to handle each connection
• Main thread initiates new client connections without waiting for previously spawned threads
• Why give up the protection of separate processes? – More efficient to create new threads
– More efficient to switch between threads
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Sockets with Concurrency, without Protection
Create Client Socket
Connect it to server (host:port)
Connection Socket
write request read response
Close Client Socket
Create Server Socket
Bind it to an Address (host:port)
Listen for Connection
Accept syscall() Connection Socket
pthread_create
Main Thread
Close Server Socket
Spawned Thread
read request write response
Close Connection Socket
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Thread Pools: More Later!
• Problem with previous version: Unbounded Threads
– When web-site becomes too popular – throughput sinks
• Instead, allocate a bounded “pool” of worker threads, representing the maximum level of multiprogramming
Maste r Threa d
master() {
allocThreads(worker,queue);
while(TRUE) {
con=AcceptCon();
Enqueue(queue,con);
wakeUp(queue);
Thread Pool
worker(queue) {
while(TRUE) {
con=Dequeue(queue);
if (con==null)
sleepOn(queue);
Joseph & Kubiatowicz CS162 © UCB Spring 2022
ServiceWebPage(con);
Computers (Cars/other things) in the news
• Y2K22? January 2022 saw a whole new class of bugs:
– Well, welcome to Y2K22 bugs. If you write a date/time in YYMMDDHHMM format (which is year, month, day, hour, and minute), it now exceeds 31 bits!
– Meaning – if they use unsigned instead of signed 32-bit numbers it breaks! – So, a bunch of systems are now broken:
Exchange (Email)
Email Security/Firewall
Honda Car Clocks/Navigation Systems
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Recall:The Process Control Block
• Kernel represents each process as a process control block (PCB)
– Status (running, ready, blocked, …)
– Register state (when not ready)
– Process ID (PID), User, Executable, Priority, … – Execution time, …
– Memory space, translation, …
• Kernel Scheduler maintains a data structure containing the PCBs
– Give out CPU to different processes – This is a Policy Decision
• Give out non-CPU resources – Memory/IO
– Another policy decision
Process Control Block
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Recall:What’s in an Open File Description?
For our purposes, the two most important things are: • Where to find the file data on disk
• The current position within the file
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.25
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
Thread’s Regs
Suppose that we execute open(“foo.txt”)
and that the result is 3
Abstract Representation of a Process
Address Space (Memory)
File Descriptors 3
File: foo.txt Position: 0
Joseph & Kubiatowicz CS162 © UCB Spring 2022
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
Thread’s Regs
Suppose that we execute open(“foo.txt”)
and that the result is 3
Next, suppose that we execute read(3, buf, 100)
and that the result is 100
Abstract Representation of a Process
Address Space (Memory)
File Descriptors 3
File: foo.txt Position: 0
Joseph & Kubiatowicz CS162 © UCB Spring 2022
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
Thread’s Regs
Suppose that we execute open(“foo.txt”)
and that the result is 3
Next, suppose that we execute read(3, buf, 100)
and that the result is 100
Abstract Representation of a Process
Address Space (Memory)
File Descriptors 3
File: foo.txt Position: 100
Joseph & Kubiatowicz CS162 © UCB Spring 2022
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
Thread’s Regs
Suppose that we execute open(“foo.txt”)
and that the result is 3
Next, suppose that we execute read(3, buf, 100)
and that the result is 100
Finally, suppose that we execute close(3)
Abstract Representation of a Process
Address Space (Memory)
File Descriptors 3
File: foo.txt Position: 100
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Instead of Closing, let’s fork()!
Thread’s Regs
• File descriptor is copied
• Open file description … is aliased
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
File Descriptors 3
File Descriptors 3
File: foo.txt Position: 100
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Open File Description is Aliased
read(3, buf, 100)
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
File Descriptors 3
File Descriptors 3
File: foo.txt Position: 100
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Open File Description is Aliased
read(3, buf, 100)
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
File Descriptors 3
File Descriptors 3
File: foo.txt Position: 200
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Open File Description is Aliased
read(3, buf, 100)
read(3, buf, 100)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
File Descriptors 3
File Descriptors 3
File: foo.txt Position: 200
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Open File Description is Aliased
read(3, buf, 100)
read(3, buf, 100)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
File Descriptors 3
File Descriptors 3
File: foo.txt Position: 300
Joseph & Kubiatowicz CS162 © UCB Spring 2022
File Descriptor is Copied
read(3, buf, 100)
read(3, buf, 100)
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
Open File Description
File Descriptors 3
File Descriptors 3
File: foo.txt Position: 300
Joseph & Kubiatowicz CS162 © UCB Spring 2022
File Descriptor is Copied
read(3, buf, 100)
read(3, buf, 100)
Open file description remains alive until … no file descriptors in
any process refer to
Open File Description
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
User Space Kernel Space
Not shown: Initially contains 0, 1, and 2 (stdin, stdout, stderr)
File Descriptors 3
File Descriptors
File: foo.txt Position: 300
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Why is Aliasing the Open File Description a Good Idea?
• It allows for shared resources between processes
2/1/2022 Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.37
Recall: In POSIX, Everything is a “File”
• Identical interface for: – Files on disk
– Devices (terminals, printers, etc.)
– Regular files on disk
– Networking (sockets)
– Local interprocess communication (pipes, sockets)
• Based on the system calls open(), read(), write(), and close()
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Example: Shared Terminal Emulator
• When you fork() a process, the parent’s and child’s printf outputs go to the same terminal
Joseph & Kubiatowicz CS162 © UCB Spring 2022 Lec 5.39
Example: Shared Terminal Emulator
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
User Space Kernel Space
Terminal Emulator
File Descriptors
File Descriptors
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Example: Shared Terminal Emulator
User Space Kernel Space
Terminal Emulator
Thread’s Regs
Address Space (Memory)
Thread’s Regs
Address Space (Memory)
File Descriptors
File Descriptors
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Example: Shared Terminal Emulator
Thread’s Regs
User Space Kernel Space
Address Space (Memory)
If one process closes
stdin (0), it remains … open in other
Terminal Emulator
Address Space (Memory)
File Descriptors
File Descriptors
Joseph & Kubiatowicz CS162 © UCB Spring 2022
Thread’s Regs
Other Examples
• Shared network connections after fork()
– Allows handling each connection in
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com