COMP 3000 Operating Systems
Inter-Process Communication and Concurrency (part 1)
Lianying Zhao
Recap: Ways to Pass Data to a Program
• Command line arguments • Standard I/O
• Files
• Network
• Pipes (e.g., FIFO)
• Here, one end is a process (program)
• The other end (which can be a process) represents a third party/user COMP 3000 (Winter 2021) 2
How Do Processes Talk to Each Other?
• If they don’t, we won’t have concurrency problems • But how to make the work done?
• Signals
• But it’s more like notification (hence signaling)
• Pipes
• Unidirectional
• Sockets
• Message queues (we will not discuss them) • Shared memory!
COMP 3000 (Winter 2021) 3
Synchronous & Asynchronous
• AKA, blocking & non-blocking
• Applicable in function invocation and communication in general
• Synchronous (blocking)
• Yield CPU (the process is no longer running)
• Returns when completed (e.g., data sent/received)
• Asynchronous (non-blocking)
• Returns right away
• Needs a mechanism to check the results
1. Polling
2. Callbacks based on
events/signals/messages/etc.
COMP 3000 (Winter 2021) 4
Pipes
• Unidirectional data channel
• One end for read and one end for write (two FDs)
• Backed up by an in-memory buffer • Size is determined by implementation
• Named pipes and unnamed (regular) pipes
COMP 3000 (Winter 2021) 5
Pipes (CONT’d)
• Unnamed pipes
• Created with the pipe() system call
• Only between forked processes or within the same process, i.e., there’s no name to refer to outside.
• Gone when involved processes are terminated
• Named pipes (FIFOs)
• Backed up by file systems (recall: one type defined in the inode)
• i.e., you see a file in a directory
• Created with the mkfifo()system call
• Supports different processes with access to the FIFO file
COMP 3000 (Winter 2021) 6
Sockets
• Just endpoints for communication • Not necessarily for Internet
• Created with the socket() system call • Supports various protocols
• E.g.,AF_INET(Internet),AF_UNIX/AF_LOCAL(local) • Socket programming
• bind(), listen(), accept(), etc. COMP 3000 (Winter 2021)
Source: www.geeksforgeeks.org
7
Performance Overhead!
• All the aforementioned mechanisms can be considered “messaging” • A third party is involved
• Messages are copied from one end to the other
• Overhead caused by data moving • Also, what about multiple parties
• Overhead caused by context switch • Don’t only think about data size
COMP 3000 (Winter 2021) 8
Shared Memory
• It seems the ultimate solution is sharing
• Can be generalized to shared resources
• Shared memory: the mmap() system call • Flag:MAP_SHARED
Review of mmap()
1. Allocate memory (tut2)
MAP_PRIVATE|MAP_ANONYMOUS
2. Allocate shared memory (tut6)
MAP_SHARED|MAP_ANONYMOUS
3. Map a file into memory (tut5)
MAP_SHARED
• But don’t forget: “There is no such thing as a free lunch”
COMP 3000 (Winter 2021) 9
Recall: the Storage Hierarchy
• What is shared may not be seen the same by all parties
• L1 and L2 are per-core • L3 is shared
• We admit and accept it, and it can be alleviated with proper configuration
COMP 3000 (Winter 2021)
Source: www.flashmemorysummit.com 10
Then How to Coordinate Updates?
• What is the main cause of concurrency problems? • Shared data!
• If all parties are allowed to update the shared data (needed in most cases), it should be OK, if:
• Updates from each party are carried out as intended • And there are rules
• In today’s multitasking environments, satisfying the former condition is nontrivial
COMP 3000 (Winter 2021) 11
Lack of Atomicity
• How update to the shared data can go unintended: • It involves multiple operations, as opposed to one
• So, for example:
counter = counter + 1;
mov 0x8049a1c, %eax
add $0x1, %eax
mov %eax, 0x8049a1c
COMP 3000 (Winter 2021) 12
Race Condition
• When multiple concurrent tasks/threads are accessing shared resources roughly at the same time leading to an undesired outcome
• The results depend on the timing • Non-deterministic
• Critical section: the piece of code that updates a shared resource • Seemingly two ways out: atomicity or mutual exclusion
• which is about the same goal (one achieves the other)
COMP 3000 (Winter 2021) 13
Atomic Operations
• Indivisible
• All or nothing
• No in-between state visible
• To enforce atomicity, in case of interruption
• Roll back
• Information about the previous state must be available before discarding incomplete operations
• Recall: crash consistency for file systems • E.g., Journaling
COMP 3000 (Winter 2021) 14
COMP 3000 Operating Systems
Misc.
COMP 3000 (Winter 2021) 15
More on Mounting File Systems
• The permission bits
• File system-specific *
• E.g., Access Control Lists (ACLs)
• How come I can mount all/many kinds of file system on Linux and still see the same view?
• Possibility: the driver fakes it • Mount options
COMP 3000 (Winter 2021) 16
Using the dd Command
• Experimenting with real devices can be risky • So let’s use a “virtual” version
• dd is a command-line utility to copy/convert data
• Always involves an input file (if) and an output file (of)
• Not necessarily regular files
• Block devices (e.g., /dev/sda)
• Other special files (e.g., /dev/null, /dev/random, /dev/zero)
COMP 3000 (Winter 2021) 17
dd vs. cp
• Well, they both copy files, so…
• They have different positionings
• cp: works at the granularity of files
• Can handle multiple files/directories
• dd: file I/O, more control over data is handled • Position control: seek (of), skip (if)
• Conversion:e.g.,encoding
• Analogy: dd is like a file-based pipeline
COMP 3000 (Winter 2021) 18