COMP 3000 Operating Systems
File Systems and Storage Management (part 4)
Lianying Zhao
The Block Device “Layer”
• The actual media
• Has its own drivers, e.g., USB Mass Storage
• May even involve multiple layers
• Block size can differ from the file system block size
• Technically, file systems can “reside” on any block devices
• Performance also matters
• The write() system call usually doesn’t immediately write to the block device • Thesynccommand
COMP 3000 (Winter 2021) 2
How Do We Access Devices?
• Again, special files
• Mostly /dev/*
• Kernel-mode code behind each special file • Special files ≢ files on a special file system
• E.g., \\.\PHYSICALDRIVE0 (Windows)
• E.g., /dev/sda (Linux)
• Evolution of node generation in /dev • Manually generated hardcoded nodes • devfs
• udev
COMP 3000 (Winter 2021) 3
File Systems Can be Corrupted
• All types of persistent storage share the same risk • On-disk data: lifespan is long and damages also persist
• In-RAM data: lifespan is short and can also be recreated
• What can happen:
• Media/data damage
• Failures during updates: power failure or system crash→inconsistency
• Atomicity?
crash consistency
COMP 3000 (Winter 2021) 4
A Few What-ifs
• Interrupted right in the middle of updating on-disk structures • The crash consistency problem
• Inodes are good, with missing/inconsistent data blocks • Good data blocks, with inodes missing or corrupted
• You lose directory entries
• The superblock is corrupted
COMP 3000 (Winter 2021) 5
The Lazy Approach: Let it Happen and Fix it
• The fsck tool checks: • Superblocks
• Allocation
• Link count ** • Bad blocks * • Etc.
• The lost+found directory
• Caution! File system integrity vs. data integrity
COMP 3000 (Winter 2021) 6
Journaling File Systems
• Save the need for scanning the whole file system • At the cost of some performance+storage overhead
• All changes must first be written to a log in persistent storage, before applied to the actual data storage
• Common file systems with journaling • Windows NTFS
• ext3 and ext4
COMP 3000 (Winter 2021) 7
Data Recovery?
• Again, not to be confused with file system repair
• Nor is it storage device (disk) repair…
• Precondition: there must still exist the data in some form… • Back up your data properly
COMP 3000 (Winter 2021) 8
User-space File Systems
• Why?
• Portability
• Convenience: e.g., the many programming languages • Security & stability
• FUSE = Filesystem in USErspace
• Another layer of abstraction
• /dev/fuse
• Can convert virtually anything into a file system • GmailFS?
static struct fuse_operations operations = {
.getattr = do_getattr, .readdir = do_readdir, .read = do_read,
};
COMP 3000 (Winter 2021) 9
Network File Systems
• SSHFS (a FUSE file system)
• So far, many things through SSH • SCP, SFTP
• Network tunneling
• Why: Showing remote files as local files
• NFS (not a FUSE file system)
• You need a dedicated server listening dedicated ports • Reasons for choosing it… performance, reliability, etc.
COMP 3000 (Winter 2021) 10