代写代考 CSE4100: System Programming

Sogang University
System-Level I/O
CSE4100: System Programming
Youngjae Kim (PhD)

https://sites.google.com/site/youkim/home
Distributed Computing and Operating Systems Laboratory (DISCOS)
https://discos.sogang.ac.kr
Office: R911, E-mail:
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
¢ Unix I/O
¢ RIO (robust I/O) package
¢ Metadata, sharing, and redirection ¢ Standard I/O
¢ Closing remarks
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Unix I/O Overview
¢ A Linux file is a sequence of m bytes:
§B0,B1,….,Bk ,….,Bm-1
¢ Cool fact: All I/O devices are represented as files: §/dev/sda2 (/usrdiskpartition)
§/dev/tty2 (terminal)
¢ Even the kernel is represented as a file:
§ /boot/vmlinuz-3.13.0-55-generic (kernel image)
§ /proc (kernel data structures)
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Unix I/O Overview
¢ Elegant mapping of files to devices allows kernel to
export simple interface called Unix I/O:
§ Opening and closing files
§ open()and close()
§ Reading and writing a file
§ read() and write()
§ Changing the current file position (seek)
§ indicates next offset into file to read or write § lseek()
Current file position = k
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
File Types
¢ Each file has a type indicating its role in the system § Regular file: Contains arbitrary data
§ Directory: Index for a related group of files
§ Socket: For communicating with a process on another machine
¢ Other file types beyond our scope § Named pipes (FIFOs)
§ Symbolic links
§ Character and block devices
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Regular Files
¢ A regular file contains arbitrary data
¢ Applications often distinguish between text files and binary files
§ Text files are regular files with only ASCII or Unicode characters § Binary files are everything else
§ e.g., object files, JPEG images
§ Kernel doesn’t know the difference!
¢ Text file is sequence of text lines
§ Text line is sequence of chars terminated by newline char (‘\n’)
§ Newline is 0xa, same as ASCII line feed character (LF)
¢ End of line (EOL) indicators in other systems
§ Linux and Mac OS: ‘\n’ (0xa) § line feed (LF)
§ Windows and Internet protocols: ‘\r\n’ (0xd 0xa) § Carriage return (CR) followed by line feed (LF)
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Directories
¢ Directory consists of an array of links
§ Each link maps a filename to a file
¢ Each directory contains at least two entries
§.(dot)is alinktoitself
§ .. (dot dot) is a link to the parent directory in the directory hierarchy (next slide)
¢ Commands for manipulating directories § mkdir: create empty directory
§ ls: view directory contents
§ rmdir: delete empty directory
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Directory Hierarchy
¢ All files are organized as a hierarchy anchored by root directory
named / (slash)
bin/ dev/ etc/
home/ usr/
bash tty1 group passwd droh/ bryant/ include/ bin/
hello.c stdio.h sys/ vim unistd.h
¢ Kernel maintains current working directory (cwd) for each process § Modified using the cd command
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
¢ Locations of files in the hierarchy denoted by pathnames
§ Absolute pathname starts with ‘/’ and denotes path from root § /home/droh/hello.c
§ Relative pathname denotes path from current working directory § ../home/droh/hello.c
bin/ dev/ etc/
/ cwd: /home/bryant
home/ usr/
bash tty1 group passwd droh/ bryant/ include/ bin/
hello.c stdio.h sys/ vim
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Opening Files
¢ Opening a file informs the kernel that you are getting ready to access that file
int fd; /* file descriptor */
if ((fd = open(“/etc/hosts”, O_RDONLY)) < 0) { perror("open"); ¢ Returns a small identifying integer file descriptor § fd == -1 indicates that an error occurred ¢ Each process created by a Linux shell begins life with three open files associated with a terminal: § 0: standard input (stdin) § 1: standard output (stdout) § 2: standard error (stderr) Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University Closing Files ¢ Closing a file informs the kernel that you are finished accessing that file int fd; /* file descriptor */ int retval; /* return value */ if ((retval = close(fd)) < 0) { perror("close"); ¢ Closing an already closed file is a recipe for disaster in threaded programs (more on this later) ¢ Moral: Always check return codes, even for seemingly benign functions such as close() Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University Reading Files ¢ Reading a file copies bytes from the current file position to memory, and then updates file position char buf[512]; int fd; /* file descriptor */ int nbytes; /* number of bytes read */ /* Open file fd ... */ /* Then read up to 512 bytes from file fd */ if ((nbytes = read(fd, buf, sizeof(buf))) < 0) { perror("read"); exit(1); } ¢ Returns number of bytes read from file fd into buf § Return type ssize_t is signed integer § nbytes < 0 indicates that an error occurred § Short counts (nbytes < sizeof(buf) ) are possible and are not errors! Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University Writing Files ¢ Writing a file copies bytes from memory to the current file position, and then updates current file position char buf[512]; int fd; /* file descriptor */ int nbytes; /* number of bytes read */ /* Open the file fd ... */ /* Then write up to 512 bytes from buf to file fd */ if ((nbytes = write(fd, buf, sizeof(buf)) < 0) { perror("write"); exit(1); } ¢ Returns number of bytes written from buf to file fd § nbytes < 0 indicates that an error occurred § As with reads, short counts are possible and are not errors! Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University Simple Unix I/O example ¢ Copying stdin to stdout, one byte at a time #include "csapp.h" int main(void) while(Read(STDIN_FILENO, &c, 1) != 0) Write(STDOUT_FILENO, &c, 1); exit(0); } Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University On Short Counts ¢ Short counts can occur in these situations: § Encountering (end-of-file) EOF on reads § Reading text lines from a terminal § Reading and writing network sockets ¢ Short counts never occur in these situations: § Reading from disk files (except for EOF) § Writing to disk files ¢ Best practice is to always allow for short counts. Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University ¢ Unix I/O ¢ RIO (robust I/O) package ¢ Metadata, sharing, and redirection ¢ Standard I/O ¢ Closing remarks Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University The RIO Package ¢ RIO is a set of wrappers that provide efficient and robust I/O in apps, such as network programs that are subject to short counts ¢ RIO provides two different kinds of functions § Unbuffered input and output of binary data § rio_readn and rio_writen § Buffered input of text lines and binary data § rio_readlineb and rio_readnb § Buffered RIO routines are thread-safe and can be interleaved arbitrarily on the same descriptor ¢ Refer to à csapp.c and csapp.h Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University Unbuffered RIO Input and Output ¢ Same interface as Unix read and write ¢ Especially useful for transferring data on network sockets #include "csapp.h" ssize_t rio_readn(int fd, void *usrbuf, size_t n); ssize_t rio_writen(int fd, void *usrbuf, size_t n); Return: num. bytes transferred if OK, 0 on EOF (rio_readn only), -1 on error § rio_readn returns short count only if it encounters EOF § Only use it when you know how many bytes to read § rio_writen never returns a short count § Calls to rio_readn and rio_writen can be interleaved arbitrarily on the same descriptor Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition Sogang University Implementation of rio_readn * rio_readn - Robustly read n bytes (unbuffered) */ ssize_t rio_readn(int fd, void *usrbuf, size_t n) { size_t nleft = n; ssize_t nread; char *bufp = usrbuf; while (nleft > 0) {
if ((nread = read(fd, bufp, nleft)) < 0) { if (errno == EINTR) /* Interrupted by sig handler return */ nread = 0; return -1; else if (nread == 0) nleft -= nread; bufp += nread; return (n - nleft); /* and call read() again */ /* errno set by read() */ /* Return >= 0 */
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Buffered RIO Input Functions
¢ Efficiently read text lines and binary data from a file partially cached in an internal memory buffer
#include “csapp.h”
void rio_readinitb(rio_t *rp, int fd);
ssize_t rio_readlineb(rio_t *rp, void *usrbuf, size_t maxlen); ssize_t rio_readnb(rio_t *rp, void *usrbuf, size_t n);
Return: num. bytes read if OK, 0 on EOF, -1 on error
§ rio_readlineb reads a text line of up to maxlen bytes from file fd and stores the line in usrbuf
§ Especially useful for reading text lines from network sockets § Stopping conditions
§ maxlen bytes read § EOF encountered
§ Newline (‘\n’) encountered Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Buffered RIO Input Functions (cont)
#include “csapp.h”
void rio_readinitb(rio_t *rp, int fd);
ssize_t rio_readlineb(rio_t *rp, void *usrbuf, size_t maxlen); ssize_t rio_readnb(rio_t *rp, void *usrbuf, size_t n);
Return: num. bytes read if OK, 0 on EOF, -1 on error
§ rio_readnb reads up to n bytes from file fd § Stopping conditions
§ maxlen bytes read
§ EOF encountered
§ Calls to rio_readlineb and rio_readnb can be interleaved
arbitrarily on the same descriptor
§ Warning: Don’t interleave with calls to rio_readn
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Buffered I/O: Implementation ¢ For reading from file
¢ File has associated buffer to hold bytes that have been read from file but not yet read by user code
¢ Layered on Unix file:
already read
rio_bufptr
Buffered Portion
Current File Position
not in buffer
already read
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Buffered I/O: Declaration ¢ All information contained in struct
already read
rio_bufptr
typedef struct {
int rio_fd;
/* descriptor for this internal buf */ /* unread bytes in internal buf */
/* next unread byte in internal buf */
int rio_cnt;
char *rio_bufptr;
char rio_buf[RIO_BUFSIZE]; /* internal buffer */
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
RIO Example
¢ Copying the lines of a text file from standard input to standard output
#include “csapp.h”
main(int argc, char **argv)
rio_t rio;
char buf[MAXLINE];
Rio_readinitb(&rio, STDIN_FILENO);
while((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0)
Rio_writen(STDOUT_FILENO, buf, n); exit(0);
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
File Metadata
¢ Metadata is data about data, in this case file data
¢ Per-file metadata maintained by kernel
§ accessed by users with the stat and fstat functions
/* Metadata returned by the stat and fstat functions */
struct stat {
unsigned long
unsigned long
st_blksize;
st_blocks;
/* Device */
/* inode */
/* Protection and file type */ /* Number of hard links */
/* User ID of owner */
/* Group ID of owner */
/* Device type (if inode device) */ /* Total size, in bytes */
/* Blocksize for filesystem I/O */ /* Number of blocks allocated */ /* Time of last access */
/* Time of last modification */ /* Time of last change */
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Example of Accessing File Metadata
int main (int argc, char **argv) {
statcheck.c
struct stat stat; char *type, *readok;
Stat(argv[1], &stat);
if (S_ISREG(stat.st_mode))
/* Determine file type */
type = “regular”;
else if (S_ISDIR(stat.st_mode))
type = “directory”; else
type = “other”;
if ((stat.st_mode & S_IRUSR)) /* Check read access */
readok = “yes”;
readok = “no”;
printf(“type: %s, read: %s\n”, type, readok); exit(0);
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition
linux> ./statcheck statcheck.c type: regular, read: yes linux> chmod 000 statcheck.c linux> ./statcheck statcheck.c type: regular, read: no linux> ./statcheck ..
type: directory, read: yes

Sogang University
How the Unix Kernel Represents Open Files
¢ Two descriptors referencing two distinct open files. Descriptor 1 (stdout) points to terminal, and descriptor 4 points to open disk file
Descriptor table
[one table per process]
fd 1 fd 2 fd 3 fd 4
Open file table
[shared by all processes] File A (terminal)
v-node table
[shared by all processes]
File access
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition
File B (disk)
File access

Sogang University
File Sharing
¢ Two distinct descriptors sharing the same disk file through two distinct open file table entries
§ E.g., Calling open twice with the same filename argument
Descriptor table
[one table per process]
fd 1 fd 2 fd 3 fd 4
Open file table
[shared by all processes] File A (disk)
v-node table
[shared by all processes]
File access
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition
File B (disk)

Sogang University
How Processes Share Files: fork ¢ A child process inherits its parent’s open files
§ Note: situation unchanged by exec functions (use fcntl to change) ¢ Beforeforkcall:
Descriptor table
[one table per process]
fd 1 fd 2 fd 3 fd 4
Open file table
[shared by all processes] File A (terminal)
v-node table
[shared by all processes]
File access
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition
File B (disk)
File access

Sogang University
How Processes Share Files: fork
¢ A child process inherits its parent’s open files ¢ Afterfork:
§ Child’s table same as parent’s, and +1 to each refcnt
Descriptor table
[one table per process] Parent
fd 0 fd 1 fd 2 fd 3 fd 4
Child fd 0
fd 1 fd 2 fd 3 fd 4
Open file table
[shared by all processes] File A (terminal)
v-node table
[shared by all processes]
File access
File B (disk)
File access
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
I/O Redirection
¢ Question: How does a shell implement I/O redirection?
linux> ls > foo.txt
¢ Answer: By calling the dup2(oldfd, newfd) function
§ Copies (per-process) descriptor table entry oldfd to entry newfd
Descriptor table
before dup2(4,1)
fd0 fd1 fd2 fd3 fd4
Descriptor table
after dup2(4,1)
fd0 fd1 fd2 fd3 fd4
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
I/O Redirection Example
¢ Step #1: open file to which stdout should be redirected § Happens in child executing shell code, before exec
Descriptor table
[one table per process]
fd 1 fd 2 fd 3 fd 4
Open file table
[shared by all processes] File A
v-node table
[shared by all processes]
File access
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition
File access

Sogang University
I/O Redirection Example (cont.)
¢ Step #2: call dup2(4,1)
§ cause fd=1 (stdout) to refer to disk file pointed at by fd=4
Descriptor table
[one table per process]
fd 1 fd 2 fd 3 fd 4
Open file table
[shared by all processes] File A
v-node table
[shared by all processes]
File access
File access
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Standard I/O Functions
¢ The C standard library (libc.so) contains a collection of higher-level standard I/O functions
§ Documented in Appendix B of K&R
¢ Examples of standard I/O functions:
§ Opening and closing files (fopen and fclose)
§ Reading and writing bytes (fread and fwrite)
§ Reading and writing text lines (fgets and fputs)
§ Formatted reading and writing (fscanf and fprintf)
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Standard I/O Streams
¢ Standard I/O models open files as streams
§ Abstraction for a file descriptor and a buffer in memory ¢ C programs begin life with three open streams
(defined in stdio.h)
§ stdin (standard input)
§ stdout (standard output) § stderr (standard error)
#include
extern FILE *stdin; /* standard input (descriptor 0) */ extern FILE *stdout; /* standard output (descriptor 1) */ extern FILE *stderr; /* standard error (descriptor 2) */
int main() {
fprintf(stdout, “Hello, world\n”);
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Buffered I/O: Motivation
¢ Applications often read/write one character at a time § getc, putc, ungetc
§ gets, fgets
§ Read line of text one character at a time, stopping at newline
¢ Implementing as Unix I/O calls expensive § read and write require Unix kernel calls
§ > 10,000 clock cycles
¢ Solution: Buffered read
§ Use Unix read to grab block of bytes
§ User input functions take one byte at a time from buffer
§ Refill buffer when empty Buffer
already read
Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Sogang University
Buffering in Stan

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts