计算机代考程序代写 file system I/O and Streams

I/O and Streams
CSci4061: Introduction to Operating Systems

October 5, 2021
Computer Science & Engineering, University of Minnesota
1

Last lecture
• Address space
• kernel-user split
• Heap vs. Stack
• Memory allocation: malloc and free • Memory Problems
• Segmentation Faults • Memory Leaks
2

Input/Output (I/O)

Input/Output (I/O)
• Programs need to read and/or write data
• I/O refers to the transfer of data either to or from a storage medium
(typically hardware devices)
• Hardware devices:
• Keyboard, monitor, printer, touchscreen, hard disk, flash drive, network card
• Programs typically access devices through the OS (device drivers)
• Device drivers—part of kernel in monolithic kernels
• Special kernel programs that control devices • Hide complexity of device
• Protect device from unauthorized access
• Tend to have more bugs and vulnerabilities
3

OS Interface to Devices
• All I/O is basically done through device drivers
• OS provides system call interface to applications to perform I/O • The I/O interface/channel could be heavily device-dependent
• E.g.: different for keyboard (expansion bus) vs. disk (SCSI bus) • Thousands of devices
• Is this a problem? How to address it?
4

Unix I/O: Files
• All devices are mapped to files
• Everything is a file—another example of OS virtualization!
• Uniform interface for I/O:
• A single set of system calls (open, close, read, write) with some additional ones
• Actual implementation of the system calls depends on device type
• Realized through indirect function calls (like virtual functions in C++)
• Benefits:
• Hide the complexity of actual device properties
• Programmers only need to learn only a few system calls
5

Further abstraction: Streams for I/O
• A stream is a fairly abstract, high-level concept
• A file is a low-level primitive
• Both file and stream represent communication channels
• It is a sequence of bytes with functions to take bytes out of one end, and put
bytes into the other end.
• What can be an end of a stream?
6

Differentiating Files and Streams
• Steams represent communication channels to not only files but also processes • Files are ends of streams
• Stream operations are abstraction over file operations
• Certain operations on special files may not be available on streams
• If you want to do control operations that are specific to a particular kind of
device, you must use a file descriptor
• But the stream interface also provides powerful formatted input and output
functions
• You will often use streams • Compare open and fopen?
• fd = open(“/etc/passwd”, O_RDONLY); • FILE *f = fopen(“/etc/passwd”, “r”);
7

I/O on Streams

Unix Streams
• Unix FILE object
• A structure containing information about a file • Built on top of low-level file descriptors
• Defined in
• User-space library
• The type of streams: “FILE * ” —Isn’t the naming confusing? • Pointer to a FILE object
• Three default streams
• stdin • stdout • stderr
8

Definition of FILE
9

Inside a FILE
• Points to actual file
• Current offset
• Buffering (open for write)
• Internal character buffer of size BUFSIZE
• Writes are done to in-memory buffer
• When buffer is full (BUFSIZE), write out buffer
• Or if line-buffered stream (e.g. stdout) when ‘\n’ is written
10

Inside a FILE (cont’d)
• Buffering (open for read)
• Reads are done from in-memory buffer
• When buffer is empty, we read a chunk of BUFSIZE
• Why is the use of an internal buffer for I/O advantageous?
• Fewer underlying system calls (read and write) are needed
• Buffering happens in both user space and kernel space
• Why?
11

FFLUSH buffers
• int fflush(FILE * stream);
• fflush() forces a write of all user-space buffered data • Why?
• See data immediately (OS auto-flushes K sec)
• Buffered data for read?
• Discarded
12

A bit more about buffer flushing
• fflush() flushes only the user-space buffers
• How about buffers in the kernel space?—How to ensure that the data is
physically stored on disk?
• Use fsync()
13

Direct uses of standard streams
FILE * stdin; // Standard input stream.
FILE * stdout; // Standard output stream.
FILE * stderr; // Standard error output stream.
• They are global • Typical cases
• stdin (fd=0): keyboard • stdout (fd=1): terminal • stderr (fd=2): terminal
• Standard streams can be redirected and inherited
• E.g., 2>&1
14

How to open/close non-standard streams
#include
// NULL on failure, errno also set
FILE *fopen (const char *filename, const char *mode); // mode = “r”, “w”, “a” (others)
int fclose (FILE *stream);
15

Error Handling
• I/O requests can fail for many reasons, e.g., buffers overflow or disk crash
• I/O requests usually return an error code
• Details about errno: man errno
• Unix systems set the global variable “errno” to one of the error codes in errno.h. Any issue?
• Thread-safe (per-thread error number)
FILE *f = fopen(“ERRTEST.txt”, “w”); if (f == NULL) {
printf(“fopen failed, errno = %d\n”, errno);
return errno; }
16

Common Classes of Streams

Character-based I/O
#include
// SUBSET of calls
int getc (FILE * stream);
int putc (int char, FILE * stream); // goes back 1 character
int ungetc (int char, FILE * stream);
Which header are things located?
• use man pages • use grep
17

Use “man”
• The manual pages are a set of pages that explain every command
• SYNOPSIS
• DESCRIPTION
• Sometime examples • Etc.
• Command
• man
18

String-based I/O
char * fgets (char * buf, int nsize, FILE * inf);
• When read stops?
• Reads nsize-1 characters
• Reads up to a newline ‘\n’
• Reads EOF into buf
• It does not care if it reads \0 bytes.
• Appends ‘\0’ at the end of buf
int fputs (const char * buf, FILE * outf);
• Outputs buf – better be ‘\0’ terminated • Failure?
• Use man
• Error codes in return value
19

Importance of newline ‘\n’
stdin is a line-oriented device
• ‘\n’ (EOL), read is done when ‘\n’ is seen
• ˆD is EOF
• fgets on stdin will include ‘\n’ in the string! • gets does not
20

Example of String-based I/O
void main () {
char buf[4];
fgets(buf, sizeof(buf), stdin); printf(“buf=%s\n”, buf);
}
Q: what would be the output of:
• “$./test 12”?
• “$./test 1234”? • “$./test 12345”?
21

What about I/O for data types?
• For example: Non-character or string data • Answer: Formatted I/O!
• Formatted output and input strings
int fprintf (FILE * outf, const char * fmt, )
int sprintf (char * string, const char * fmt, )
More: printf, snprintf, asprintf, dprintf, vprintf, vfprintf, vsprintf, vsnprintf, vasprintf, vdprintf
22

Formatting outputs
• Formats: %d, %f %c, %x, %s, %p, and many others
• Again, use man
• FORMAT fields must match in type and number!
23

Example of sprintf
void main(int argc, char **argv) {
char *buf;
sprintf(buf, “%d, %s”, argc, argv[0]);
}
Any potential problem?
24

Example of sprintf
void main(int argc, char **argv) {
char *buf;
sprintf(buf, “%d, %s”, argc, argv[0]);
}
Any potential problem?
buf in sprintf must be first allocated.
24

Another Example of sprintf
void main(int argc, char **argv) {
char *buf;
sprintf(buf, “%d, %s, %s”, argc, argv[0]);
}
Any potential problem?
25

Another Example of sprintf
void main(int argc, char **argv) {
char *buf;
sprintf(buf, “%d, %s, %s”, argc, argv[0]);
}
Any potential problem?
Format fields do not match—out-of-bound read, infoleak, etc.
25

Formatting Input
fscanf (FILE *in, const char *fmt, );
sscanf (FILE *in, const char *fmt, );
More: scanf, vfscanf, vscanf, vsscanf
26

Example of formatting inputs
int i1, i2;
float flt;
char str1[10], str2[10];
char *str1, *str2; // problem?
// From string
scanf (“11 12 34.07 keith ben”, “%d %d %f %s %s”,
&i1, &i2, &flt, str1, str2);
// From stdin
scanf (stdin, “%d %d %f %s %s”,
&i1, &i2, &flt, str1, str2);
What if I forget to allocate or use ‘&’?
27

Binary I/O
• Files may contain binary data, e.g., 12345 instead of “12345” • Reading it as “12345” uses more memory and is slow
• Parsing, translation, etc.
• Better way: Use binary I/O to read/write fewer
fread/fwrite (void *buffer, size_t size, size_t nitems, FILE *f);
28

Binary I/O Example
typedef struct {
int birth; // 8 digits int phone; // 10 digits
} info_t;
info_t mine = {12345678, 538493747};
// write as string
fprintf (F, “%d %d”, mine.birth, mine.phone); // write as binary
fwrite ((void *)&mine, sizeof (info_t), 1, F);
• size of ascii file: 12345678 538493747 = (8+10) * 1 bytes • size of binary file: ?
29

Random I/O—Positioning
• Thusfar, all I/O was assumed to be sequential
• fgets, fgets, . . . , returns consecutive lines
• Current file ptr advances sequentially
• Sometime we need random I/O—read/write a specific location
• Think of any examples of applications that access data in a non-sequential
pattern?
30

Example
• Suppose a file “residents” contains occupants of a hotel
• Line i is the occupant of room i
• Simple assumption: lines (even empties) are padded out to the same length,
MAX_LINE
•
• •
•
•
• …
31

Who is in room i?
• Sequential I/O
• How much must we read?
• Random I/O
• Advance file offset pointer w/o reading
#include
int fseek (FILE *stream, long off, int whence);
32

How to actually find who is in room i?
33

How to actually find who is in room i?
// move offset from the beginning
fseek (F, (i-1) * MAX_LINE, SEEK_SET); // do the read
fgets (F, …);
// other call: what is the current file offset?
long ftell (FILE *stream);
33

Summary
• Devices as files • I/O on streams
• Abstraction facilitating programming
• Three standard streams; what are they and how to use? • Open/close non-standard streams
• Common classes of streams
• Character-based I/O
• String-based I/O—be careful about formatting • Binary I/O
• Random I/O
34

Quiz
• How do you differ streams from file descriptors?
• User programs need to use system calls if it wishes to perform an I/O?
• The OS returns _____ in an event of an I/O failure.
• Would gets() stop the reading upon \0? Does it read and take \n?
• What syscall() should be used to move the seek to an offset while accessing a
file in a non-sequential manner?
35

Next lecture
• File I/O operations
• File Descriptors and redirection • Pipes and FIFOs
36

Related Posts