CITS2002 Systems Programming
1 next → CITS2002 CITS2002 schedule
System-calls and system-defined structures
Most system-calls accept integers and pointers to characters as parameters, and typically return integer values indicating their success. When more information must be passed to a system- call, or the call needs to return multiple values, we employ system-defined (C11) structures defined in operating system provided header files.
Accessing structures using pointers
We’ve seen that we can access fields of a structure using a single dot (‘.’ or fullstop). What if, instead of accessing the structure directly, we only have a pointer to a structure?
We’ve seen “one side” of this situation, already – when we passed the address of a structure to a function:
struct timeval start_time;
gettimeofday( &start_time, NULL );
The function gettimeofday(), must have been declared to receive a pointer:
extern int gettimeofday( struct timeval *time, ……);
Consider the following example, in which a pointer to a structure is returned from a function.
We now use the → operator (pronounced the ‘arrow’, or ‘points-to’ operator) to access the fields via the pointer:
#include
#include
void greeting(void) {
time_t NOW
struct tm *tm
= time(NULL);
= localtime(&NOW);
printf(“Today’s date is %i/%i/%i\n”,
tm->tm_mday, tm->tm_mon + 1, tm->tm_year + 1900);
if(tm->tm_hour < 12) {
printf("Good morning\n"); }
else if(tm->tm_hour < 17) {
printf("Good afternoon\n"); }
printf("Good evening\n");
CITS2002 Systems Programming, Lecture 16, p1, 21st September 2021.
CITS2002 Systems Programming
←prev 2 next→ CITS2002 CITS2002schedule
Another example - accessing a system's password entries
On a stand-alone Linux system, one not dependent on a network-based server to provide its user information, some local user information is (historically) stored in the textfile /etc/passwd, with each user's information stored one-per-line with fields separated by colons.
Rather than expecting every user program to parse this information correctly, Linux systems host standard XOPEN header files and libraries to conveniently provide the information.
We can iterate through this information with the help of the getpwent() function, which returns a pointer to a 'struct passwd' structure:
#include
#define __USE_XOPEN_EXTENDED #include
*pw_passwd;
*pw_gecos;
*pw_shell;
/* username */
/* user password */
/* user ID */
/* group ID */
/* user information */
/* home directory */
/* shell program */
struct passwd *getpwent(void);
int main(int argc, char *argv[]) {
struct passwd *pwd;
while((pwd = getpwent()) != NULL) { printf(“%20s\t%-30s\t%s\n”,
return 0; }
pwd->pw_name, pwd->pw_dir, pwd->pw_shell);
CITS2002 Systems Programming, Lecture 16, p2, 21st September 2021.
CITS2002 Systems Programming
←prev 3 next→ CITS2002 CITS2002schedule
Defining our own datatypes
We can further simplify our code, and more clearly identify related data by defining our own datatypes.
Recall material from Lecture-6 where structures, and arrays of structures were introduced. We preceded each structure’s name with the struct keyword, and accessed structure elements using the ‘dot’ operator.
Instead, we can use the typedef keyword to define our ownnew datatype in terms of anold (existing) datatype, and then not have to always provide the struct keyword:
// DEFINE THE LIMITS ON PROGRAM’S DATA-STRUCTURES
#define MAX_TEAMS 24 #define MAX_TEAMNAME_LEN 30 ….
typedef struct {
char teamname[MAX_TEAMNAME_LEN+1]; ….
int played;
TEAM team[MAX_TEAMS];
// +1 for null-byte
As a convention (but not a C11 requirement), we’ll define our user-defined types using uppercase names:
// PRINT EACH TEAM’S RESULTS, ONE-PER-LINE, IN NO SPECIFIC ORDER
for(int t=0 ; t
tp->played, tp->won, tp->lost, tp->drawn,
tp->bfor, tp->bagainst,
(100.0 * tp->bfor / tp->bagainst), // calculate percentage tp->points);
CITS2002 Systems Programming, Lecture 16, p3, 21st September 2021.
CITS2002 Systems Programming
←prev 4 next→ CITS2002 CITS2002schedule
Defining our own datatypes, continued
Let’s consider another example – the starting (home) and ending (destination) bustops from the CITS2002 1st project of 2015.
We starting with some of its definitions:
// GLOBAL CONSTANTS, BEST DEFINED ONCE NEAR THE TOP OF FILE
#define MAX_FIELD_LEN 100
#define MAX_STOPS_NEAR_ANYWHERE 200 // in Transperth: 184
// 2-D ARRAY OF VIABLE STOPS FOR COMMENCEMENT OF JOURNEY
char viable_home_stopid [MAX_STOPS_NEAR_ANYWHERE][MAX_FIELD_LEN]; char viable_home_name [MAX_STOPS_NEAR_ANYWHERE][MAX_FIELD_LEN]; int viable_home_metres [MAX_STOPS_NEAR_ANYWHERE];
int n_viable_homes = 0;
// 2-D ARRAY OF VIABLE STOPS FOR END OF JOURNEY
char viable_dest_stopid [MAX_STOPS_NEAR_ANYWHERE][MAX_FIELD_LEN]; char viable_dest_name [MAX_STOPS_NEAR_ANYWHERE][MAX_FIELD_LEN]; int viable_dest_metres [MAX_STOPS_NEAR_ANYWHERE];
int n_viable_dests = 0;
(After a post-project workshop) we later modified the 2-dimensional arrays to use dynamically- allocated memory and ‘pointers-to-pointers’:
// 2-D ARRAY OF VIABLE STOPS FOR COMMENCEMENT OF JOURNEY
char **viable_home_stopid char **viable_home_name int *viable_home_metres int n_viable_homes
// 2-D ARRAY OF VIABLE STOPS FOR END OF JOURNEY
char **viable_dest_stopid char **viable_dest_name int *viable_dest_metres int n_viable_dests
We can now gather the (many) related global variables into a single structure, and use typedef to define our own datatype:
// A NEW DATATYPE TO STORE 1 VIABLE STOP
typedef struct { char *stopid;
char *name;
int metres; } VIABLE;
// A VECTOR FOR EACH OF THE VIABLE home AND dest STOPS
VIABLE *home_stops = NULL;
VIABLE *dest_stops
int n_home_stops int n_dest_stops
CITS2002 Systems Programming, Lecture 16, p4, 21st September 2021.
CITS2002 Systems Programming
←prev 5 next→ CITS2002 CITS2002schedule
Finding the attributes of a file
As seen in Lecture-15, many operating systems manage their data in a file system, in particular maintaining files in a hierarchical directory structure – directories contain files and other (sub)directories.
As we saw with time-based information, we may request file- and directory information from the operating system by calling system-calls. We may employ another POSIX† function, stat(), and the
system-provided structure struct stat, to determine theattributes of each file:
#include
char *progname;
void file_attributes(char *filename) {
struct stat stat_buffer;
if(stat(filename, &stat_buffer) != 0) { // can we ‘stat’ the file’s attributes? perror( progname );
exit(EXIT_FAILURE);
else if( S_ISREG( stat_buffer.st_mode ) ) {
printf( “%s is a regular file\n”, filename );
printf( “is %i bytes long\n”, (int)stat_buffer.st_size );
printf( “and was last modified on %i\n”, (int)stat_buffer.st_mtime);
printf( “which was %s”, ctime( &stat_buffer.st_mtime) ); }
†POSIX is an acronym for “Portable Operating System Interface”, a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix (such as macOS and Linux) and other operating systems (e.g. Windows has a POSIX emulation layer).
CITS2002 Systems Programming, Lecture 16, p5, 21st September 2021.
CITS2002 Systems Programming
←prev 6 next→ CITS2002 CITS2002schedule
Reading the contents of a directory
Most modern operating systems store their data in hierarchical file systems, consisting of directories which hold items that, themselves, may either be files or directories.
The formats used to store information in directories in different file-systems are different(!), and so when writing portable C programs, we prefer to use functions that work portably.
Consider the strong similarities between opening and reading a (text) file, and opening and reading a directory:
#include
void print_file(char *filename) {
char line[BUFSIZ];
fp = fopen(filename, “r”); if(fp == NULL) {
perror( progname );
exit(EXIT_FAILURE); }
while(fgets(line, sizeof(buf), fp) != NULL) { printf( “%s”, line);
fclose(fp); }
#include
void list_directory(char *dirname) {
DIR *dirp;
struct dirent *dp;
dirp = opendir(dirname); if(dirp == NULL) {
perror( progname );
exit(EXIT_FAILURE); }
while((dp = readdir(dirp)) != NULL) { printf( “%s\n”, dp->d_name );
closedir(dirp); }
With directories, we’re again discussing functions that are not part of the C11 standard, but are defined by POSIX standards.
The inconsistent naming of system-defined datatypes – for example, struct dirent versus DIR – can be confusing (annoying) but, over time, renaming datatypes across billions of lines of open-source code becomes impossible.
CITS2002 Systems Programming, Lecture 16, p6, 21st September 2021.
CITS2002 Systems Programming
←prev 7 next→ CITS2002 CITS2002schedule
Investigating the contents of a directory
We now know how to open a directory for reading, and to determine the names of all items in that directory.
What is each “thing” found in the directory – is it a directory, is it a file…?
To answer those questions, we need to employ the POSIX function, stat(), to determine theattributes of the items we find in directories:
#include
void list_directory(char *dirname) {
char fullpath[MAXPATHLEN];
while((dp = readdir(dirp)) != NULL) {
struct stat stat_buffer;
sprintf(fullpath, “%s/%s”, dirname, dp->d_name );
if(stat(fullpath, &stat_buffer) != 0) { perror( progname );
else if( S_ISDIR( stat_buffer.st_mode )) {
printf( “%s is a directory\n”, fullpath ); }
else if( S_ISREG( stat_buffer.st_mode )) { printf( “%s is a regular file\n”, fullpath );
printf( “%s is unknown!\n”, fullpath ); }
closedir(dirp);
#include
void list_directory(char *dirname) {
char fullpath[MAXPATHLEN];
while((dp = readdir(dirp)) != NULL) {
struct stat stat_buffer;
struct stat *pointer = &stat_buffer;
sprintf(fullpath, “%s/%s”, dirname, dp->d_name );
if(stat(fullpath, pointer) != 0) { perror( progname );
else if( S_ISDIR( pointer->st_mode )) {
printf( “%s is a directory\n”, fullpath ); }
else if( S_ISREG( pointer->st_mode )) {
printf( “%s is a regular file\n”, fullpath );
printf( “%s is unknown!\n”, fullpath ); }
closedir(dirp);
CITS2002 Systems Programming, Lecture 16, p7, 21st September 2021.
CITS2002 Systems Programming
←prev 8 next→ CITS2002 CITS2002schedule
Integers and their representation
Computers store information in binary – not ‘as decimals’, ‘as integers’, or ‘as characters’ – they just use binary.
All data in digital computers is stored as a sequence of bits, with (modern) computers aggregating 8 bits to a byte.
Contemporary laptops, for example, will store integers in 4 bytes(32 bits), or 8 bytes(64 bits). Consider how these bits form bytes, and how integers are represented:
Produces the output:
0: 00000000000000000000000000000000 00000000 0x0000 1: 00000000000000000000000000000001 00000001 0x0001 2: 00000000000000000000000000000010 00000002 0x0002 3: 00000000000000000000000000000011 00000003 0x0003 4: 00000000000000000000000000000100 00000004 0x0004 5: 00000000000000000000000000000101 00000005 0x0005 6: 00000000000000000000000000000110 00000006 0x0006 7: 00000000000000000000000000000111 00000007 0x0007 8: 00000000000000000000000000001000 00000010 0x0008 9: 00000000000000000000000000001001 00000011 0x0009
10: 00000000000000000000000000001010 00000012 0x000a 11: 00000000000000000000000000001011 00000013 0x000b 12: 00000000000000000000000000001100 00000014 0x000c 13: 00000000000000000000000000001101 00000015 0x000d 14: 00000000000000000000000000001110 00000016 0x000e 15: 00000000000000000000000000001111 00000017 0x000f 16: 00000000000000000000000000010000 00000020 0x0010
#include
char *binary(unsigned int x)
#define NBITS ((sizeof x) * NBBY)
static char bits[NBITS+1]; char *b = bits;
for(int n=NBITS-1 ; n>=0 ; –n) { *b = ((x & (1<
CITS2002 Systems Programming, Lecture 16, p8, 21st September 2021.
CITS2002 Systems Programming
←prev 9 next→ CITS2002 CITS2002schedule
Bitwise operators in C11
Knowing that data, such as integers, are sequences of bits, we may use the multiple bits of an integer to represent multiple pieces of data in asingle integer.
Firstly, let’s consider a few of C’s operations on bits and bit patterns. When used in combination, these provide some powerful facilities not readily supported in some other programming languages.
The bitwise operators
bitwise-and bitwise-or exclusive-or (xor) not
left shift
Example Result
Description
3 & 5 3 | 5 3 ^ 5 ~3
3 << 2 n << p
1 if both bits are 1.
1 if either bit is 1.
1 if both bits are different.
Inverts the bits.
Shifts the bits of n left p positions. Zero bits are shifted into the low-order positions.
right shift
5 >> 2 n >> p
Shifts the bits of n right p positions. If n is a 2’s complement signed number, the sign bit is shifted into the high-order positions.
Note: Don’t confuse &&, the short-circuit logical-and, with &, which is the less commonbitwise- and.
While the bitwise-and can also be used with Boolean operands, this is extremely rare and is almost always a programming error.
CITS2002 Systems Programming, Lecture 16, p9, 21st September 2021.
CITS2002 Systems Programming
←prev 10 next→ CITS2002 CITS2002schedule
Storing (packing) multiple items in an integer
The most frequently seen example of C’s bitwise operators, is the use of the left-shift operator to store multiple “items” in a single integer variable.
Consider the following example, using an unsigned 32-bit integer (uint32_t) to represent colours in an RGB format of 24 bits-per-pixel (24bpp).
#include
typedef uint32_t RGBCOLOUR;
RGBCOLOUR set_rgb(char red, char green, char blue) {
return (red << 16) + (green << 8) + blue; }
RGBCOLOUR white = set_rgb(255, 255, 255); RGBCOLOUR black = set_rgb( 0, 0, 0); RGBCOLOUR skyblue = set_rgb(135, 206, 235); RGBCOLOUR yellow = set_rgb(255, 255, 0);
Here, the left-shift operator is used to quickly multiply† a small value by a constant power of two.
On most modern architectures, bit-shifting operations are considerably faster than the equivalent multiplications (use left-shifting) and divisions (use right-shifting).
If unfamiliar with RGB colour representations, you may like to read:
www.rapidtables.com/web/color/RGB_Color.htm www.w3schools.com/colors/colors_rgb.asp
†C compilers can optimize their generated code by observing that the shift and addition operators are faster than straight multiplication. For example, your C code such as
a = b * 10;
will be translated - internally, before processor instructions are generated - to
a = (b << 3) + (b << 1);
CITS2002 Systems Programming, Lecture 16, p10, 21st September 2021.
CITS2002 Systems Programming
←prev 11 next→ CITS2002 CITS2002schedule
Extracting (unpacking) multiple items from an integer
Storing (packing) multiple bit-wise items in an integer is relatively simple. Recovery of the individual components is more difficult (to understand). There are a wide variety of ways to undertake this:
typedef uint32_t RGBCOLOUR;
char get_blue(RGBCOLOUR rgb) {
return (rgb & 0xff); }
char get_green(RGBCOLOUR rgb) {
return ((rgb >> 8) & 0xff); }
char get_red(RGBCOLOUR rgb) {
return ((rgb >> 16) & 0xff); }
Such code often mixes octal and hexadecimal values, and left- and right-shift operations. Fortunately, modern C compilers are able to generate the optimal machine-level code for their targeted architecture.
CITS2002 Systems Programming, Lecture 16, p11, 21st September 2021.
CITS2002 Systems Programming
←prev 12 next→ CITS2002 CITS2002schedule
Extracting (unpacking) multiple items from an integer, continued
In efforts to extract even more speed from programming involving bit-wise operations, advanced programmers will
often use C preprocessor macros, providing inline functions, to reduce the (tiny) cost of the function call overhead:
typedef uint32_t RGBCOLOUR;
#define set_rgb(r, g, b) (((b) << 16) + ((g) << 8) + (r)))
#define get_blue(rgb) ((rgb ) & 0xff) #define get_green(rgb) (((rgb) >> 8) & 0xff) #define get_red(rgb) (((rgb) >> 16) & 0xff)
void RGB_to_greyscale(RGBCOLOUR *greypixels[], RGBCOLOUR *colourpixels[], int width, int height) {
for(int w=0 ; w
CITS2002 Systems Programming, Lecture 16, p12, 21st September 2021.
CITS2002 Systems Programming
←prev 13 next→ CITS2002 CITS2002schedule
File and Directory Permissions
Recall that:
Like most modern systems, Linux arranges its file system in a hierarchical structure of files and directories. Directories may contain entries for files and other directories.
File entries contain the file’s name, together with a pointer to a structure termed the inode (the information node?), which represents the details of the file.
These are the familiar items shown by the standard ls -l command:
Multiple file entries, from possibly different directories, may point to the same inode. Thus, it is possible to have a single file with multiple names – we say that the names are links to the same file.
One unsigned 32-bit integer field in each inode contains the file’s permission (or protection) mode bits – see /usr/include/bits/typesizes.h
From history, the permission mode bits appear in the same integer defining the file’s type (regular, directory, block device, socket, …) –
See man 2 stat for details.
CITS2002 Systems Programming, Lecture 16, p13, 21st September 2021.
drwxr-xr-x 11 chris staff 1024 Jul 29 20:29 WWW
-rw——- 1 chris chris 53436 Aug 1 16:28 autonomous.pdf -rw-r–r– 1 chris chris 88 Dec 20 2016 scrolldown.gif
CITS2002 Systems Programming