7/24/2021
COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
Assignment 2: shuck, A Simple Shell
Contents
Aims Introduction Getting Started
Reference Implementation Subset 0: cd and pwd
Subset 1: Running Commands
Subset 2: Making history
Subset 3: Filename Expansion (Globbing) Subset 4: Input/Output Redirection Subset 5: Pipes
Testing
Assumptions and Clarifications Assessment
Testing
Submission
Due Date
Assessment Scheme Intermediate Versions of Work Assignment Conditions
Change Log
Aims
version: 1.1.4 last updated: 2021-07-23 13:45:00
to explore Linux process manipulation system calls
to further explore string manipulation in C
to further experience data structures in C
to build a very simple command-line interpreter
to explore writing C code to manipulate processes
to give you experience with interprocess communication (pipes)
Introduction
Your task in this assignment is to write shuck, a small but useful subset of the core functionality typical of a Unix/Linux shell.
A shell provides users with an interface to the operating system: it takes a user’s input, and determines how to fulfill those requests. For
example, on Microsoft Windows, the shell is ¡°Windows Explorer¡±; on Apple macOS, the shell is ¡°Finder¡± and ¡°Dock¡±.
On most Unix-like systems, the shell is a lot more rudimentary in its appearance. It deals with characters on a terminal (whether a physical device that looks like a typewriter’s worst nightmares, or a virtual device like a terminal emulator), prompting the user for input by printing out some characters, and reading in some more characters from the user, which it attempts to evaluate and execute as a command. Each line of input consists of the name of a program, along with some parameters, or arguments, that are passed to this program; so:
One of the things that made Unix quite unique compared to its contemporaries was that the shell couldn’t do much by itself ¡ª it can’t list directories, or print out files ¡ª but it did (and still does!) know how to find and start other programs. Most Unix-like systems come with a raft of utility programs, all filed away in a range of directories; the shell knows a list of these, which it calls the PATH, and when a user commands the system, the shell searches through each of the directories in the PATH for executable files matching the name the user requested.
So, in the above list, the first and last commands are effectively exactly the same: the only difference is that the last command uses the full path to the ls command, whilst the first one doesn’t.
ls -l
cat -n shuck.c
file shuck.c
wc -l Makefile
/bin/ls -l
# displays files in current directory
# show shuck.c with line numbers
# show the file type of shuck.c
# show how many lines in the Makefile
# displays files in current directory
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
1/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
There are several different shells available for Unix-like systems. They all do the same thing, more or less; but their main difference is in the interfaces they provide, either in the commands and behaviours they add themselves, or in the ways they let us interact with them. One of the most common shells on Unix-like systems, including most Linuxes, is bash, and it’s very likely what you use on CSE.
For our purposes, a shell is just a program that executes other programs. It reads a line of text, breaks that line into words, then executes the command indicated by the first token. For example:
… is a command with three tokens: “ls”, “-al” and “~cs1521/bin”. The shell works out where the executable for the ls command is located, and executes it, passing the second two tokens as command-line arguments.
Shells can do a lot more than just reading command lines and executing them. They can keep a history of previous commands to make it easy to re-execute them. They can also allow users to capture the output of a command, by redirecting its input or output from or into a file. And, importantly on Unix/Linux, they allow users to build a pipeline of commands to achieve powerful effects without having to write a dedicated program. For example, the following pipeline produces a list of the top ten most frequently used words in a text file:
COMP2041 goes into much more detail about many Unix/Linux commands, and how they can be combined in this way.
You will not be required to implement a shell as powerful as bash ¡ª that would be an incredibly big task! ¡ª but you will build many of
its core features, and you will be given a number of simplifying assumptions, which make your task much easier.
Getting Started
Create a new directory for this assignment called shuck, change to this directory, and fetch the provided code by running these commands:
If you’re not working at CSE, you can download the provided files as a zip file or a tar file.
This will get you shuck.c, which contains code to start the assignment. As provided, it will compile and run, but the only functionality
implemented is exit:
However, shuck.c also contains some functions that make your task much easier. For example, the tokenizer function will break a string into words, using specified separators. You should read through the provided code before you begin work on this assignment.
Reference Implementation
To clarify how your implementation should behave, there is a reference implementation available:
Subset 0: cd and pwd
For this subset, you will add code to shuck.c ¡ª in particular, the function execute_command ¡ª to implement the shell built-in commands cd (change directory), and pwd (print working directory).
ls -al ~cs1521/bin
cat blah.txt | tr -cs ‘[a-z]’ ‘\n’ | sort | uniq -c | sort -nr | head -10
$ mkdir shuck
$ cd shuck
$ 1521 fetch shuck
$ make
dcc shuck.c -o shuck $ ./shuck
shuck& exit
$
$ 1521 shuck
shuck& echo hello, reference implementation hello, reference implementation
/bin/echo exit status = 0
shuck& exit
$
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
2/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
//
// Execute a command, and wait until it finishes.
//
// * `words’: a NULL-terminated array of words from the input command line // * `path’: a NULL-terminated array of directories to search in;
// * `environment’: a NULL-terminated array of environment variables.
//
static void execute_command(char **words, char **path, char **environment) {
assert(words != NULL);
assert(path != NULL);
assert(environment != NULL);
char *program = words[0];
if (program == NULL) { // nothing to do
return; }
if (strcmp(program, “exit”) == 0) { do_exit(words);
// `do_exit’ will only return if there was an error.
return; }
// [[ TODO: add code here to implement subset 0 ]]
// [[ TODO: change code below here to implement subset 1 ]]
if (strrchr(program, ‘/’) == NULL) {
fprintf(stderr, “— UNIMPLEMENTED: searching for a program to run\n”);
}
if (is_executable(program)) {
fprintf(stderr, “— UNIMPLEMENTED: running a program\n”);
} else {
fprintf(stderr, “— UNIMPLEMENTED: error when we can’t run anything\n”);
} }
execute_command takes three arguments. For this subset, we are only interested in words, the command that has been entered, as an array of strings.
Once you have implemented this subset, your shell should match these behaviours:
If the cd command runs with no argument, it should change directory to the value specified in the HOME environment variable.
(Your home directory will, of course, be different to the example above.)
$ 1521 shuck
shuck& cd /tmp
shuck& pwd
current directory is ‘/tmp’ shuck& cd /usr/local/bin shuck& pwd
current directory is ‘/usr/local/bin’
$ 1521 shuck
shuck& cd
shuck& pwd
current directory is ‘/home/z5555555’
HINT:
You may find the following lecture examples useful:
my_cd.c, which shows how use chdir to change directory;
getcwd.c, which shows how to print your current directory; and get_status.c, which shows how to get the value of an environment variable.
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
3/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
https://cgi.cse.u
4/10
Subset 1: Running Commands
For this subset, you will add code to shuck.c ¡ª in particular, to the function execute_command ¡ª you will to modify shuck.c ¡ª in particular, the function execute_command ¡ª to execute commands which are not built-in.
execute_command takes three arguments. Of interest to us are the first two: words, described elsewhere; and path a list of directories to search for the corresponding executable file, as an array of strings. The first word of the command specifies a program to run. Based on these arguments, you need to find an executable file corresponding to the command and execute it.
Of interest is the provided function, is_executable:
//
// Check whether this process can execute a file. This function will be // useful while searching through the list of directories in the path to // find an executable file.
//
static int is_executable(char *pathname)
{
struct stat s; return
}
// does the file exist?
stat(pathname, &s) == 0 &&
// is the file a regular file?
S_ISREG(s.st_mode) &&
// can we execute it?
faccessat(AT_FDCWD, pathname, X_OK, AT_EACCESS) == 0;
Once you have implemented this subset, your shell should match these behaviours:
A command may have multiple words, which should be passed as that program’s argv. For example:
$ 1521 shuck
shuck& date
Tue 14 Sep 2021 19:33:35 AEST /bin/date exit status = 0
$ 1521 shuck
shuck& file shuck.c
shuck.c: C source, ASCII text /usr/bin/file exit status = 0
shuck& echo this command has 4 arguments this command has 4 arguments
/bin/echo exit status = 0
shuck& echo this command has 4 arguments this command has 4 arguments
/bin/echo exit status = 0
shuck& cp shuck.c shuck.c.backup
/bin/cp exit status = 0
shuck& ls shuck.c.backup
shuck.c.backup
/bin/ls exit status = 0
shuck& hdhhfdhjf
hdhhfdhjf: command not found
The exit status of a command is printed when it exits. For example:
If a command contains a ‘/’, you do not need to search for it in the path directories. For example:
$ 1521 shuck
shuck& ls cow.c
ls: cannot access ‘cow.c’: No such file or directory bin/ls exit status = 2
$ 1521 shuck
shuck& /home/cs1521/bin/spim (spim)
HINT:
You may find the following lecture examples useful:
spawn.c, which shows how to run a program using posix_spawn, and how to wait for a new process to finish using waitpid.
nsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
7/24/2021
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html 5/10
COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
Subset 2: Making history For this subset, you will need to modify shuck.c
to save every command run to the file $HOME/.shuck_history;
to implement a built-in command history n which prints the last n commands, or, if n is not specified, 10; and
to implement a built-in command ! n which prints the nth command and then executes it, or, if n is not specified, the last command.
Once you have implemented this subset, your shell should match these behaviours:
$ rm -f $HOME/.shuck_history $ 1521 shuck
shuck& echo hello
hello
/bin/echo exit status = 0 shuck& echo shuck
shuck
/bin/echo exit status = 0 shuck& history
0: echo hello
1: echo shuck
shuck& !0
echo hello
hello
/bin/echo exit status = 0 shuck& !1
echo shuck
shuck
/bin/echo exit status = 0 shuck& !
echo shuck
shuck
/bin/echo exit status = 0
You can store commands in-memory ¡ª for example, in an array ¡ª but you should also immediately append each command to the file $HOME/.shuck_history. When appending commands, put a space between each word, and a ‘\n’ after the command; this will often differ from what whitespace the user actually entered.
For ‘!’ commands, add the command it executed to the history file ¡ª ‘!’ should not ever appear itself as a command in the history file.
Subset 3: Filename Expansion (Globbing)
For this subset, you will need to modify shuck.c to support filename expansion, sometimes referred to as globbing.
If any of the characters ‘*’, ‘?’, ‘[‘, or ‘~’ appear in a word, that word should be taken as a pattern, and should be replaced by all of the words matching that pattern using the glob library function; or, if there are no matches, use the pattern-word unchanged. This may result in the word list becoming longer than it initially was.
Filename expansion should be done before any of the actions described below.
HINT:
The tokenize function will always returns ! as a separate word, so tokenising “!n” will always produce two words, “!” and “n”.
You should read the documentation for waitpid: it includes discussion about WIFEXITED and WEXITSTATUS, which you will need.
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
$ 1521 shuck
shuck& cd /web/cs1521/21T2/topic/mips_basics/code shuck& echo *.c
i_love_mips.c
/bin/echo exit status = 0
shuck& wc *mips*
6 11 82 i_love_mips.c
12 38 230 i_love_mips.s
18 49 312 total
/usr/bin/wc exit status = 0 shuck& ls *.[cs] i_love_mips.c i_love_mips.s /bin/ls exit status = 0
HINT:
You may find the following lecture examples useful: glob.c, which shows how to call the glob function.
You should use GLOB_NOCHECK|GLOB_TILDE as the second parameter of the glob function.
Subset 4: Input/Output Redirection
For this subset, you will need to modify shuck.c to support input and output redirections:
If the first two tokens of the command line are <, and a filename, the command should be executed with its standard input connected to
the specified file. If the file does not exist, or is not readable, an error message should be printed instead.
If the last two words of the command line are >, and a filename, the command should be executed with its standard output connected to the specified file. If the file is not writable, an error message should be printed instead. If the file exists, it should be over-written.
If the last three words of the command line are >, >, and a filename, the command should be executed with its standard output connected to the specified file such that its output is appended to the file, rather than the file being overwritten. If the file is not writable, an error message should be printed instead.
Additionally, an error message should be printed
if > or < appear anywhere elsewhere on the command-line; if no command is specified to redirect; or
if a builtin command is specified with I/O redirection.
Once you have implemented this subset, your shell should match these behaviours:
$ 1521 shuck
shuck& echo hello shuck >hello.txt /bin/echo exit status = 0
shuck& cat hello.txt
hello shuck
/bin/cat exit status = 0
shuck& echo hello again >>hello.txt /bin/echo exit status = 0
shuck& cat hello.txt
hello shuck
hello again
/bin/cat exit status = 0
shuck& echo good bye shuck >hello.txt /bin/echo exit status = 0
shuck& cat hello.txt
good bye shuck
/bin/cat exit status = 0
shuck& < shuck.c wc -l
636
/usr/bin/wc exit status = 0
shuck& wc -l shuck.c >number_of_lines /usr/bin/wc exit status = 0
shuck& cat number_of_lines
636 shuck.c
/bin/cat exit status = 0
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
6/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
HINT:
You may find the following lecture examples useful:
spawn_read_pipe.c and spawn_write_pipe.c, which show the use of the posix_spawn_file_actions_adddup2 function.
The tokenize function will (as provided) always return < and > as a separate words: for example, tokenize’ing “>file” will give two words, {“>”, “file”, NULL}; and
tokenize’ing “>>file” will give three words {“>”, “>”, “file”, NULL}.
Subset 5: Pipes
For this subset, you will need to modify shuck.c to support pipelines, which, within a command line, connect the standard output of one command to the standard input of the next.
If a ‘|’ appears between two commands, the standard output of the first command should be connected to the standard input of the second command. For example:
$ 1521 shuck
shuck& cat shuck.c | wc -l
2458
/usr/bin/wc exit status = 0
shuck& < shuck.c cat | wc -l >number_of_lines
/usr/bin/wc exit status = 0
shuck& cat number_of_lines
2458
/bin/cat exit status = 0
shuck& < shuck.c cat | grep include | wc -l >number_of_includes /usr/bin/wc exit status = 0
shuck& cat number_of_includes
18
/bin/cat exit status = 0
HINT:
You may find the following lecture examples useful:
spawn_read_pipe.c and spawn_write_pipe.c, which show the use of the pipe, posix_spawn_file_actions_adddup2, and
posix_spawn_file_actions_addclose functions.
The tokenize function will (as provided) always return | as a separate word: for example, tokenize’ing “cat|dog” will give three words, {“cat”, “|”, “dog”, NULL}.
Testing
You are expected do your own testing. Some autotests are available to help you get started:
You can create extra .c or .h files; but you will need to supply them explicitly to autotest; for example:
Assumptions and Clarifications
Like all good programmers, you should make as few assumptions as possible.
Your submitted code must be a single C program only. You may not submit code in other languages.
To create processes, you must use the function posix_spawn. You are not permitted to use functions such as posix_spawnp, system, popen, fork, vfork, clone, or any of the exec* family of functions, like execve.
$ 1521 autotest shuck …
$ 1521 autotest shuck extra1.c extra2.c extra3.h …
Subset 5 must be implemented using pipes. It may not be implemented with temporary files.
You may not use functions from non-default libraries; in other words, you cannot use dcc’s -l flag.
You should avoid leaking memory wherever possible. (Some leaks may require a little thought to resolve.)
v1.1 v1.1
As provided, exit will exit under circumstances that the reference implementation will not; this avoids prescribing a specific approach in your implementation. You will need to resolve this.
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
7/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
If you need clarification on what you can and cannot use or do for this assignment, ask in the class forum. You are required to submit intermediate versions of your assignment. See below for details.
Your program must not require extra compile options. It must compile with dcc *.c -o shuck, and it will be run with dcc when marking. Run-time errors from illegal C will cause your code to fail automarking.
If your program writes out debugging output, it will fail automarking tests. Make sure you disable debugging output before submission.
Testing
When you think your program is working, you can use autotest to run some simple automated tests: 1521 autotest will not test everything.
Always do your own testing.
Automarking will be run by the lecturer after the submission deadline, using a superset of tests to those autotest runs for you.
Submission
When you are finished working on the assignment, you must submit your work by running give:
You must run give before Week 10 Friday 21:00:00 to obtain the marks for this assignment. Note that this is an individual exercise, the
work you submit with give must be entirely your own. You can run give multiple times.
Only your last submission will be marked.
If you are working at home, you may find it more convenient to upload your work via give’s web interface. You cannot obtain marks by e-mailing your code to tutors or lecturers.
You can check your latest submission on CSE servers with:
You can check the files you have submitted here.
Manual marking will be done by your tutor, who will mark for style and readability, as described in the Assessment section below. After
your tutor has assessed your work, you can view your results here; The resulting mark will also be available via give’s web interface.
Due Date
This assignment is due Week 10 Friday 21:00:00.
If your assignment is submitted after this date, each hour it is late reduces the maximum mark it can achieve by 2%. For example, if an assignment worth 74% was submitted 10 hours late, the late submission would have no effect. If the same assignment was submitted 15 hours late, it would be awarded 70%, the maximum mark it can achieve at that time.
Assessment Scheme
This assignment will contribute 15 marks to your final COMP1521 mark.
80% of the marks for assignment 2 will come from the performance of your code on a large series of tests.
20% of the marks for assignment 2 will come from hand marking. These marks will be awarded on the basis of clarity, commenting, elegance and style. In other words, you will be assessed on how easy it is for a human to read and understand your program.
An indicative assessment scheme follows. The lecturer may vary the assessment scheme after inspecting the assignment submissions, but it is likely to be broadly similar to the following:
Assessment
$ 1521 autotest shuck shuck.c [any other .c or .h files]
$ 1521 classrun give ass2_shuck shuck.c [other .c or .h files]
$ 1521 classrun check ass2_shuck
HD (85+) DN (75+) CR (65+) PS (50-60) 0%
0 FL for COMP1521
subsets 0¡ª5 work; beautiful code
subsets 0¡ª3 work; good, clear code
subsets 0, 1 work
subset 0 works; subset 1 partly working
knowingly providing your work to anyone and it is subsequently submitted (by anyone).
submitting any other person’s work; this includes joint work.
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
8/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
academic submitting another person’s work without their consent; misconduct paying another person to do work for you.
Intermediate Versions of Work
You are required to submit intermediate versions of your assignment.
Every time you work on the assignment and make some progress you should copy your work to your CSE account and submit it using the give command below. It is fine if intermediate versions do not compile or otherwise fail submission tests. Only the final submitted version of your assignment will be marked.
All these intermediate versions of your work will be placed in a Git repository and made available to you via a web interface at https://gitlab.cse.unsw.edu.au/z5555555/21T2-comp1521-ass2_shuck (replacing z5555555 with your own zID). This will allow you to retrieve earlier versions of your code if needed.
Assignment Conditions
Joint work is not permitted on this assignment.
This is an individual assignment. The work you submit must be entirely your own work: submission of work even partly written by
any other person is not permitted.
Do not request help from anyone other than the teaching staff of COMP1521 ¡ª for example, in the course forum, or in help sessions.
Do not post your assignment code to the course forum. The teaching staff can view code you have recently submitted with give, or recently autotested.
Assignment submissions are routinely examined both automatically and manually for work written by others.
Rationale: this assignment is designed to develop the individual skills needed to produce an entire working program. Using code written by, or taken from, other people will stop you learning these skills. Other CSE courses focus on skills needed for working in a team.
The use of code-synthesis tools, such as GitHub Copilot, is not permitted on this assignment.
Rationale: this assignment is designed to develop your understanding of basic concepts. Using synthesis tools will stop you
learning these fundamental concepts, which will significantly impact your ability to complete future courses. Sharing, publishing, or distributing your assignment work is not permitted.
Do not provide or show your assignment work to any other person, other than the teaching staff of COMP1521. For example, do not message your work to friends.
Do not publish your assignment code via the Internet. For example, do not place your assignment in a public GitHub repository.
Rationale: by publishing or sharing your work, you are facilitating other students using your work. If other students find your assignment work and submit part or all of it as their own work, you may become involved in an academic integrity investigation.
Sharing, publishing, or distributing your assignment work after the completion of COMP1521 is not permitted.
For example, do not place your assignment in a public GitHub repository after this offering of COMP1521 is over.
Rationale: COMP1521 may reuse assignment themes covering similar concepts and content. If students in future terms find your assignment work and submit part or all of it as their own work, you may become involved in an academic integrity investigation.
Violation of any of the above conditions may result in an academic integrity investigation, with possible penalties up to and including a mark of 0 in COMP1521, and exclusion from future studies at UNSW. For more information, read the UNSW Student Code, or contact the course account.
Change Log
Version 1.0
(2021-07-19 11:00:00)
Version 1.1
(2021-07-21 16:00:00)
Version 1.1.1
(2021-07-22 01:00:00)
Initial release onto unsuspecting students.
Adjust provided code to make ‘dcc -Werror’ happy.
Add clarifications marked [v1.1] around exit and memory leaks.
Fix an incorrect example in Subset 4.
Improve navigation around the assignment specification document.
Fix incorrect code inclusion into spec.
Fix some bugs in the reference implementation.
Version 1.1.2
Fix incorrect output in example.
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
9/10
7/24/2021 COMP1521 21T2 ¡ª Assignment 2: shuck, A Simple Shell
(2021-07-23 11:45:00)
Version 1.1.3
(2021-07-23 12:45:00)
Version 1.1.4
(2021-07-23 13:45:00)
Fix bug in the reference implementation. Fix bug in the reference implementation.
COMP1521 21T2: Computer Systems Fundamentals is brought to you by the School of Computer Science and Engineering
at the University of New South Wales, Sydney.
For all enquiries, please email the class account at cs1521@cse.unsw.edu.au CRICOS Provider 00098G
https://cgi.cse.unsw.edu.au/~cs1521/21T2/assignments/ass2/index.html
10/10