Github clone link
Github clone (https://classroom.github.com/a/a_S9Jk4B)
Github clone link
Goals
Overview
Background – Handwritten Digit Classification
Check Yourself
Do you know how to run venus at the command line and in browswer ?
What is the RISC-V Calling Convention?
How to trace and debug values in Venus ?
Source
Inputs, Out and Ref Outputs
Part A
Matrix and Arrays
Task 1: ReLU
Testing: ReLU
Task 2: ArgMax
Testing: Argmax
Task 3 Dot Product
Testing: Dot Product
Task 4: Matrix Multiplication
Testing: Matrix Multiplication
Part B: File Operations and Main
Matrix File Format
Task 1: Read Matrix
Testing: Read Matrix
Task 2: Putting it all Together
Command Line Arguments and File Paths
Algorithm steps
Validation
Validating Simple
Validating MNIST
End-to-End tests
Grading
Debugging hints
Acknowledgments
Goals
The purpose of this project is to have you implement a simple, yet extremely useful system in RISC-V assembly language.
You will learn to use registers efficiently, write functions, use calling conventions for calling your functions, as well as
external ones, allocate memory on the stack and heap, work with pointers and more!
Overview
You will implement functions which operate on matrices and vectors – for example, matrix multiplication. You will then use
these functions to construct a simple Artificial Neural Net (ANN), which will be able to classify handwritten digits to their actual
number! You will see that ANNs can be simply implemented using basic numerical operations, such as vector inner product,
matrix multiplications, and thresholding.
https://classroom.github.com/a/a_S9Jk4B
Note: Although the spec is quite long, please make sure to read through the whole thing as it contains a lot of important
information about the functions provided, running Venus, and testing your code.
wget https://www2.cs.sfu.ca/~ashriram/Courses/CS295/assets/distrib/Venus/jvm/venus.jar
# We can also encourage you to use the online version.
# Do not check it in under any circumstance or you may fail the travis test.
# https://www2.cs.sfu.ca/~ashriram/Courses/CS295/assets/distrib/Venus/
Background
At a basic level, a neural networks tries to approximate a (non-linear) function that maps your input into a desired output. A
basic neuron consists of a weighted linear combination of the input, followed by a non-linearity – for example, a threshold.
Consider the following neuron, which implements the logical AND operation:
It is easy to see that for A=0, B=0, the linear combination 0*0.6 + 0*0.6 = 0, which is less than the threshold of 1 and will result
in a 0 output. With an input A=0, B=1 or A=1, B=0 the linear combination will results in 1*0.6 + 0*0.6 = 0.6, which is less than 1
and result in a 0 output. Similarly, A=1, B=1 will result in 1*0.6+1*0.6=1.2, which is greater than the threshold and will result in a
1 output! What is interesting is that the simple neuron operation can also be described as an inner product between the vector
[A,B]^T and the weights vector [0.6,0.6]^T followed by as thresholding, non-linear operation.
More complex functions can not be described by a simple neuron alone. We can extend the system into a network of neurons,
in order to approximate the complex functions. For example, the following 2 layer network approximates the logical function
XOR What is XOR? (https://en.wikipedia.org/wiki/Exclusive_or)
The above is a 2 layer network. The network takes 2 inputs, computes 2 intemediate values, and finally computes a single final
output. It can be written as matrix multiplications with matrices m_0 and m_1 with thresholding operations in between as
shown below:
You are probably wondering how the weights of the network were determined? This is beyond the scope of this project. We
encourage you to take a course on machine learning. We will only say that the weights can be trained by giving the network
pairs of correct inputs and outputs and changing the weights such that the error between the outputs of the network and the
correct outputs is minimized. Learning the weights is called: “Training’’. Using the weights on inputs is called “Inference”. We
will only perform inference, and you will be given weights that were pre-trained by your dedicated TA’s.
Handwritten Digit Classification
https://en.wikipedia.org/wiki/Exclusive_or
In this project we will implement a similar, but slightly more complex network which is able to classify handwritten digits. As
inputs, we will use the MNIST (http://yann.lecun.com/exdb/mnist/) data set, which is a dataset of 60,000 28×28 images
containing handwritten digits ranging from 0-9. We will treat these images as “flattened” input vectors sized 784×1.
In a similar way to the example before, we will perform matrix multiplications with pre-trained weight matrices m_0 and m_1 .
Instead of thresholding we will use two different non-linearities: The ReLU and ArgMax functions.
hidden_layer = matmul(m0, input). Input: [128,784] * [784,1], Output = [128*1]
relu(hidden_layer) # Recall that relu is performed in-place Input [128,1] = Output [128,1]
scores = matmul(m1, hidden_layer) Input: [10,128] * [128,1] Output : [10:1].
argmax(scores)
Check Yourself
Do you know how to run venus at the command line and in browswer ?
All the code you write will be in RISC-V, and will be run in Venus. There are two ways to run your code with Venus, either
through the web interface or from the command line using a Java .jar file. We recommend that you do most of your
development locally with the .jar file, and only use the web interface for debugging or making changes to one file at
a time.
Load factorial, assemble and run link (https://www.cs.sfu.ca/~ashriram/Courses/CS295/assets/distrib/Venus/index.html?
override=true&save=true&target=https://www.cs.sfu.ca/~ashriram/Courses/CS295/assets/demos/Part4/factorial.s)
Download factorial.s to your computer
Change the number for which factorial is calculated.
Upload back to venus
Assemble and simulate
Get venus on your computer
Run program at the command line.
What is the RISC-V Calling Convention?
We will be testing all of your code on RISC-V calling conventions, as described in lecture/lab/discussion. All functions that
overwrite registers that are preserved by convention must have a prologue and epilogue where they save those register values
to the stack at the start of the function and restore them at the end.
Follow the calling conventions. It is extremely important for this project, as you’ll be writing functions that call your other
functions, and maintaining the abstraction barrier provided by the conventions will make your life a lot easier.
We’ve provided # Prologue and # Epilogue comments in each function as a reminder. Note that depending on your
implementation, some functions won’t end up needed a prologue and epilogue. In these cases, feel free to delete/ignore the
comments we’ve provided.
For an closer look at RISC-V calling conventions, refer here (“../../assets/notebooks/RISCV/RISCV_CALL.pdf).
http://yann.lecun.com/exdb/mnist/
https://www.cs.sfu.ca/~ashriram/Courses/CS295/assets/distrib/Venus/index.html?override=true&save=true&target=https://www.cs.sfu.ca/~ashriram/Courses/CS295/assets/demos/Part4/factorial.s
https://www.cs.sfu.ca/~ashriram/Courses/CS295/hw/assets/notebooks/RISCV/RISCV_CALL.pdf
Watch lab 6.
What is the prologue and epilogue in the factorial?
What is the purpose of addi sp, sp, -8 and addi sp, sp, 8 ?
Why do we need this statement ? sw s0, 4 (sp)
How to trace and debug values in Venus ?
what is the value in a0 register when the program returns from factorial
what is the value in ra ?
Source
test_files/ : contains the driver programs. These programs set up the arguments before invoking the functions and then
display the results.
main.s : Main driver program for running the neural network end-to-end
other .s files : Implementations of various functionality. See table below
You will have to modify the files listed below.
Source Description
test_relu.s, relu.s Driver for relu, implement relu
test_argmax.s, argmax.s Driver for argmax, implement argmax
test_dot.s, dot.s Driver for dot product, implement dot product
test_matmul.s, matmul.s Driver for matmul, implement matmul
test_read_matrix.s, read_matrix.s Driver for reading matrix, read the matrix
main.s Main starting file for end-to-end run
In the test_files subdirectory, you’ll find several RISC-V files to test your code with. There is a test file corresponding to
every function you’ll have to write, except for the part 2.
DO NOT MODIFY THE INPUTS WHEN COMMITTING THE FILES TO GIT. IT WILL FAIL THE REFERENCE CHECKS
Inputs, Out and Ref Outputs
input/ : the various input files. There are totally three four sets of inputs, mnist, simple0, simple1, and simple2.
# e.g., listing of mnist
$ ls inputs/mnist
bin txt
Each network folder contains a bin and txt subfolder. The bin subfolder contains the binary files that you’ll run main.s
on, while the txt subfolder contains the plaintext versions for debugging and calculating the expected output.
Within the bin and txt subfolders, you’ll find files for m0 and m1 which define the network, and a folder containing several
inputs.
$ ls mnist/bin/inputs
mnist_input0.bin mnist_input3.bin mnist_input6.bin
mnist_input1.bin mnist_input4.bin mnist_input7.bin
mnist_input2.bin mnist_input5.bin mnist_input8.bin
out/
To aid in the process of testing and debugging, create an output folder after you clone.
# DO NOT CHECK IN out/. IT MIGHT BREAK THE AUTOGRADERS
mkdir -p out/mnist
mkdir -p out/simple0
mkdir -p out/simple1
mkdir -p out/simple2
ref/ : The reference outputs.
The test_[*].out represent the output of each sub-function.
ls ./ref/
mnist test_argmax.out test_read_matrix.out
simple0 test_dot.out test_relu.out
simple1 test_matmul.out
simple2 test_output.bin
We include the reference outputs for each of the inputs. The traces display the out array of each of the stages in main. (look for
the jal print_array calls in main.s). They include the outputs from each stage of main for verification.
$ ls ./ref/mnist/
mnist_input0.trace mnist_input3.trace mnist_input6.trace
mnist_input1.trace mnist_input4.trace mnist_input7.trace
mnist_input2.trace mnist_input5.trace mnist_input8.trace
Part A
Expected completion time (within 7 days)
Some of these test files have been provided for you in their entirety, and you’ll only have to edit them to change their inputs.
Other ones just include starter code, and have been left for you to fill out. You will implement.
dot product (test_dot.s dot.s)
matrix multiplication (test_matmul.s, matmul.s)
elementwise ReLU (test_relu.s, relu.s)
argmax function. (test_argmax.s, argmax.s)
Matrix and Arrays
In this project, all two-dimensional matrices will be stored as a one-dimensional vector in row-major order. One way to think
about it is that we can create a 1D vector from a 2D matrix by concatenating together all the rows in the matrix. Alternatively,
we could concatenate all the columns together instead, which is known as column-major order.
For a more in-depth look at row-major vs. column-major order, see this Wikipedia page (https://en.wikipedia.org/wiki/Row-
_and_column-major_order).
The stride of a vector is the number of memory locations between consecutive elements of our vector, measured in the size of
our vector’s elements. If our stride is n, then the memory addresses of our vector elements are n * sizeof(element) bytes apart.
So far, all the arrays/vectors we’ve worked with have had stride 1, meaning there is no gap betwen consecutive elements.
Now, to do the row * column dot products with our row-major matrices that we’ll need for matrix multiplication, we will need
to consider vectors with varying strides. Specifically, we’ll need to do this when considering a column vector in a flattened,
https://en.wikipedia.org/wiki/Row-_and_column-major_order
row-major representation of a 2D matrix
Let’s take a look at a practical example. We have the vector int *a with 3 elements.
If the stride is 1, then our vector elements are *(a) , *(a + 1) , and *(a + 2) , in other words a[0] , a[1] , and a[2] .
However, if our stride is 4, then our elements are at *(a) , *(a + 4) , and *(a + 8) or in other words a[0] , a[4] , and
a[8] .
To summarize in C code, to access the i th element of a vector int *a with stride s , we use *(a + i * s) , or
a[i * s] . We leave it up to you to translate this memory access into RISC-V.
For a closer look at strides in vectors/arrays, see this Wikipedia page (https://en.wikipedia.org/wiki/Stride_of_an_array).
Task 1: ReLU
In relu.s , implement our relu function to apply the mathematical ReLU function on every element of the input array. This
ReLU function is defined as \text{ReLU}(a) = max(a, 0), and applying it elementwise on our matrix is equivalent to setting every
negative value equal to 0.
Additionally, notice that our relu function operates on a 1D vector, not a 2D matrix. We can do this because we’re applying
the function individually on every element of the matrix, and our 2D matrix is stored in memory as a row-major 1D vector.
Testing: ReLU
We’ve provided test_relu.s for you to test your relu function. In it, you can set the values and dimensions of a matrix in
static memory. Running the file will print that matrix before and after calling your relu function on it.
$ java -jar venus.jar ./test_files/test_relu.s
## See the screen output
## To validate
$ java -jar venus.jar ./test_files/test_relu.s > ./out/test_relu.out
$ diff ./out/test_relu.out ./ref/test_relu.out
# If diff report any lines, then check your program. Pay particular attention to new lines in the file. if the numbers match bu
t diff still reports errors.
By default, in the starter code we’ve provided, m0 point to the start of an array of the integers. We should get the following:
v0 = [3, -42, 432, 7, -5, 6, 5, -114, 2]
reLu(v0) = zero out all -ve values = [3, 0, 432, 7, 0, 6, 5, 0, 2]
Task 2: ArgMax
Near the end of our neural net, we’ll be provided with scores for every possible classification. For MNIST, we’ll be given a
vector of length 10 containing scores for every digit ranging from 0 to 9. The larger the score for a digit, the more confident we
are that our handwritten input image contained that digit. Thus, to classify our handwritten image, we pick the digit with the
highest score.
The score for the digit i is stored in the i-th element of the array, to pick the digit with the highest score we find the array index
with the highest value. In argmax.s , implement the argmax function to return the index of the largest element in the array. If
there are multiple largest elements, return the smallest index.
Just like relu , this function takes in a 1D vector. The index you’re expected to return is the index of the largest element in this
1D vector.
Testing: Argmax
We’ve provided test_argmax.s for you to test your argmax function. You can edit it to set a static vector v0 along with its
length, and then run the file to print the output returned by running your function, which should be the index of the largest
element in v0 .
$ java -jar venus.jar ./test_files/test_argmax.s
## See the screen output
## To validate
$ java -jar venus.jar ./test_files/test_argmax.s > ./out/test_argmax.out
$ diff ./out/test_argmax.out ./ref/test_argmax.out
# If diff report any lines, then check your output
https://en.wikipedia.org/wiki/Stride_of_an_array
By default, in the starter code we’ve provided, v0 point to the start of an array of the integers. We should get the following:
v0 = [3, -42, 432, 7, -5, 6, 5, -114, 2]
argmax(v0) = index of 432 = 2
Task 3 Dot Product
In dot.s , implement the dot function to compute the dot product of two integer vectors. The dot product of two vectors a
and b is defined as dot(a, b) = \sum_{i=0}^{n-1} a_ib_i = a_0 * b_0 + a_1 * b_1 + \cdots + a_{n-1} * b_{n-1}, where a_i is the ith
element of a.
Notice that this function takes in the a stride as a variable for each of the two vectors, make sure you’re considering this when
calculating your memory addresses. We’ve described strides in more detail in the background section above, which also
contains a detailed example on how stride affects memory addresses for vector elements.
Also note that we do not expect you to handle overflow when multiplying. This means you won’t need to use the mulh
instruction.
For a closer look at dot products, see this Wikipedia page (https://en.wikipedia.org/wiki/Dot_product#Algebraic_definition).
Testing: Dot Product
This time, you’ll need to fill out test_dot.s , using the starter code and comments we’ve provided. Overall, this test should
call your dot product on two vectors in static memory, and print the result (285 for the sample input).
$ java -jar venus.jar ./test_files/test_dot.s
## See the screen output
## To validate
$ java -jar venus.jar ./test_files/test_dot.s > ./out/test_dot.out
$ diff ./out/test_dot.out ./ref/test_dot.out
# If diff report any lines, then check your output
By default, in the starter code we’ve provided, v0 and v1 point to the start of an array of the integers 1 to 9, continuous in
memory. Let’s assume we set the length and stride of both vectors to 9 and 1 respectively. We should get the following:
v0 = [1, 2, 3, 4, 5, 6, 7, 8, 9]
v1 = [1, 2, 3, 4, 5, 6, 7, 8, 9]
dot(v0, v1) = 1 * 1 + 2 * 2 + … + 9 * 9 = 285
What if we changed the length to 3 and the stride of the second vector v1 to 2, without changing the values in static
memory? Now, the vectors contain the following:
v0 = [1, 2, 3]
v1 = [1, 3, 5]
dot(v0, v1) = 1 * 1 + 2 * 3 + 3 * 5 = 22
Note that v1 now has stride 2, so we skip over elements in memory when calculating the dot product. However, the pointer
v1 still points to the same place as before: the start of the sequence of integers 1 to 9 in memory.
Task 4: Matrix Multiplication
Now that we have a dot product function that can take in varying strides, we can use it to do matrix multiplication. In
matmul.s , implement the matmul function to compute the matrix multiplication of two matrices.
The matrix multiplication of two matrices A and B results in the output matrix C = AB, where C[i][j] is equal to the dot product
of the i-th row of A and the j-ith column of B. Note that if we let the dimensions of A be (n*m), and the dimensions of B be (m *
k), then the dimensions of C must be (n * k). Additionally, unlike integer multiplication, matrix multiplication is not commutative,
AB != BA.
https://en.wikipedia.org/wiki/Dot_product#Algebraic_definition
// A[n][m], B[m][k] .
// A – Base address, B – Base Address
//
for (i = 0; i
$ diff ./out/test_matmul.out ./ref/test_matmul.out
# If diff report any lines, then check your output
For test_matrix, we have provided the sample output ( ./ref/test_matmul.out ).
*IF YOU TRY OUT OTHER INPUTS, YOU CAN CHECK THE RESULT USING WOLFRAM ALPHA. BUT DO NOT CHECK IN
THE CHANGED INPUTS
$ cat ./ref/test_matmul.out
30 36 42
66 81 96
102 126 150
m0 = [1, 2, 3
4, 5, 6,
7, 8, 9]
m1 = [1, 2, 3
4, 5, 6,
7, 8, 9]
matmul(m0, m1) =
[30, 36, 42
66, 81, 96
102, 126, 150]
Part B: File Operations and Main
https://en.wikipedia.org/wiki/Matrix_multiplication#Definition
In this part, you will implement functions to read matrices from the binary files. Then, you’ll write a main function putting
together all of the functions you’ve written so far into an MNIST classifier, and run it using pre-trained weight matrices that
we’ve provided.
Matrix File Format
Our matrix files come in two forms: binary and plaintext.The binary format is the one we will be using for all our programs as it
is easier to read. The plain text version has been provided to help with debugging The usage is as follows:
We recommend the xxd command to open the binary file (DO NOT USE YOUR EDITOR). You can find it’s man page here
(https://linux.die.net/man/1/xxd), but its default functionality is to output the raw bits of the file in a hex representation.
The first 8 bytes of the binary file represent two 4 byte integers. These integers are the number of rows and columns of the
matrix. Every 4 following bytes represents an integer that is an element of the matrix, in row-major order. In this case each of
the 4 bytes represents a vallue of the pixel There are no gaps between elements.. It is important to note that the bytes are in
little-endian order. This means the least significant byte is placed at the lowest memory address. For files, the start of the file
is considered the lower address . This relates to how we read files into memory, and the fact that the start/first element of an
array is usually at the lowest memory address.
$ xxd ./inputs/mnist/bin/inputs/mnist_input0.bin | more
# hit q to exit the viewer
We’ve included a python script called convert.py to convert between the text and binary formats
python convert.py file.bin file.txt –to-ascii to go from binary to plaintext
python convert.py file.txt file.bin –to-binary to go from plaintext to binary
https://linux.die.net/man/1/xxd
For example, let’s say the plaintext example in the previous section is stored in file.txt . We can run
python convert.py file.txt file.bin –to-binary to convert it to a binary format.
Task 1: Read Matrix
In read_matrix.s , implement the read_matrix function which uses the file operations we described above to read a binary
matrix file into memory. If any file operation fails or doesn’t return the expected number of bytes, exit the program with exit
code 1 . The code to do this has been provided for you, simply jump to the eof_or_error label at the end of the file.
Recall that the first 8 bytes contains the two 4 byte dimensions of the matrix, which will tell you how many bytes to read from
the rest of the file. Additionally, recall that the binary matrix file is already in row-major order.
You’ll need to allocate memory for the matrix in this function as well. This will require calls to malloc , which is in util.s and
also described in the background section above.
Finally, note that RISC-V only allows for a0 and a1 to be return registers, and our function needs to return three values: The
pointer to the matrix in memory, the number of rows, and the number of columns. We get around this by having two int
pointers passed in as arguments. We set these integers to the number of rows and columns, and return just the pointer to the
matrix.
Testing: Read Matrix
Testing this function is a bit different from testing the others, as the input will need to be a properly formatted binary file that we
can read in.
We’ve provided a skeleton for test_read_matrix.s , which will read the file test_input.bin , and then print the output. The
file test_input.bin is the binary format of the plaintext matrix file test_input.txt . To change the input file read by the test
you’ll need to edit test_input.txt first, then run the convert.py script with the –to-binary flag to update the binary.
From the root directory, it should look something like this:
python convert.py –to-binary test_files/test_input.txt test_files/test_input.bin After this, you can run the
test again, and it’ll read your updated test_input.bin .
Another thing to note is that you’ll need to allocate space for two integers, and pass in those memory addresses as arguments
to read_matrix . You can do this either with malloc .
$ java -jar venus.jar ./test_files/test_read_matrix.s
## See the screen output
## To validate
$ java -jar venus.jar ./test_files/test_read_matrix.s > ./out/test_read_matrix.out
$ diff ./out/test_read_matrix.out ./ref/test_read_matrix.out
# If diff report any lines, then check your output
Task 2: Putting it all Together
In main.s , implement the main function. This will bring together everything you’ve written so far, and create a basic
sequence of functions that will allow you to classifiy the preprocessed MNIST inputs using the pretrained matrices we’ve
provided. You may need to malloc space when reading in matrices and computing the layers of the network, but remember to
always free all data allocated at the end of this process. More information about the free function is available in utils.s
and the background section above.c
Note that for THIS PROJECT/FUNCTION ONLY, we will NOT require you to follow RISC-V calling convention by
preserving saved registers in the main function. This is to make testing the main function easier, and to reduce its
length. Normally, main functions do follow convention with a prologue and epilogue.
Command Line Arguments and File Paths
The filepaths for the input , m0 , and m1 to write to will all be passed in on the command line. RISC-V handles command line
arguments in the same way as C, at the start of the main function a0 and a1 will be set to argc and argv respectively.
We will call main.s in the following way: java -jar venus.jar main.s
Note that this means the pointer to the string INPUT will be located at index 1 of argv , M0_PATH at index 2, and so on.
If the number of command line arguments is different from what is expected, you code should exit with exit code 3.
This will require a call to a helper function in utils.s . Take a look at the starter code for matmul , read_matrix , and
write_matrix for hints on how to do this.
Algorithm steps
1. Read Matrix: Read the inputs input m0 and m1 by making calls to read_matrix . The paths to the matrices are passes
in the command-line. Remember to store them and pass two integer pointers as arguments.
2. Next, you’ll want to use those three matrices to calculate the scores for our input. Our network consists of a matrix
multiplication with m0 , followed by a relu on the result.
3. Then a second matrix multiplication with m1 . At the end of this, we will have a matrix of scores for each classification.
4. Finally, we pick the index with the highest score, and that index is the classification for our input.
Note that when calling argmax and relu , you should treat your inputs as 1D arrays. That means the length you pass into the
function should be the number of elements in your entire matrix.
hidden_layer = matmul(m0, input). Input: [128,784] * [784,1], Output = [128*1]
relu(hidden_layer) # Recall that relu is performed in-place Input [128,1] = Output [128,1]
scores = matmul(m1, hidden_layer) Input: [10,128] * [128,1] Output : [10:1].
argmax(scores)
Validation
Validating Simple
Apart from MNIST, we’ve provided several smaller input networks for you to run your main function on. simple0 , simple1 ,
and simple2 are all smaller networks that will be easier to debug.
# Testing simple1. input0
java -jar venus.jar ./test_files/main.s ./inputs/simple1/bin/inputs/input0.bin ./inputs/simple1/bin/m0.bin ./inputs/simple1/bi
n/m1.bin -ms -1
# To validate
java -jar venus.jar ./test_files/main.s ./inputs/simple1/bin/inputs/input0.bin ./inputs/simple1/bin/m0.bin ./inputs/simple1/bi
n/m1.bin -ms -1 > ./out/simple1/input0.trace
python3 part2_tester.py simple1/input0
Note that these files cover a variety of dimensions. For example the simple2 inputs have more than one column in them,
meaning that your ‘scores’ matrix will also have more than one column. Your code should still work as expected in this case,
writing the matrix of “scores” to a file, and printing a single integer that is the row-major index of the largest element of that
matrix.
Validating MNIST
For MNIST, there are two additional folders:
txt/labels/ contains the true labels for each input, which are the actual digits that each corresponding input image
contains.
student_inputs/ contains a script to help you run your own input images, as well as an example.
All the files for testing the mnist network are contained in inputs/mnist . There are both binary and plaintext versions of m0 ,
m1 , and 9 input files (inputs/mnist/bin/inputs/mnist_input[0-9]*.bin).
To test on the first input file for example, run the following:
# testing input 0.
java -jar venus.jar ./test_files/main.s ./inputs/mnist/bin/inputs/mnist_input0.bin ./inputs/mnist/bin/m0.bin ./inputs/mnist/bi
n/m1.bin -ms -1 > ./out/mnist/mnist_input0.trace
python3 part2_tester.py mnist/mnist_input0
# you can try inputs from mnist_input[0-9]. The labels for each input can be found in mnist/txt/labels/
# (Note that we run with the `-ms -1` flag, as MNIST inputs are large and we need to increase the max instructions Venus will r
un)
This should print out the digit with the highest score. You can compare the printed digit versus the one in
inputs/mnist/txt/labels/label0.txt . It will also print out the output of each stage.
Not all inputs will classify properly. A neural network will practically never have 100% accuracy in its predictions. In our test
cases specifically, mnist_input2 and mnist_input7 will be misclassified to 9 and 8 respectively. All other test cases should
classify correctly.
To check the final label provided. We have also included a python script that implements MNIST.
python3 mnist.py ./inputs/mnist/bin/inputs/mnist_input0.bin ./inputs/mnist/bin/m0.bin ./inputs/mnist/bin/m1.bin
6
You can check the printed digit printed by main against the plaintext labels for each of the input files in the
mnist/txt/labels folder.
We’ve also included a script inputs/mnist/txt/print_mnist.py , which will allow you to view the actual image for every
mnist input. For example, you can run the following command from the directory inputs/mnist/txt to print the actual image
for mnist_input8 as ASCII art alongside the true label.
cd inputs/mnist/txt/
python3 print_mnist.py 8
End-to-End tests
$ bash ./scripts/localci.sh
# if you see SUCCESS and *.log.sucess then you passed. You can also check your *_Grade.json to see your tentative grade.
# If you see FAILED, then inspect *.log.failed. Check the failed section to see what tests failed.
Remember from the testing framework section that these sanity tests are not comprehensive, and you should rely on
your own tests to decide whether your code is correct. Your score will be determined mostly by hidden tests that will be
ran after the submission deadline has passed.
We will also be limiting the number of submissions you can make to the travis-ci. Each test can take up to 5-6 minutes. To give
the 150-200 students a fair chance. For any given 2 hour period, you’re limited to 6 submissions.
Overall, there aren’t that many edge cases for this project. We’re mainly testing for correctness, RISC-V calling convention, and
exiting with the correct code in the functions where you’ve been instructed to.
Grading
Test Points
test_relu 10
test_argmax 10
test_dot 10
test_matmul 10
test_read_matrix 10
simple0/input0 15
simple0/input1 15
simple0/input2 15
simple1/input0 15
simple1/input1 15
simple1/input2 15
simple2/input0 15
simple2/input1 15
simple2/input2 15
mnist/mnist_input0 40
mnist/mnist_input1 40
mnist/mnist_input2 40
Test Points
mnist/mnist_input3 40
mnist/mnist_input4 40
mnist/mnist_input5 40
mnist/mnist_input6 40
mnist/mnist_input7 40
mnist/mnist_input8 40
Debugging hints
unable to venusbackend.assembler.AssemblerError: Could not find the library: Check the path of the
imported files.
unable to create ./out/….trace : Check if the out/mnist, out/simple0, out/simple1 and out/simple2 folders exist.
File read errors : Note that the file paths we specify ./ are relative and assume you are in the top folder. You should
include complete path when in web-based venus In web-based venus make sure you are at the top of the folder to invoke
in the same manner as described here. File paths if in root repo (https://lucid.app/lucidchart/4c174be4-a85a-4757-a2c2-
0bd6c3162783/view?page=0_0#). If you are in a different folder (e.g., test_files then relative path will be ../ )
Ran for more than max allowed steps! : Venus will automatically terminate if your program runs for too many steps.
This is expected for large MNIST sized inputs, and you can workaround it with the -ms flag. If you’re getting this for small
inputs, you might have an infinite loop.
Attempting to access uninitialized memory between the stack and heap. : Your code is trying to read or write
to a memory address between the stack and heap pointers, which is causing a segmentation fault. Check that you’re
allocating enough memory, and that you’re accessing the correct addresses.
Assembler errors : You cannot have any .word and .data sections in your included files.
Check your calling convention. We show an example below on how to use venus to check the calling convention
cd $REPO
java -jar venus.jar -cc –retToAllA ./test_files/main.s ./inputs/simple2/bin/inputs/input2.bin ./inputs/simple2/bin/m0.bin ./in
puts/simple2/bin/m1.bin -ms -1 > ./out/simple2/input2.trace
# You forgot to save and restore a register that is expected. Check your prologue and epilogue in matmul.s
Save register s8 not correctly restored before return! Expected 0x10008030, Actual 0x00000000. ../matmul.s:91 ret
# You have not written to t3 but are trying to use it. Initialize or see what t3 should be.
# t3 is being used in line 150
150 Usage of unset register t3! ./test_files/main.s
Generating Your Own MNIST Inputs
Just for fun, you can also draw your own handwritten digits and pass them to the neural net. First, open up any basic drawing
program like Microsoft Paint. Next, resize the image to 28×28 pixels, draw your digit, and save it as a .bmp file in the directory
/inputs/mnist/student_inputs/ .
Inside that directory, we’ve provided bmp_to_bin.py to turn this .bmp file into a .bin file for the neural net, as well as an
example.bmp file. To convert it, run the following from inside the /inputs/mnist/student_inputs directory:
python bmp_to_bin.py example
This will read in the example.bmp file, and create an example.bin file. We can then input it into our neural net, alongside the
provided m0 and m1 matrices.
java -jar venus.jar main.s -ms -1 -it inputs/mnist/student_inputs/example.bin inputs/mnist/bin/m0.bin inputs/mnist/bin/m1.bin
You can convert and run your own .bmp files in the same way. You should be able to achieve a reasonable accuracy with your
own input images.
Acknowledgments
This assignment has been modified the CMPT 750 instructor and the authors of RISC-V.
https://lucid.app/lucidchart/4c174be4-a85a-4757-a2c2-0bd6c3162783/view?page=0_0#
Last updated October 31, 2021.
Derived from github (https://github.com/mt-class/jhu) and modified by (https://www.cs.sfu.ca/~ashriram)Arrvindh Shriraman
(https://github.com/ashriram)
https://github.com/mt-class/jhu
https://www.cs.sfu.ca/~ashriram
https://github.com/ashriram