Project 3
CMSC421: Intro to Artificial Intelligence
Which one doesn’t belong?
Due: May 13th 2019
1 Introduction
**Updated on 04/26/19**
For this project, you will be required to make use of the AI systems that we have created in the previous projects to accomplish a higher level task. The main task that your system must accomplish is to identify the image in a set of images that ”doesn’t belong”.
This task involves two main steps. The first step is to integrate the infer- ence system from project 2 with the pre-trained CIFAR-10 nnet model that we have provided, by having it predict the subjects of images. The second step is an inference task, in which you must make use of a knowledge base of rules like that of project 2 to identify the subject in the set of images that does not belong with the others.
2 Example
You will be provided with sets of pictures similar to the following. (Note that these pictures are not necessarily in the CIFAR-10 data set).
1
As you can (hopefully) tell, these pictures correspond to a bird, cat, dog, and horse respectively. Once your CNN identifies this, you will have to make inferences using a knowledge base that we will provide. As such, you may be provided with the following rules:
• Adogisamammal
• A cat is a mammal
• A horse is a mammal
• A mammal is an animal
• Abirdisatheropod
• A theropod is an animal
• A theropod is not a mammal
From this set of rules, you must determine which subject is the odd one out. As you might notice, the dog, cat, and horse are all mammals, but the bird is not a mammal (a theropod is not a mammal).
3 Recognition and Inference
CIFAR-10 is a dataset of 60000 32×32 color images with 10 classes. We have provided a pretrained CNN which takes this dataset as input. You will be tasked with integrating this model with the inference system. As an example, your inference system should be able to answer the question ”is a puppy1.jpg an animal” given the rule ”a dog is an animal. This will be necessary in two functions: the answer wodb function and the answer query function.
EDIT: The provided modified project 2 code has been altered to make it easier to read. The code has been simplified and more heavily commented. IF YOU ARE MORE COMFORTABLE WITH YOUR OWN CODE FROM
2
PROJECT 2, YOU ARE ALLOWED TO SUBSTITUTE IT IN, just be sure everything still works, including the old tests! Furthermore, you’ll need a python module named ”imageio”. This can be easily downloaded using pip for python3. Visit this website for instructions on installing pip, except don’t forget to use python3 and not just python. Once pip is installed, you can run the following command to download imageio:
python3 -m pip install imageio
In this portion of the task, you must make inferences about which recognized subject is the odd one out using the provided knowledge base. We have provided you with a pre-implemented and modified version of project 2 which you will have to edit. Every provided set of subjects will have exactly one correct answer given the rules in the knowledge base. All of the rules about the behavior of the knowledge base from project 2 still apply in this project.
Note that you will have to use self.sess.run(fetches, feed dict) to predict values. If you would like more information on how to use this function, reference the tensorflow documentation at this link.
EDIT: You now will have access to a function called classify image in the kb.py code. This function will take in a loaded image and will return a list of probabilities for each class for the image from the neural network. You’ll need to use this tool to figure out which class has the highest probability and thus which class the network thinks is most likely for the input image. As mentioned below, if the confidence for all classes for an image is below 0.7, then you’ll need to return the LOWCONF response. If you successfully classify an image and it has a high enough confidence, you’ll need to treat this image as its class in the kb. This requires creating a unique moniker for the class that can’t be typed in as a regular/proper noun (similar to when you differentiated between regular/proper nouns in project 2).
3.1 add to kb
You will have to edit this function such that if it detects an image then it will
predict its class and proceed normally.
You may use the self.load image(image) function to load the image before passing it into the CNN.
DO NOT EDIT ANY OTHER PART OF THIS FUNCTION THAN IS INDICATED IN THE CODE! EDIT: IF YOU CHANGE CODE TO YOUR CODE FROM PROJECT 2, just be careful to make sure the new tuple numbering
is correct and that you account for loading in images properly.
3
3.2 answer query
You will have to edit this function such that if it detects an image then it will
predict its class and proceed normally.
You may use the self.load image(image) function to load the image before passing it into the CNN.
DO NOT EDIT ANY OTHER PART OF THIS FUNCTION THAN IS INDICATED IN THE CODE! EDIT: IF YOU CHANGE CODE TO YOUR CODE FROM PROJECT 2, just be careful to make sure the new tuple numbering
is correct and that you account for loading in images properly.
3.3 answer wodb
You will have to implement the entirety of this function.
This function will be called by the shell. It takes a string of comma sepa- rated names like in the following example: dog,cat,pony1.png,car baby.jpg
You must parse each value and determine whether it is an image. If it is, then you must use the model to predict its class.
You may use the self.load image(image) function to load the image before pass- ing it into the CNN.
• It should then be able to answer the ”Which of these doesn’t belong? ” question.
• Your model should be able to answer questions that have either images, simple names, or both.
• If the image does not exist, you should return the NOIMG response.
• If the confidence is below a threshold of 0.7, then you should return the
LOWCONF response.
• If there is no way to determine which one does not belong, then you should return the NODIFF response.
• Otherwise, your function should determine the odd one out and return its name as a string.
4 Code
EDIT: The code in this section will be very similar to that of projects 1 and 2. You will be provided with the files kb.py, parser proj3.py, shell.py, run all tests.py, run old tests.py, run new tests.py.
4
The kb.py file will contain the function
The parser proj3.py file contains a parser like that of project 2. DO NOT
CHANGE THIS FILE.
The shell.py file contains a shell that will allow you to input queries and facts, as well as ask questions of the form ”Which of these does not belong?”. DO NOT CHANGE THIS FILE
For information on how to run shell.py, please consult the project descrip- tion for project 2.
cifar10.py and cifar10 input.py deal with actually running the model to predict values. DO NOT CHANGE THIS FILE.
5 Testing
Tests will cover a variety of cases, including that your program is returning the correct failure responses, and is making the correct inferences.
To run all the public tests, simply run the following command in the termi- nal:
python3 run all tests.py
If you’d rather only run the old tests or the new tests run, respectively (note the old|new below is meant to be one or the other, not both:
python3 run old|new tests.py
You can also generate output for new tests by creating tests of the form test¡x¿.input.txt
and running the command:
python generate tests.py
6 Deliverables
You will turn in your updated kb.py file.
5