Individual Pre-project and related Assignments
The Pre-project
A one-month piece of work, designed to:
build familiarity with various aspects of Django and deploying to Heroku. give some experience of consuming 3rd party APIs
With small embedded assignments to practise creating key agile artefacts:
Vision Document
Wireframe UI Prototypes
Product Backlog as a set of User Stories
12%
COMP3297: Software Engineering
2
5%
The Pre-project
Not a true Individual SE Project:
Structured more like an assignment A little artificial in parts
Only casual use of process
Application has two main responsibilities:
Allow user to create (as in CRUD) and persist Location records.
Will use Django Forms and a Location model class
Retrieve data from 3rd party API, calculate derived data, and display
Your goal should be to produce a Minimum Viable Product with just sufficient core functionality to be usable. It should satisfy Research Group Members’ needs requested for the first release.
COMP3297: Software Engineering 3
Timeline
Establish the Vision Create wireframes Initial Product Backlog Explore API
Refine Vision Refine Backlog First increment
Refine Vision Refine Backlog Second increment
Assemble definitive set of User Stories Deploy application
Inception
Construction 1
Feature 1
Construction 2
Feature 2
Transition
COMP3297: Software Engineering
DELIVERABLES
4
Wireframes
Refined Vision
Complete Backlog as User Stories
Deployed Solution Source Code
API
https://data.gov.hk/en-data/dataset/hk-dh-chpsebcddr-novel-infectious-agent
Minimal Data Dictionary – source for formats
Data source we’ll use
…
COMP3297: Software Engineering
5
API
https://data.gov.hk/en-data/dataset/hk-dh-chpsebcddr-novel-infectious-agent/resource/cc01597e-8a77-4c83-bb51-c76b7d93d854
URL of the resource we’ll use
To explore query string format
COMP3297: Software Engineering 6
API
Query builder. Specify filtering and sorting rules here
See resulting query here
COMP3297: Software Engineering 7
API
Rule example
Resulting query request
Result in response
API Spec
COMP3297: Software Engineering 8
API Spec
API endpoint
Specification for JSON object to be encoded as query string
COMP3297: Software Engineering 9
Example from the Spec of what you need to specify
[After class, a classmate mentioned they had no previous experience consuming APIs in Python. That’s fine and we’ll help when needed.
It’s a commonly required skill and that is exactly why it is part of the Individual work for the course – so that everyone will gain that experience.
The next few slides have been extended to add a few more Python-related details.]
COMP3297: Software Engineering 10
During Inception: determine required format
Drive down risks in working with the API
Review the DATA.GOV.HK API Spec.
Write a Python script to check you can retrieve the required COVID-19 case statistics successfully.
First determine the required name/value pairs for the JSON string argument you’ll use to assign a value to the query string parameter q as follows:
Use the API Builder to generate a query request.
Examine the generated URL and the key/value pair passed in the query string. E.g.
q=%7B%22resource%22%3A%22http%3A%2F%2Fwww.chp.gov.hk%2Ffiles%2F misc%2Flatest_situation_of_reported_cases_covid_19_eng.csv%22%2C%22secti on%22%3A1%2C%22format%22%3A%22json%22%2C%22filters%22%3A%5B% 5B1%2C%22eq%22%2C%5B%2225%2F12%2F2021%22%5D%5D%5D%7D
This is, of course, URL-encoded. You can decode it, or just examine it directly to identify the name/value pairs in the serialized JSON object.
COMP3297: Software Engineering 11
During Inception: build and make a GET request
Now you have confirmed the format of the JSON object, you can write a Python script to make an identical query.
Create a Python dictionary containing key/value pairs required in the JSON string. Serialize the dict object to a JSON formatted string. JSON encoding/decoding is
provided by the built-in Python module json.
Use the function json.dumps() to serialize your dict object.
Then build the URL for the GET request specifying the API endpoint and a query string assigning your JSON string as the value associated with key q
Simplest way to do this is to use the Requests library which elegantly wraps many of the useful functions of urllib package modules. Or just use urllib modules if you prefer.
Using Requests, the function requests.get() will build, URL-encode, and make the GET request.
COMP3297: Software Engineering 12
During Inception: check the response
You now have a Response object.
Requests contains a built-in JSON decoder. You can use this to deserialize to a dict from which you can extract the values of interest.
The names for the name/value pairs in the response content can be seen on the next slide.
Or you can use the function json.loads() from the json module to deserialize the JSON formatted string of the response content.
Now check that you got what you expected, as shown on the following slide
In your actual application, the user wants to know if the request made to the API endpoint was successful. To do that, you must check the status code in the response.
For a response object resp, this can be obtained as resp.status_code if using Requests, or resp.status if using urllib (version 3.9 onwards)
The entire script is just a few lines of Python.
COMP3297: Software Engineering 13
Example For Dec 25, 2020:
Deserialized object from response (code updated after class to take the date as argument):
COMP3297: Software Engineering 14
For an easy life:
Don’t be a Deadline Fighter. You have a month.
If you leave everything until Transition you will die!
Keep it simple. All that is required is an MVP. The simpler the solution, the easier the deployment.
We will test your deployed solution. Don’t assume your deployment is guaranteed to go smoothly – that would be taking a big risk.
Deploy a partial product early as a check.
COMP3297: Software Engineering 15