Services and the Web
Brief notes, mostly terminology, for students without previous experience.
Includes extracts from slides for Construction from the Assignment overview
Copyright By PowCoder代写 加微信 powcoder
Already saw (Django Environment slides) that SaaS clients and servers connect over TCP/IP using HTTP.
The client sends a request to access or operate on a resource. Composition of the request:
Request line specifying an HTTP route and the HTTP version number. Optional set of headers.
Blank line to indicate all meta-information has been sent
Optional body (GET requests do not have a body)
COMP3297: Software Engineering 2
HTTP Route
Describes the action to be performed on the resource and identifies the resource.
The action is specified as an HTTP method such as GET, POST, PUT, PATCH, DELETE
The resource is identified by its URL (more correctly, its URI).
Optionally, can include a query string, preceded by a “?” separator which is typically followed by one or more name/value pairs. Query strings are not standardized.
Example from the Overview to request a resource by making an API call:
GET https://api.data.gov.hk/v2/filter?q={“resource”:”http: … }
COMP3297: Software Engineering 3
method Scheme Hostname Resource path Query string
Prefix Resource Parameter Parameter
or service
name value
GET https://api.data.gov.hk/v2/filter?q={“resource”:”http: … }
API base URL
An endpoint represents the location or a resource or service. Detailed in API reference.
The prefix, here “v2” (just a digit is also common, e.g. “5”), is a version number. Having it as part of the URL allows the publishers of the API to extend and release new versions without breaking older clients. For example, GOV.HK still maintains “v1” of the API.
Base URLs may take other forms. For example: https://xxxx.gov.hk/api/5/ is a common form.
COMP3297: Software Engineering 4
JSON is a format for data interchange and a file format.
There are many formats that could be used to exchange data between service providers and clients. The industry has converged on JSON as a common default format for data interchange.
It is simple, human-readable, and, although it is based on JavaScript object literal syntax (hence the name JavaScript Object Notation), it is built on two data structures that have equivalents in almost all programming languages currently in use. Thus, it is language-independent.
Those two JSON structures are:
An object, which is an unordered set of name/value pairs enclosed by braces, { }. In Python, the same structure is realized as a dictionary.
An array, which is an ordered collection of values enclosed in square brackets, [ ]. In Python, this is realized as a list.
A value may be a string (double-quoted, “”), a number, true, false or null. Or
an object. Or an array.
COMP3297: Software Engineering 5
JSON and the DATA.GOV.HK API
From your examination of the API spec in Inception, and your spikes in Construction, you already know:
• To specify the data you wish to retrieve, you must provide a query string in your GET request.
• That query string will assign a value to the parameter q.
• The spec indicates the value must be a URL-encoded, serialized JSON object. Remember, in Python, JSON objects are realized as dictionaries.
And so, to construct the GET request in Python, you must:
Create a dictionary containing the name/value pairs for the JSON object that you
will assign as the value of q as described in the API spec. Serialize the dictionary to a JSON-formatted string.
Build and execute the GET request. This is the API call to the DATA.GOV.HK endpoint.
COMP3297: Software Engineering 6
Construct the GET request
Like most modern languages, Python provides native support for encoding and
decoding JSON data. In Python it is provided by the built-in package json. Use the function json.dumps() to serialize your dictionary object for the query
string to a JSON formatted str object.
Now you want to build the GET request, URL-encode and execute it. From
earlier, you want a request in the form:
GET https://api.data.gov.hk/v2/filter?q={“resource”:”http: … }
endpoint your JSON object string
Using the Requests library, for example, you can do all of the above with the
function requests.get():
resp = requests.get(‘endpoint’, params={‘q’: your JSON object string}) COMP3297: Software Engineering 7
The response
Remember (also from Django Environment slides) an HTTP response, just like a request, contains:
A status line, part of which is the 3-digit status code. Optional set of headers describing the response body Blank line to indicate all meta-information has been sent Optional body
The responses for some requests don’t require a body, but for most requests the body of the response contains the requested resource, or details of the status of the requested action.
COMP3297: Software Engineering 8
Handling the response
For your request to api.data.gov.hk, the JSON encoded response will represent one or more rows of data, each in the form of a JSON object which you can deserialize to a dictionary and extract the values of interest.
Thus the JSON content you receive will be in the form of an array of objects. In Requests we can use the built-in JSON decoder to obtain this from the Response
object as a list of dicts:
result = resp.json()
E.g. request for first 5 days of March, obtains a list containing 5 dicts
[{‘As of date’: ’01/03/2022′, … }, {‘As of date’: ’02/03/2022′, … }, {‘As of date’: ’03/03/2022′, … }, {‘As of date’: ’04/03/2022′, … }, {‘As of date’: ’05/03/2022′, … }]
Alternatively, we can use the function json.loads() from the json module to deserialize the JSON formatted response body. It gives the same result.
Finally, to check the response status code, you can obtain it as resp.status_code if using Requests, or resp.status if using urllib (version 3.9 onwards)
COMP3297: Software Engineering
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com