COMP5347: Web Application Development Web Services
Dr. Basem Suleiman
School of Computer Science
The University of Sydney
Page 1
COMMONWEALTH OF Copyright Regulations 1969 WARNING
This material has been reproduced and communicated to
you by or on behalf of the University of Sydney pursuant to Part VB of the Copyright Act 1968 (the Act).
The material in this communication may be subject to copyright under the Act. Any further reproduction or communication of this material by you may be the subject of copyright protection under the Act.
Do not remove this notice.
The University of Sydney Page 2
Outline
– Service Oriented Architecture – Web Services
– SOAP Services and RESTful Services
– Creating REST web services in Expressjs Application – Consuming REST web services in Expressjs application
The University of Sydney
COMP5347 Web Application Development
Page 3
Service-Oriented Architecture (SOA)
“a style of software design where services are provided to the other components by application components, through a communication protocol over a network” – Wikipedia (May 2019)
– Paradigm, approach or style
– Can help to architect applications so that certain quality
attributes are satisfied
– Services are the core of this approach
The University of Sydney
Page 4
SOA – Web Applications
– Building large-scale integrated applications from services – Specific enough to have clear focus
– Integration and communication
– Platform independence
– Abstracted from end users
– Avoid rewriting legacy code
Supplier 2
Linux/ EJB
Reseller
On-line retail application (J2EE/IIOP/JMS)
Web Server
Smart client app
Browser-based client
The University of Sydney
Page 5
Supplier 1
(Win2K3 .NET)
User authentication
Win2K3 EJB
Shipper 1
(Win2K/ MQ)
Shipper 2
(Solaris/ CORBA)
Credit Card processor
(Win2K3/ COM+)
Bank
(MVS/CICS)
Outline
– Service Oriented Architecture – Web Services
– SOAP Services and REST Services
– Creating REST web services in Expressjs Application – Consuming REST web services in Expressjs application
The University of Sydney
COMP5347 Web Application Development
Page 8
Web Services
– Relatively standardized mechanism by which one piece of software application can connect and communicate with another using web protocols
– It provides a simple and open way of integrating functions or data from various systems
– It can be used within an organization and/or across the public Internet
– At least two implementations: SOAP-based vs. RESTful services
– Original design of Web Services is very application-centric in contrast to the resource-centric Web and REST style
The University of Sydney
COMP5347 Web Application Development
Page 9
A Service-based Application
The University of Sydney Page 10
Web Service Standards
– Simple Object Access Protocol (SOAP): a messaging protocol for transferring information (XML format)
– Web Service Description Language (WSDL): A standard for describing Web services (XML format)
– Interfaces, methods, parameters, request, response
– Generated and consumed by tools
– Universal Description, Discovery and Integration (UDDI): a registry and protocol for publishing and discovering web services
– Like white, Yellow and Green ‘pages’
– Not really used
– Also, WS-Policy framework for non-functional properties (e.g., security, reliability)
The University of Sydney
COMP5347 Web Application Development
Page 11
Calling A Service – SOAP
– SOAP – Simple Object Access Protocol!
– Originally simple remote procedure calls using XML-
formatted protocol
– Envelope, header, body – Extensible protocol
• Add new header elements for new semantics – Very widely accepted standard
Envelope (Mandatory) –
Marks the start and end of a message
Header (Optional) –
General information about message – e.g. authentication and transaction management
Body (Mandatory) –
Data for the actual message or document being sent
The University of Sydney
Page 12
SOAP Example – Stock Quote
GET /StockQuote HTTP/1.1
Host: www.stockquoteserver.com
Content-Type: text/xml
Content-Length: nnnn
SOAPMethodName: Stock-Namespace-URI#GetLastTradePrice
HTTP/1.1 200 OK
Content-Type: text/xml
Content-Length: nnnn
The University of Sydney Page 13
Publishing and Consuming a Service
The University of Sydney Page 14
WSDL Structure
definition of types……..
definition of a message….
definition of a binding….
The University of Sydney
Page 16
SOAP-based Services – Workflow
The University of Sydney Page 17
Web Service APIs – Examples
– Twitter API
– https://dev.twitter.com/rest/public
– MediaWiki API
– https://www.mediawiki.org/wiki/API:Main_page
– Flickr API
– http://www.flickr.com/services/api/
– Google Maps – Directions APIs
– https://developers.google.com/maps/documentation/directions/intro
– New York Times API
– http://developer.nytimes.com/docs
– Youtube API
– https://developers.google.com/youtube/getting_started#data_api
The University of Sydney
COMP5347 Web Application Development
Page 19
REpresentation State Transfer (REST)
– REST is an architectural style rather than a strict protocol
– REST-style architectures consist of clients and servers
– Requests and responses are built around the transfer of representations of resources
– A resource can be essentially any coherent and meaningful concept that may be addressed
– A representation of a resource is typically a document that captures the current or intended state of a resource
Based on Roy Fielding’s doctoral dissertation, rephrased by Wikipedia http://en.wikipedia.org/wiki/Representational_State_Transfer
The University of Sydney
COMP5347 Web Application Development
Page 20
REST – API Format (Early Days)
– Many early day RESTful API’s URL has a format consists of:
– API end point (a concept coming from SOAP)
– a parameter to specify the action: query, update, etc..
– and many action specific parameters
– Most of which are expressed as query strings
– Many API providers provide API sandbox or API explorer to help developer
build the request URL – Example:
– https://api.flickr.com/services/rest/?method=flickr.test.echo&n ame=value
– https://en.wikipedia.org/w/api.php?action=query&name=value
The University of Sydney
COMP5347 Web Application Development
Page 21
REST – API Common Format
– The commonly agreed REST API URL format conforms to general web architecture
– Using URI/URL to specify resources
– Using HTTP method to indicate action
– A URI as a resource identifier is one of the central concepts of WWW
– The representation of an available resource, identified by a URI, is fetched
using an HTTP GET request without affecting the resource in any way
– The simplicity and scalability of the Web is largely due to the fact that
there are a few “generic” methods (GET, POST, PUT, DELETE)
The University of Sydney
COMP5347 Web Application Development
Page 22
REST – Basic Design Principles
– Use HTTP methods explicitly to interact with any resource on the Web
– GET: query the state
– POST: create a resource
– PUT: modify or create a resource
– DELETE: delete a resource
– Be stateless
– Address the resources explicitly in the request message
– Expose directory structure-like URIs
– http://www.myservice.org/discussion/topics/{topic}
– Transfer XML, JSON, or both
http://www.ibm.com/developerworks/webservices/library/ws-restful/
The University of Sydney
COMP5347 Web Application Development
Page 23
REST – Stateless vs. Stateful
http://www.ibm.com/developerworks/webservices/library/ws-restful/
The University of Sydney
COMP5347 Web Application Development
Page 24
Resource Types
– A resource is something “interesting” in your system – E.g., Blog posting, transaction, Printer
– Most of the time we can differentiate between collection type of resources and individual resource
– Revisions and revision
– Articles and article
– The URL’s directory structure is based on that
– This can be nested and it is up to developers to decide the nesting direction – /movies/ForrestGump/actors/TomHanks
– /directors/AngLee/movies/LifeOfPi
The University of Sydney
COMP5347 Web Application Development
Page 25
Web Services – MediaWiki API
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=ids|timestam p&rvstart=2016-12-01T00:00:00Z&rvend=2017-01- 01T00:00:00Z&rvdir=newer&format=jsonfm&titles=cat&rvlimit=max
{
“batchcomplete”: “”,
“query”: {
“normalized”: [
{
“from”: “cat”,
“to”: “Cat” }
], “pages”: {
“6678”: {
“pageid”: 6678,
“ns”: 0,
“title”: “Cat”,
“revisions”: [
{
“revid”: 752709621,
“parentid”: 752304215, “timestamp”: “2016-12-02T20:51:06Z”
}, {
},
The University of Sydney
COMP5347 Web Application Development
Page 26
“revid”: 752713783,
“parentid”: 752709621, “timestamp”: “2016-12-02T21:17:08Z”
REST – Request URLs and Methods
Action
URL path
Parameters
Example
Create new revision
/revisions
http://localhost:3000/revisions
Get all revisions
/revisions
http://localhost:3000/revisions
Get a revision
/revisions
revision_id
http://localhost:3000/revisions/123
Update a revision
/revisions
revision_id
http://localhost:3000/revisions/123
Delete a revision
/revisions
revision_id
http://localhost:3000/revisions/123
Request Method
Use case
Response
POST
Add new data in a collection
New data created
GET
Read data from data source
Data objects
PUT
Update existing data
Updated object
DELETE
Delete an object
NULL
The University of Sydney
COMP5347 Web Application Development
Page 27
Outline
– Service Oriented Architecture – Web Services
– SOAP Services and REST Services
– Creating REST web services in Expressjs Application – Consuming REST web services in Expressjs application
The University of Sydney
COMP5347 Web Application Development
Page 28
Create REST API in Express.js
– Most Web frameworks provide feature to support developing Web services
– Express.js
– Additional route feature: route parameters
– Route parameters are named URL segments that are used to capture the values specified at their position in the URL
– The values are populated in req.params object
The University of Sydney
COMP5347 Web Application Development
Page 29
Create REST API in ExpressJs
– Example
– Routepath:/users/:userId/books/:bookId
– RequestURL:http://localhost:3000/users/34/books/8989 – req.params: { “userId”: “34”, “bookId”: “8989” }
app.get(‘/users/:userId/books/:bookId’, function (req, res) {
res.send(req.params)
})
The University of Sydney
COMP5347 Web Application Development
Page 30
Specifying Client Data
– Sending data from client to server
– Route parameter
• Routepath:/users/:userId/books/:bookId
• Url:http://localhost:3000/users/34/books/8989 • req.params.userId
• req.params.bookId
– Query String
• url: http://localhost:3000/usersbooks?userId=34&bookId=8989 • req.query.userId
• req.query.bookId
– Request body
• data {userId:34, bookId:8989} is sent as part of request body • if using body-parser middleware
– req.body.userId – req.body.bookId
The University of Sydney
COMP5347 Web Application Development
Page 31
Create REST API Using Express.js
RevisionSchema.statics.getByTitle = function(title, callback){ return this.find({‘title’:title}).exec(callback)
}
model
module.exports.getByTitle=function(req,res){
title = req.params.title Revision.getByTitle(title,function(err,result){
controller
if (err){
console.log(“Cannot find revisions of title: ” + title)
}else{ res.json(result)
} })
}
route
router
.get
(‘/revisions/:title’, controller.getByTitle)
The University of Sydney
COMP5347 Web Application Development
Page 32
RESP API – Response
http://localhost:3000/revision/revisions/BBC
app.use(‘/revision’,revroutes)
The University of Sydney
COMP5347 Web Application Development
Page 33
Outline
– Service Oriented Architecture – Web Services
– SOAP Services and REST Services
– Creating REST web services in Expressjs Application – Consuming REST web services in Expressjs application
The University of Sydney
COMP5347 Web Application Development
Page 34
Consume REST API in Express.js
– Complex API calls may benefit from using a package that wraps up the API
– Simple GET type of queries can always be implemented using general modules designed for handling http requests
– Corenode.jsmodules:http,https – requestmodule
– The request module (https://github.com/request/request)
– To install
• npm install request –save
– To make a request: request(options, callback)
The University of Sydney
COMP5347 Web Application Development
Page 35
–
Consume REST API in Express.js
The request module (https://github.com/request/request)
– To install
• npm install request –save
– To make a request: request(options, callback)
The University of Sydney
COMP5347 Web Application Development
Page 36
Call Wikipedia API – Code Example
var request = require(‘request’);
var wikiEndpoint = “https://en.wikipedia.org/w/api.php”, parameters = [“action=query”,
“format=json”,
“prop=revisions”, “titles=australia”, “rvstart=2016-11-01T11:56:22Z”, “rvdir=newer”,
“rvlimit=max”,
“rvprop=timestamp|userid|user|ids”] var url = wikiEndpoint + “?” + parameters.join(“&”) console.log(“url: ” + url)
var options = {
url: url,
Accept: ‘application/json’, ‘Accept-Charset’: ‘utf-8’
}
End point
Action and all parameters
Constructing an URL
Wikipedia PAI documentation: https://www.mediawiki.org/wiki/API:Main_page
The University of Sydney COMP5347 Web Application Development
Page 37
Request header
Wikipedia API Call – Making Request
request(options, function (err, res, data){ if (err) {
Send request
console.log(‘Error:’, err); Call back function } else if (res.statusCode !== 200) {
console.log(‘Status:’, res.statusCode); } else {
json = JSON.parse(data); Convert JSON format string into JavaScript object pages = json.query.pages
revisions = pages[Object.keys(pages)[0]].revisions console.log(“There are ” + revisions.length + ” revisions.”); var users=[]
Object.keys(obj) returns a array of obj’s property for (revid in revisions){ names. We only need the first one.
users.push(revisions[revid].user); }
uniqueUsers = new Set(users);
console.log(“The revisions are made by ” + uniqueUsers.size + ”
unique users”);
}
});
The University of Sydney
COMP5347 Web Application Development
Page 38
MediaWiki API
– https://en.wikipedia.org/w/api.php?action=query&prop=revisions&r vprop=ids|timestamp&rvstart=2016-12-01T00:00:00Z&rvend=2017-01- 01T00:00:00Z&rvdir=newer&format=jsonfm&titles=cat&rvlimit=max
{
“batchcomplete”: “”,
“query”: {
“normalized”: [
{
“from”: “cat”,
“to”: “Cat” }
], “pages”: {
“6678”: { “pageid”: 6678,
“ns”: 0, “title”: “Cat”, “revisions”: [
{
“revid”: 752709621,
“parentid”: 752304215, “timestamp”: “2016-12-02T20:51:06Z”
}, {
},
The University of Sydney
COMP5347 Web Application Development
Page 39
“revid”: 752713783,
“parentid”: 752709621, “timestamp”: “2016-12-02T21:17:08Z”
Calling Wikipedia API – https Version
var https = require(‘https’)
var wikiEndpointHost = “en.wikipedia.org”, path = “/w/api.php”
parameters = [“action=query”,
“format=json”, “prop=revisions”, “titles=australia”, “rvstart=2016-11-01T11:56:22Z”, “rvdir=newer”,
“rvlimit=max”,
“rvprop=timestamp|userid|user|ids”], headers = {
Accept: ‘application/json’,
‘Accept-Charset’: ‘utf-8’ }
Slightly different way of specifying options
var full_path = path + “?” + parameters.join(“&”)
var options = {
host: wikiEndpointHost, path: full_path, headers: headers}
All header information are grouped as an object
https://nodejs.org/api/https.html The University of Sydney
COMP5347 Web Application Development
Page 40
https version (cont’d)
https.get(options,function(res){ var data =”;
res.on(‘data’,function(chunk){ data += chunk
}) res.on(‘end’,function(){
json = JSON.parse(data);
pages = json.query.pages
revisions = pages[Object.keys(pages)[0]].revisions console.log(“There are ” + revisions.length + ” revisions.”); var users=[]
for (revid in revisions){
users.push(revisions[revid].user); }
uniqueUsers = new Set(users);
console.log(“The revisions are made by ” + uniqueUsers.size + ” unique users”); })
}).on(‘error’,function(e){ console.log(e)
})
The University of Sydney
COMP5347 Web Application Development
Page 41
References
– Randy Connolly, Ricardo Hoar, Fundamentals of Web Development, Global Edition, Pearson
– Wikipedia PAI documentation: https://www.mediawiki.org/wiki/API:Main_page
– Enterprise Scale Software Architecture (COMP5348) slides
– Node.js https API. https://nodejs.org/api/https.html
The University of Sydney
COMP5347 Web Application Development
Page 43
W10 Tutorial: Web Services (Wikipedia API)
W11 Lecture: Web Application Security
The University of Sydney Page 44