CE303 Lecture 4
PART I: JSON and GSON
JSON Data-Interchange Format
Java/C# serialisation only works between Java/C# programs
How to exchange data more generally?
JSON is a text format that can represent objects
Human-readable and editable (within limits)
Derived from “JavaScript Object Notation”
Supported in many different programming languages with libraries for (de-)serialisation, parsers, etc.
Popular for web applications and for web-services
3
JSON Format
Primitive values: strings, numbers, booleans, or null:
“abc”, 123, true, false, null
Two data structures: objects and arrays
An object (“record”, “dictionary”, “map”, …) is an unordered collection of name/value pairs
{“field1”: value1, “field2”: value2, …}
Do not forget the quotes around the field names
An array is an ordered collection of values:
[value1, value2, …]
JSON objects are just data
They have no methods
No inbuilt “unique identifier” such as the JVM address
They do not belong to a “class”
4
Support of JSON in Java/C#
Limited support by standard Java/C# libraries
In C#, one of the best libraries to handle JSON is Json.NET
We will not study it in this module
In Java, one of the best libraries is Gson
Gson (Java)
Open source library to convert Java objects to JSON and back
Created by Google developers, distributed as a jar-file
Does not require annotation of classes
Supports generics and nested classes
Ignores static and transient fields by default
6
Sample Classes: Book and Author
Gson: Basic Usage
Gson gson = new Gson();
Author author = new Author(“01”, “Joyce, James”);
// serialisation
String authorJson = gson.toJson(author);
System.out.println(“Json: ” + authorJson);
// round trip: de-serialisation of serialised object
Author authorCopy = gson.fromJson(authorJson,
Author.class);
System.out.println(“Round trip: ” + authorCopy);
Program output
Json: {“id”:”01″,”name”:”Joyce, James”}
Round trip: Author[01, Joyce, James]
Example: A List of Books serialised to JSON
[ { “isbn”: “0-13-110362-8”,
“price”: 42.0,
“title”: “The C Programming Language”,
“authors”: [
{ “id”: “KERN”, “name”: “Kernighan, Brian” },
{ “id”: “RITC”, “name”: “Ritchie, Dennis” } ]
},
{ “isbn”: “0-201-83595-9”,
“price”: 29.95,
“title”: “The Mythical Man-Month”,
“authors”: [
{ “id”: “BROO”, “name”: “Brooks, Frederick P.” } ]
}, …
]
public class BooksJson {
public static final String BOOKS_JSON_FILE =
“data/books.json”;
public static final String BOOKS_JSON_COPY_FILE =
“data/books_copy.json”;
public static final Gson GSON = new GsonBuilder()
.setPrettyPrinting().disableHtmlEscaping().create();
public static void writeBooks(List
String fileName) throws IOException {
Path path = Paths.get(fileName);
BufferedWriter writer = Files.newBufferedWriter(path);
GSON.toJson(books, writer);
writer.close();
}
…
public static List
throws IOException {
Path path = Paths.get(fileName);
BufferedReader reader = Files.newBufferedReader(path);
Type t = new TypeToken
List
reader.close();
return result;
}
public static void main(String[] args) throws IOException {
List
for (Book book : books)
System.out.println(book);
BooksJson.writeBooks(books, BOOKS_JSON_COPY_FILE);
}
}
Comments about Book List Example
GsonBuilder for creating a configurated Gson instance
Gson can read / write from files via readers / writers
Don’t forget to close these when you are done with them!
setPrettyPrinting() configures Gson to output Json that fits in a page for pretty printing. This option only affects Json serialisation
disableHtmlEscapting. By default, Gson escapes HTML characters such as <>. Use this option to configure Gson to pass-through HTML characters as is
Use of Type and TypeToken for deserialisation of generics
This is necessary because the Java compiler erases information about type parameters in generics – we will discuss this further later on in this module
Prettyprint is the application of various stylistic formatting conventions to text files, such as source code. As a result, the serialization of the object is saved in an easy-to-read “code” format.
12
Concluding Remarks about Gson
Gson serialises whole “object trees” similar to Java’s serialisation, but it does not include JVM addresses nor types
It parses numbers by default as Double values; this can occasionally cause problems when you try to de-serialise generic types instantiated with integer type parameters
Both serialisation and de-serialisation can be customised
In particular you can exclude certain fields
You can even code custom (de)serialisation for a class
Please see the online user guide for more details
There are several other JSON libraries for Java
Notably Jackson and javax.json
END OF PART I
PART II: HTTP Refresher
HTTP Refresher
Hypertext Transfer Protocol
Request-response protocol
Client sends request to server
Server sends response back to client
Supports authentication, caching and use of proxy servers
Makes use of URIs to identify resources
Normally run on top of TCP
HTTPS is HTTP over encrypted connections (SSL or TLS)
Versions 1.1 and 2.0 improve performance (reduce latency, etc.)
HTTP Requests
Messages sent from client to web server via TCP
Request line specifies method:
GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH, …
And resource: URI
Request header fields (optional):
For example an Accept field that specifies different “acceptable” content types for the response
Message body (optional)
For example to submit form content in POST requests
HTTP Request Example
POST /examples/rest/bank/transfer/cust1/ HTTP/1.1
Host: localhost:8080
Accept: application/json
Content-Type: application/json
{ “fromAccount”:105, “toAccount”:100, “amount”:50.0}
18
GET Requests
GET request means: “give me this resource”
GET requests should be preferred choice for queries that do not modify data
GET requests often originate from
typing a URL into a web browser
clicking on a hyper link in a web page
redirecting a web page
submitted forms with method set to GET (e.g. Google)
The URL can encode parameters (“query string”) which are decomposed on the server (“URL rewriting”):
http://www.mysite.org/myservice?fName=Fred&sName=Flintstone
POST Requests
POST requests send data
POST requests should be used preferably when you want to create or modify data
They often originate from HTML elements where the method is set to POST:
Form values are not visible in the URL
Can send more information than on address line
HTTP Response
Message sent from server to client via the TCP connection on which request was received
Status line with status code and reason message
Response header fields
such as Content-Type: text/html
Empty line
Message body (optional)
Contains the “resource” which has been requested, provided request could be satisfied
HTTP Response Example
Status
Status 200 OK
Headers
Content-Length: 17
Content-Type: application/json
Date: Sat, 05 Aug 2017 16:09:33 GMT
Body
4795.847610355551
Some HTTP Status Codes
2xx Success
200 OK
204 No Content
3xx Redirection
301 Moved Permanently
4xx Client Error
400 Bad Request
401 Unauthorized
404 Not Found
406 Not acceptable
418 I’m a teapot
5xx Server Error
500 Internal Server Error
Media Types and Content Negotiation
HTTP clients can specify the media type they are sending and what media type they expect in the response of the server, see Accept and Content-Type headers in the request example above
Similarly, HTTP servers can specify the media types they can provide for a certain resource and what media types they expect in the request sent by the client
HTTP supports “content negotiation” – selecting the “best” representation for a given response
Some basic media types:
application/json, application/x-www-form-urlencoded
text/plain, text/html, text/css, image/png, …
Please see literature for examples
END OF PART II
PART III: Web Services (Introduction)
Suppose we want to book a trip…
Not sure what airline to use, so we try an online travel agency
lastminute.com
lastminute.com
Cathay Pacific
China Airways
Qatar Airways
Aeroflot
lastminute.com
Cathay Pacific
China Airways
Qatar Airways
Aeroflot
lastminute.com
Cathay Pacific
China Airways
Qatar Airways
Aeroflot
Note that the server is providing information about its web services, which describes all the attributes and functionalities of the Web Services.
31
How did the communication between the agency and the air companies take place?
Database access?
Are all of these websites written in the same language?
Note that the server is providing information about its web services, which describes all the attributes and functionalities of the Web Services.
32
Web Service Introduction
A web service provides some “computational service” for the purpose of machine-to-machine interaction via HTTP(S)
not intended for direct human consumption
Data typically transferred as JSON, XML or text
platform and programming language neutral
Two contrasting approaches
Resource-oriented (“RESTful”)
RPC-style (“remote-procedure call”, SOAP)
We will use a resource-oriented approach in this module
Web Services vs Web Applications
Web Services can be used to transfer data between Web Applications.
Web Services can be accessed from any languages or platform.
A Web Application is meant for humans to read, while a Web Service is meant for computers to read.
Web Application is a complete Application with a Graphical User Interface (GUI), however, web services do not necessarily have a user interface since it is used as a component in an application.
Web Application can be accessed through browsers.
Source: https://www.carmatec.com/blog/web-services-vs-web-applications/
34
END OF PART III
PART IV: RESTful Services
PART A: INTRODUCTION
REST Architecture Style
Approach to client-server applications based around resources (= data) and standard web infrastructure
Resources are identified by URIs
Use standard HTTP verbs to operate on resources, notably
POST, GET, PUT, DELETE
Stateless protocols
No session information is retained by the receiver, e.g. the server
Use of standard content types: JSON, XML, text, …
Use of web features: HTTP status codes, caching, …
Aims: simplicity, efficiency/scalability, re-use
REST = “Representational State Transfer”
REST Principles: Use of HTTP Verbs
Be disciplined in your choice of HTTP operation
GET should not change resources on the server
For example,
GET http://myserver/myresource/delete
should NOT delete the resource
POST: create or update
PUT: create or replace at specified URL
PUT and DELETE operations should be idempotent
Repetition of operation should have no further effect
Please see online discussions for “POST” vs “PUT”
Implementation of RESTful services
In Java, one can use JAX-RS framework to build a RESTful service
In C#, RESTful services can be build using ASP.NET
We will only study building a Restful service in Java
Web Services using JAX-RS / Jersey
JAX-RS is a Java API specification for “RESTful” web services
It uses annotations to define resources and the operations that can be performed on them with injection of request parameters and path-segments
The resource classes can be POJOs (“plain old java objects”)
They do not need to implement any particular interface or extend a particular class
Jersey is an implementation of JAX-RS
We will deploy it on Tomcat launched from a Java program
“embedded Tomcat”
No need for JavaEE or for special web dev support in IDE
@Path(“/hello”)
public class HelloServer {
@GET @Path(“/{name}”) @Produces(MediaType.TEXT_PLAIN)
public String helloPlain(@PathParam(“name”) String name) {
return “Hello, ” + name + “!”;
}
@GET @Produces(MediaType.APPLICATION_JSON)
public String helloJson(@QueryParam(“name”) String name) {
return “{\”hello\”:\”” + name + “\”}”;
}
}
HelloServer Example
Access in Web Browser
42
Example of a web service provider (server)
We can see the available info this web service can provide: Customer, Invoice, Item, Product. If I put one of these in a GET request, then I’d get the relevant info (see next slide).
43
JAX-RS Fundamentals
Resource methods are annotated with a request method designator @GET, @PUT, @POST, or @DELETE
These are the methods which are invoked in response to HTTP requests
@Path annotations determine the mapping from URLs to resource methods
Class level @Path value is prefixed to URL if present
Resource methods can be “injected” with HTTP request details such as request parameters, media type, segments of the request URL path, etc.
Some Basic JAX-RS Annotations
@Path(“/{id}”) path parameter declaration
@PathParam(“…”) path parameter binding
@QueryParam(“…”) query parameter binding
@FormParam(“…”) form parameter binding
Content negotiation
@Consumes specifies one or more media types a resource can accept (“consume”) from the client
@Produces specifies one or more media types a resource can “produce” and send back to the client
47
Deployment and Testing
Jersey web services need to be deployed on a web server
Both the web server and Jersey can be configured extensively
Once the server has started, the web services can be accessed via HTTP
PART IV: RESTful Services
PART B:
BANK WEBSERVICE EXAMPLE
SERVER-SIDE
public class TomcatJerseyServer {
public static final int TOMCAT_PORT = 8080;
public TomcatJerseyServer() {}
public void start() throws ServletException,
LifecycleException {
// JAX-RS (Jersey) configuration
ResourceConfig config = new ResourceConfig();
// Packages where Jersey looks for resource classes
config.packages(“bank.rest”, “books.rest”, “hello”);
// Configure Jersey to use Gson
config.register(util.GsonMessageBodyHandler.class);
// Enable logging for debugging purposes
config.property(
LoggingFeature.LOGGING_FEATURE_LOGGER_LEVEL, “INFO”);
…
This and the next page are the server configuration pages. The main thing you need to focus on here is the config.packages, where you indicate the folders where your files are in.
51
// Tomcat configuration
Tomcat tomcat = new Tomcat();
tomcat.setPort(TOMCAT_PORT);
// Add web application
Context context = tomcat.addWebapp(“/examples”,
new File(“./WebContent”).getAbsolutePath());
// Add Jersey servlet
Tomcat.addServlet(context, “jersey”,
new ServletContainer(config));
// Map certain URLs to Jersey
context.addServletMappingDecoded(“/rest/*”, “jersey”);
// Start server
tomcat.start();
tomcat.getServer().await();
}
public static void main(String[] args) throws Exception {
new TomcatJerseyServer().start(); }
}
A servlet is a small Java program that runs within a Web server. Servlets receive and respond to requests from Web clients, usually across HTTP, the HyperText Transfer Protocol.
52
Example: Bank Web Service
Method URL Action
GET /customers Retrieve all customer ids
GET /customer/{customerId} Retrieve accounts of a
customer
GET /customer/{customerId}/{account}
Get balance of account
POST /transfer{customerId} Transfer (details passed
as JSON in request body)
@Path(“/bank”)//Path of my class
@Produces(MediaType.APPLICATION_JSON)
public class BankServer {
private final static Bank BANK = new Bank();
public BankServer() {}
@GET @Path(“/customers”)//Path of my method
public List
return BANK.getCustomers();
}
@GET @Path(“/customer/{customerId}”)
public int[] getAccounts(
@PathParam(“customerId”) String customerId) {
return BANK.getAccounts(customerId);
}
…
To make the methods accessible through RESTful Web Services, we add “GET” (or POST, PUT, DELETE). We also add @Path to define which path needs to be called to access this method.
Also note that this is a NEW class, which didn’t exist before! Before (in the previous lecture), we only had Bank.java and ServerProgram.java.
54
@GET
@Path(“/customer/{customerId}/{account}”)
public Double getBalance(
@PathParam(“customerId”) String customerId,
@PathParam(“account”) int account) {
return BANK.getBalance(customerId, account);
}
@POST
@Consumes(MediaType.APPLICATION_JSON)
@Path(“/transfer/{customerId}”)
public Double transfer(
@PathParam(“customerId”) String customerId,
TransferRequest req) {
return BANK.transfer(customerId, req.fromAccount,
req.toAccount, req.amount);
}
}
TransferRequest req doesn’t need a @PathParam, because in the URL we only use customerId as a parameter.
We use @Consumes because we want to obtain an object (TransferRequest) from the Client, so we will be obtaining that as a JSON/GSON object. See slide “Using Gson in Jersey – Server Side” for more information.
55
public class TransferRequest {
public int fromAccount, toAccount;
public double amount;
public TransferRequest() {}
public TransferRequest(int fromAccount,
int toAccount, double amount) {
this.fromAccount = fromAccount;
this.toAccount = toAccount;
this.amount = amount;
}
@Override
public String toString() {
return “TransferRequest: ” + fromAccount + ” -> ” +
toAccount + “: ” + amount;
}
}
Web Service URL Example: Bank
GET http://localhost:8080
/examples/rest/bank/customer/cust1/105
URL components:
Server: http://localhost:8080
Web application: /examples
Jersey: /rest
Bank resource: /bank
Customer identification: /customer/cust1
Account identification: /105
Using Gson in Jersey – Server Side
The bank server has been configured to automatically convert Java objects to/from JSON using Gson; this relies on a utility class GsonMessageBodyHandler
For example, suppose the server gets this POST request:
POST http://localhost:8080/examples/rest/bank/transfer/cust2
{“fromAccount”:100,”toAccount”:101,”amount”:791.04}
This request will be handled in BankServer.transfer with Gson converting the message to a TransferRequest object
Similarly, the server uses Gson to serialise objects to JSON before sending the response to the client
If there are problems with the implicit invocation of Gson, you can invoke it explicitly
END OF SERVER-SIDE EXAMPLE
PART IV: RESTful Services
PART B:
BANK WEBSERVICE EXAMPLE
CLIENT-SIDE
Clients for RESTful Web Services
Clients need to generate appropriate HTTP requests and parse responses
Web browsers can do this to some extend
but they are not built for this purpose
for example you cannot directly issue a POST request by entering it in the address bar of Chrome
curl is a command line tool for generating HTTP requests
curl -H “Accept: text/plain” http://localhost:8080/examples/rest/hello/Sam
Invoking RESTful Services from Java
JAX-RS Jersey also has a Client API
Usage
create a web target pointing at a Web resource
build the request
submit request and process the response
Web target creation includes building a client which may have some configuration such as setting of message handlers and logging levels
Request building may involve construction of entities that are passed in the body, especially for POST and PUT
“Fluent API” with chaining of method calls
See sample code below
public class BankClient {
public final WebTarget target;
public BankClient(String bankUrl) {
ClientConfig config = new
ClientConfig(GsonMessageBodyHandler.class);
target = ClientBuilder.newClient(config).target(bankUrl);
}
public List
return target.path(“customers”).request().get(
new GenericType
});
}
public List
return target.path(“customer”).path(customerId).request()
.get(new GenericType
}
…
public Double getBalance(String customerId,
int accountNumber) {
return target.path(“customer”).path(customerId)
.path(Integer.toString(accountNumber)).request()
.get(Double.class);
}
public String transfer(int fromAccount, int toAccount,
double amount) {
TransferRequest req = new TransferRequest(fromAccount,
toAccount, amount);
return
target.path(“transfer”).path(customer).request()
.post(Entity.entity(req, MediaType.APPLICATION_JSON))
.readEntity(String.class);
}
}
Entity.entity encapsulates message entity including the associated variant information. Essentially we are passing to the server the object req, which contains the transfer request information fromAccount, toAccount, and amount. We are passing this information as a JSON object.
readEntity specifies the class of the object, which is String.
65
Putting it all together with a web interface
66
Closing Remarks
RESTful web services have become popular in recent years, especially for “public facing” web services
They are supported in many different programming languages and web application frameworks
JAX-RS / Jersey makes it reasonably straightforward to code web services in Java
Supports programming of both servers and clients
Please see the literature for more details of JAX-RS and Jersey such as the asynchronous client API and server-sent events
/docProps/thumbnail.jpeg