COMP 3234B
Computer and Communication Networks
2nd semester 2020-2021
Application Layer (Part II)
Prof. C Wu
Department of Computer Science The University of Hong Kong
Roadmap
Application layer
Key concepts and implementation aspects of network applications (ILO1)
How to build a client-server application using (ILO4)
socket programming over TCP socket programming over UDP
Protocols of example applications: (ILO2) Web, Email, DNS
application
application
transport
network
link
physical
Application-layer protocol
What it specifies
Types of messages exchanged
e.g., request, response
Syntax of each message type
e.g., what fields in a message, how fields are delineated
Semantics of fields
meaning of information in the fields
Rules for when and how processes send messages and respond to messages received
Application
Application-layer protocol
Defined by
Web
HTTP
RFC
Email
SMTP, IMAP, POP3
File transfer
FTP
Remote login
Telnet
Instant messaging, interactive game, Skype
Proprietary
Application designer
Web
The Web, or World Wide Web (WWW, W3)
an Internet application
an information space where documents and other web resources are identified by URLs, interlinked by hypertext links, and can be accessed via the Internet
A web page contains a base HTML file
referenced objects
html files, JPEG image, audio/video clip, etc.
each object is addressable by a URL
Web browser
Web server
URL
URL (Uniform Resource Locator): a string of characters to identify a resource, specifying the means of locating the resource with both its primary access mechanism and network location
Examples:
http:// www.cs.hku.hk/~cwu/index.html
or IP address
scheme
HTTP: basics
Hypertext Transfer Protocol
Web’s application-layer protocol includes two types of messages:
HTTP request and HTTP response Implemented in client-server mode
client program
browser that requests, receives and “displays” web objects
server program
Web server sends objects in response to requests
PC running Chrome browser
server running
Apache Web server
iPhone running Safari browser
HTTP request
HTTP response
HTTP request
HTTP response
HTTP: client and server interaction
enters www.cs.hku.hk/~cwu/index.html (contains references to 1 jpg file and 1 css file)
HTTP uses TCP as the transport protocol
1a. HTTP client initiates TCP connection to HTTP server
(process) at www.cs.hku.hk on port 80
1b. HTTP server at host www.cs.hku.hk waiting for TCP
connection at port 80 “accepts” connection, notifying client
2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object
3. HTTP server receives request message, forms response
message containing requested base html file, and sends message into its socket
5. HTTP server closes TCP connection.
~c0234b/index.html ~cwu/index.html
4. HTTP client receives response message containing html file, displays html.
time
Parsing html file, finds 1 referenced jpeg object and 1 referenced css object
What next?
HTTP: client and server interaction (cont’d)
HTTP 1.1 (using persistent connections) is the widely used HTTP version
enters www.cs.hku.hk/~cwu/index.html (contains references to 1 jpg file and 1 css file)
1a. HTTP client initiates TCP connection to HTTP server
(process) at www.cs.hku.hk on port 80
1b. HTTP server at host www.cs.hku.hk waiting for TCP
connection at port 80 “accepts” connection, notifying client
2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object
3. HTTP server receives request message, forms response
message containing requested base html file, and sends message into its socket
5. HTTP server closes TCP connection.
~c0234b/index.html ~cwu/index.html
time
4. HTTP client receives response message containing html file, displays html.
Parsing html file, finds 1 referenced jpeg object and 1 referenced css object
Persistent connection (HTTP 1.1): Steps 2-4 repeated for fetching each of the referenced objects, before step 5 is done.
Non-persistent connection (HTTP 1.0): Steps 1-5 repeated for fetching each referenced object
HTTP protocol messages
Two types
request response
both in ASCII text
HTTP request message
An example
General format
request line (method, URL, HTTP version)
GET /~cwu/index.html HTTP/1.1
Host: www.cs.hku.hk
User-Agent: Mozilla/4.0
Accept: text/html
Accept-Language: en-us,en;q=0.5
Keep-Alive: timeout=10, max=100
Connection: keep-alive
header lines
GET: entity body is empty
POST: entity body contains what user sends to the server (e.g., entered in a form)
HTTP methods (HTTP/1.1)
GET
browser requests an object identified in the URL
POST
e.g., used when web page includes form input browser requests a Web page, whose content depends on input to server in entity body
HEAD
asks server to leave requested object out of response body
used for debugging purpose by application developers
PUT
uploads object in entity body to path specified in URL field
DELETE
deletes object specified in the URL field
General format
For a complete list of HTTP methods, refer to : h6p://tools.ie9.org/html/rfc7231#page-24
For a complete list of HTTP request header fields and their meaning, refer to: h6p://tools.ie9.org/html/ rfc7231#page-33
status line (protocol,status code,status phrase)
HTTP response message
An example
General format
header lines
HTTP/1.1 200 OK
Date: Sun, 26 Jan 2020 20:09:20 GMT
Server: Apache/2.4.41 (Unix)
Last-Modified: Tue, 29 Oct 2019 17:00:02 GMT
ETag: “17dc6-a5c-bf716880”
Content-Length: 2652
Keep-Alive: timeout=10, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
data data data data data …
Entity body (e.g., requested HTML file)
HTTP response status code/phrase
Status code and phrase indicate result of the request
200 OK
• request succeeded, requested object in “Entity Body” of this msg 301 Moved Permanently
• requested object moved, new location specified in “Location:” header field of this msg
403 Forbidden
• service denied, e.g., not authorized to access the resource
404 Not Found
• requested document not found on this server
500 Internal Server Error
• some error occurred on server side
For a complete list of status codes, refer to : h6p://tools.ie9.org/html/rfc7231#page-47
For a complete list of HTTP response header fields and their meaning, refer to: h6p://tools.ie9.org/html/ rfc7231#page-64
HTTP summary
HTTP as an example application-layer protocol
typical request/response message exchange
client requests info or service
server responds with data, status code, etc.
message formats
headers: fields giving information about data data: information being communicated
Email (Electronic Mail)
A client-server application Three major components
user agents
“email client”: composing, editing, reading mail messages
e.g., Microsoft Outlook, Mozilla Thunderbird, Apple Mail
mail servers, which maintain
mailbox: contains incoming messages for each
user
mail server
user agent
user agent
SMTP
mail server
SMTP
SMTP
mail server
user agent
user agent
user agent
message queue of outgoing mail messages user
protocols
agent
outgoing message queue
user mailbox
Email protocols
Simple Mail Transfer Protocol (SMTP)
specifies how mail message is transferred from sender’s mail server to recipient’s mail server
Mail access protocols
specify how user agent retrieves messages from mail server
POP3: Post Office Protocol
IMAP: Internet Mail Access Protocol
mail server
outgoing message queue
user mailbox
user agent
user agent
SMTP
mail server
SMTP
SMTP
mail server
user agent
user agent
user agent
HTTP: browser-based Gmail, Hotmail, Yahoo mail, etc. user agent
Simple Mail Transfer Protocol (SMTP): basics I
Use TCP to reliably transfer mail message from sender’s mail server to recipient’s mail server (port 25)
An example scenario
1) Alice uses outlook to compose message “to” bob@someschool.edu
2) Alice’s outlook sends message to her mail server; mail server places message in message queue
3) Client side of SMTP on Alice’s mail server opens TCP connection with Bob’s mail server
4) SMTP client sends Alice’s message over the TCP connection
5) Server side of SMTP on Bob’s mail server receives the message. Message placed in Bob’s mailbox
6) Bob invokes his user agent to read the message at his
convenience
user agent
mail server
mail server
3
1 user agent
2
SMTP
4
6
Alice’s mail server
5
Bob’s mail server
Simple Mail Transfer Protocol (SMTP): basics II
Every mail server runs both the client and server side of SMTP
SMTP “client”
sending party
SMTP “server”
receiving party
user agent
mail server
mail server
3
1 user agent
2
SMTP
4
6
Alice’s mail server
5
Bob’s mail server
Simple Mail Transfer Protocol (SMTP): messages
SMTP
Message
must all be in 7-bit ASCII client commands
HELO, MAIL FROM, RCPT TO, DATA, QUIT
server reply
status code and phrase
alice@crepes.fr
sender’s mail server
receiver’s mail server
bob@hamburger.edu
user agent
user agent
handshaking required by SMTP
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM:
S: 250 alice@crepes.fr… Sender ok
C: RCPT TO:
S: 250 bob@hamburger.edu … Recipient ok
C: DATA
S: 354 Enter mail, end with “.” on a line by itself C: Message-ID: <4F2F3837.8010200@crepes.fr> C:Date: Tue, 19 Jan 2021 10:17:27 +0800
C: From: Alice
C: Thunderbird/3.0.1
C: To: bob@hamburger.edu
C: Subject: Hello from Alice
C: Content-Type: text/plain; charset=ISO-8859-1 C: xxx:xxx
C:
C: email content
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection
220 –
250 – Requested mail action okay, completed
354 – Start mail input
221 –
generated by mail client
an empty line
the end of message
Mail Access Protocols
user agent
SMTP
sender’s mail server
mail access protocol
SMTP
user agent
receiver’s mail server
How user agent retrieves (pulls) messages from mail servers
POP3: Post Office Protocol
IMAP: Internet Mail Access Protocol
HTTP: browser-based Gmail, Hotmail, Yahoo mail, etc.
POP3 (Post Office Protocol)
User agent opens a TCP connection to its mail server (port 110)
Three phases
authorization phase
❒ client commands:
user: declare username pass: password
❒ server responses +OK
-ERR
transaction phase, client:
❒ list: list message numbers
❒ retr: retrieve message by number
❒ dele:delete
❒ quit
update phase
S: +OK POP3 server ready
C: user bob
S: +OK
C: pass hungry
S: +OK user successfully logged on
C: list
S: 1 498
S: 2 912
S: .
C: retr 1
S:
S: .
C: dele 1
C: retr 2
S:
S: .
C: dele 2
C: quit
S: +OK POP3 server signing off
mail server deletes messages marked for deletion
1
2
IMAP (Internet Mail Access Protocol)
More features (more complex)
Manipulation of stored messages on servers Keeps state information across sessions
POP3
§❒ p“rDeovwionulsoaedxaamnpdleduesletseP”O: P3 Bob cannot re-read e-
“download and delete” mode mail if he changes
(Bob cannot re-read e-mail if he
client
changes client)
❒ “Download-and-keep”:
§ POP3 is stateless across sessions
copies of messages on different clients
❒ POP3 is stateless across sessions
IMAP
❒ Keep all messages in § allows user to organize
one place: the server
messages in folders
❒ Allows user to
§ keeps user state across
organize messages in
sessions: names of folders
folders
and mappings between
❒ IMAP keeps user state message IDs and folder
across sessions:
name names of folders and mappings between
message IDs and folder name
Web-based Email
SMTP
user agent
HTTP
sender’s mail receiver’s mail server server
user agent
HTTP
user agent: Web browser Access protocol: HTTP
e.g., Gmail, Hotmail, Yahoo mail
DNS (Domain Name System)
Identifiers for hosts in the Internet
IP address: e.g., 147.8.179.15 hostname: e.g., www.cs.hku.hk
DNS services
hostname to IP address translation
host aliasing: map alias hostname to canonical hostname and IP address
mail server aliasing: map alias hostname to canonical hostname and IP address
load distribution: map one canonical name to a set of IP addresses (e.g., replicated web servers)
DNS uses a large number of name servers, organized in a hierarchical fashion and distributed around the world
DNS servers are often Unix servers running Berkeley Internet Name Domain (BIND) software
over UDP, port 53
DNS protocol: application-layer protocol by which hosts and name servers communicate to resolve name/address translation
Domain Name
An identification string with structure indicated by dots, to define a realm of administrative autonomy of Internet hosts
e.g., com, example.com, cs.example.com
Organized in subordinate levels of the DNS root domain (which does not have a name)
top-level domains (TLDs), including
generic top-level domains (gTLDs), e.g., com, edu, org, gov, net, mil country code top-level domains (ccTLDs), e.g., hk, ca, au
second (third, fourth, etc.) level domains
typically open for reservation by end users
registration of these domain names administered by domain name registrars
Root DNS servers
root$
Root$DNS$servers$
There are 13 root DNS servers in the Internet
The root servers are e. NASA Mt View, CA
c. Cogent, Herndon, VA (5 other sites) d. U Maryland College Park, MD
h. ARL Aberdeen, MD
j. Verisign, Dulles VA (69 other sites )
k. RIPE London (17 other sites)
i. Netnod, Stockholm (37 other sites)
m. WIDE Tokyo (5 other sites)
labeled from server A to
M, maintained by different Palo Alto, CA (and 48 other
f. Internet Software C. sites)
operators
a. Verisign, Los Angeles CA (5 other sites)
The root servers refer b. USC-ISI Marina del Rey, CA DNS queries to the l. ICANN Los Angeles, CA
(41 other sites)
OH (5 other sites) correct top-Level domain g. US DoD Columbus,
DNS servers
TLD DNS servers
www.cs.hku.hk i.cs.hku.hk(
Top$Level$Domain$(TLD)$servers$
root$
hk$ com$ org$ edu$
responsible for generic top-level domains and country-code top- level domains
Verisign maintains the TLD servers for the .com top-level domain Educause maintains the TLD servers for the .edu top-level domain
Hong Kong Internet Registration Corporation Limited (HKIRC) maintains the TLD servers for the .hk top level domain.
Authoritative DNS servers
www.cs.hku.hk i.cs.hku.hk(
Authorita)ve,DNS,servers,
hk$
root$
com$ org$
edu$ …,
govv$.h…k ,
Every organization with publicly accessible hosts (such as web, mail
servers) on the Internet must provide such authoritative DNS servers
providing authoritative hostname to IP mappings for this organization’s servers
hku.$hk
Authoritative DNS servers (cont’d)
w.cs.hku.hk i.cs.hku.hk”
ww
Local”name”servers”
E.g., the CS department in HKU may maintain an authoritative name server for cs.hku.hk
root$
com$ org$ gov$ …”
ccss$.hku.hk engg$.hk…u.h”k
i$ virtue$ gatekeeper$ www$ …”
www.cs.hku.hk gatekeeper.cs.hku.hk
hosts in domain cs.hku.hk
hk$
edu$ …”
hku$
hku.hk
gov.hk
Local DNS servers
i.cs.hku.hk” Local”name”servers”
Each organization provides one
it forwards the organization’s users’ DNS queries to the DNS server hierarchy, and can cache query results
hk$
root$
com$ org$ gov$ …”
edu$ …”
hku$
hku.hk
gov.hk
ccss$.dhnksu..chsk.hku.hk engg$.hk…u.h”k
user 1
i$
virtue$ $ www$ …” user 2
gatekeeper
How does DNS work
When the host connects to its network, it learns IP address(es) of its local DNS server(s)
Root$DNS$server$
edu(TLD(DNS(server(
IP#address#of## www.cs.yale.edu?#
Reques1ng&host&
Local&DNS&server&
Root$DNS$server$takes$note$of$the$“.edu”$suffix$ and$contacts$a$TLD(server(responsible(for(“.edu”.(
Local&DNS&server&asks&a&Root&DNS&server&
The#reques0ng#host#first#sends#a#DNS# query#message#to#its#local#DNS#server.# E.g.,#“What#is#the#IP#address#of# www.cs.yale.edu?”#
How does DNS work
Root)DNS)server)
edu)TLD)DNS)server)
Local)DNS)server)
Authorita&ve)DNS)server)
dns.yale.edu#
The#TLD#server#takes#note#of# the#“yale.edu”#suffix#and# contacts#the)authorita&ve) DNS)server)for)the)Yale) University,#namely# “dns.yale.edu”.#
IP#address#of## www.cs.yale.edu?#
Reques&ng)host)
How does DNS work
Root)DNS)server)
edu)TLD)DNS)server)
Authorita&ve)DNS)server)
dns.yale.edu#
The#Authorita6ve#DNS#server#in#
yale#knows#the#IP#address#of#
www.cs.yale.edu,#and#replies#
the#edu#TLD#DNS#server#with#the#
IP#address#(e.g.,#128.36.229.30)#
# The$edu$TLD$DNS$server$further$
sends$the$IP$address$to$the$Root$
DNS$server$
$ The$Root$DNS$server$sends$the$
IP$address$to$the$Local$DNS$ server$
$
128.36.229.30 128.36.229.30
IP#address#of## www.cs.yale.edu?#
Reques&ng)host)
Local)DNS)server)
128.36.229.30
How does DNS work
Root)DNS)server)
edu)TLD)DNS)server) Authorita&ve)DNS)server)
dns.yale.edu#
Finally,#the#local#DNS#server#sends#the# answer#of#the#DNS#query#(i.e.,#The#IP# address#of#www.cs.yale.edu#is# 128.36.229.30#)#to#the#reques@ng#host.##
Recursive query:
enabled by default on DNS servers
IP#address#of## www.cs.yale.edu?#
Reques&ng)host)
Local)DNS)server)
128.36.229.30)
Non-recursive query
Root)DNS)server) edu)TLD)DNS)server)
Authorita&ve)DNS)server)
dns.yale.edu#
IP of edu TLD DNS server
IP of dns.yale.edu
IP#address#of## www.cs.yale.edu?#
Reques&ng)host)
Local)DNS)server)
128.36.229.30)
Non-recursive query (iterative query):
local DNS server contacts each responsible DNS server in the hierarchy
128.36.229.30
DNS caching and records
Once any name server knows a (hostname, IP address) mapping, it caches mapping
A local name server can also cache IP addresses of the TLD servers (e.g., “.com”, “.net” TLD servers), allowing the local name server to bypass the root DNS servers in a query chain (this often happens)
DNS resource record (RR)
cached mapping removed after ttl time
RR format: (name, value, type, ttl)
Type=A
name is hostname
value is IP address
Type=NS
name is domain (e.g., foo.com)
Type=CNAME
name is alias name (e.g. www.ibm.com) for some
value is hostname of authoritative name server for this domain
canonical name (e.g. servereast.backup2.ibm.com)
value is canonical name Type=MX
Name is alias name of mailserver
value is canonical name
identification
# questions
flags
# answer RRs
Name, type fields for a query
RRs in response to query
records for authoritative servers
# authority RRs
questions (variable # of questions)
answers (variable # of RRs)
authority (variable # of RRs)
additional info (variable # of RRs)
# additional RRs
additional “helpful” info that may be used
DNS messages
DNS query and reply messages have the same format
2 bytes
2 bytes
16 bit # for query, reply to query uses same #
indicating the following:
query or reply recursion available etc.
Required reading:
Ch. 2.2, 2.3, 2.4, Computer Networking: A Top Down Approach (7th Edition)
Acknowledgement:
Some materials are extracted from the slides created by Prof. Jim F. Kurose and Prof. Keith W. Ross for the textbook.