CS计算机代考程序代写 python dns database flex cache FTP Week 3 internet-applications

Week 3 internet-applications

Advanced Network Technologies
Applications

School of Computer Science
Dr. Wei Bao | Lecturer

Web caches (proxy server)

› user sets browser: Web
accesses via cache

› browser sends all HTTP
requests to cache

› if object in cache:
– then cache returns object
– else cache requests object

from origin server, then
returns object to client

goal: satisfy client request without involving origin server

client

proxy
server

client

HTT
P re

que
st

HT
TP

res
pon

HTTP request HTT
P re

que
st

origin
server

HTTP response HTT
P re

spo
nse

More about Web caching

›Q: Does the cache act as a client or a server?

More about Web caching

›R: cache acts as both
client and server
– server for original requesting

client

– client to origin server

› typically cache is
installed by ISP
(university, company,
residential ISP)

why Web caching?

› reduce response time for
client request

› reduce traffic on an
institution’s access link

origin
servers

public
Internet

institutional
network

1 Gbps LAN

1.54 Mbps
access link

assumptions:
v avg object size: 100K bits
v avg request rate from browsers to origin

servers:15/sec (1.5 Mbps service )
v RTT from institutional router to any

origin server: 2 sec
v access link rate: 1.54 Mbps

consequences:
v LAN utilization: 0.15%
v LANU = avg req rate * size / link

bandwidth
v access link utilization = 99%
v ALU = avg req rate * size / link

bankwidth
v total delay = 2 sec + seconds + usecs

Q: what happens with fatter access link?

problem!

Caching example

assumptions:
v avg object size: 100K bits
v avg request rate from browsers to origin

servers:15/sec
v RTT from institutional router to any

origin server: 2 sec
v access link rate: 1.54 Mbps

consequences:
v LAN utilization: 0.15%
v access link utilization = 99%
v total delay = 2 sec + seconds + usecs

origin
servers

1.54 Mbps
access link

154 Mbps

msecs

Cost: increased access link speed (not cheap!)

0.99%

public
Internet

institutional
network

1 Gbps LAN

Caching example: fatter access link

institutional
network

1 Gbps LAN

origin
servers

1.54 Mbps
access link

local web
cache

assumptions:
v avg object size: 100K bits
v avg request rate from browsers to origin

servers:15/sec
v RTT from institutional router to any

origin server: 2 sec
v access link rate: 1.54 Mbps

consequences:
v LAN utilization: 0.15%
v access link utilization = 0%
v total delay = usecs

Cost: web cache (cheap!)

public
Internet

Caching example: install local cache

Calculating access link utilization, delay with
cache:

› suppose cache hit rate is 0.4
– 40% requests satisfied at cache,
– 60% requests satisfied at origin

› access link utilization:
– 60% of requests use access link
› average total delay

= 0.6 * (delay from origin servers) +0.4 * (delay
when satisfied at cache)
Link utilization is around 60%, queueing delay is
small enough
= 0.6 (~2.x second) + 0.4 (~usecs)

less than with 154 Mbps link (and cheaper
too!)

origin
servers

1.54 Mbps
access link

public
Internet

institutional
network

1 Gbps LAN

local web
cache

Caching example: install local cache

Conditional GET

› Goal: don’t send object if client
has up-to-date cached version
– no object transmission delay

– lower link utilization

› client: specify date of cached
copy in HTTP request
If-modified-since:

› server: response contains no
object if cached copy is up-to-
date:
HTTP/1.0 304 Not
Modified

HTTP request msg
If-modified-since:

HTTP response
HTTP/1.0

304 Not Modified

object
not

modified

HTTP request msg
If-modified-since:

HTTP response
HTTP/1.0 200 OK

object
modified

after

client server

HTTP/2

Key goal: decreased delay in multi-object HTTP requests

HTTP1.1: introduced multiple, pipelined GETs over single
TCP connection
§ server responds in-order (FCFS: first-come-first-served

scheduling) to GET requests
§ with FCFS, small object may have to wait for transmission

(head-of-line (HOL) blocking) behind large object(s)
§ loss recovery (retransmitting lost TCP segments) stalls object

transmission

HTTP/2

Key goal: decreased delay in multi-object HTTP requests

HTTP/2: [RFC 7540, 2015] increased flexibility at server in
sending objects to client:
§ methods, status codes, most header fields unchanged from

HTTP 1.1
§ transmission order of requested objects based on client-

specified object priority (not necessarily FCFS)
§ push unrequested objects to client
§ divide objects into frames, schedule frames to mitigate Head-

of-line (HOL) blocking

HTTP/2: mitigating HOL blocking

HTTP 1.1: client requests 1 large object (e.g., video file, and 3
smaller objects)

client

server

GET
O1

GET
O2

GET
O3

GET
O4

O
1 O

2 O
3 O

object data requested

O2
O3
O4

objects delivered in order requested: O2, O3, O4 wait behind
O1

HTTP/2: mitigating HOL blocking

HTTP/2: objects divided into frames, frame transmission interleaved

client

server

GET
O1

GET
O2

GET
O3

GET
O4

O
2 O
4

object data requested

O2
O3
O4

O2, O3, O4 delivered quickly, O1 slightly
delayed

O
3

O
1

HTTP/2 Streams and frames

Client Server

HTTP/2 Streams and frames

Frames:
• Basic HTTP/2 data unit, replacing HTTP/1.1 header

and body format.
• HTTP/2 frames have a binary encoding (more

efficient).
• Header frames, Data frames
Streams
• Bidirectional channel where frames are transmitted
• Replacing HTTP/1.1 Request-Response mode
A single TCP connection to carry multiple streams

HTTP/2 push

The HTTP/2 Server Push mechanism allows the server
to send resources proactively without waiting for a
request, when it believes the client will need them.

https://blog.golang.org/h2push

Outline

› Web and HTTP (Done)

› FTP

› Email

› DNS

› P2P

FTP

file transfer
FTP

server
FTP
user

interface

FTP
client

local file
system

remote file
system

user
at host

v transfer file to/from remote host
v client/server model

§ client: side that initiates transfer (either to/from remote)
§ server: remote host

v ftp: RFC 959
v ftp server: port 21, 20

FTP: the file transfer protocol

FTP: separate control, data connections

› FTP client contacts FTP server
at port 21, using TCP

› client authorized over control
connection

› client browses remote directory,
sends commands over control
connection

› when server receives file
transfer command, server opens
2nd TCP data connection (for
file) to client

› after transferring one file, server
closes data connection

FTP
client

FTP
server

TCP control connection,
server port 21

TCP data connection,
server port 20

› server opens another TCP
data connection to transfer
another file

› control connection: “out of
band”

› FTP server maintains “state”:
current directory, earlier
authentication

FTP commands, responses

sample commands:
› sent as ASCII text over control

channel
› USER username
› PASS password
› LIST return list of file in

current directory

› RETR filename retrieves
(gets) file

› STOR filename stores
(puts) file onto remote host

sample return codes
› status code and phrase (as in

HTTP)
› 331 Username OK,
password required

› 125 data connection
already open;
transfer starting

› 425 Can’t open data
connection

› 452 Error writing
file

SMTP: Simple Mail Transfer Protocol

IMAP: Internet Message Access Protocol
POP3: Post Office Protocol 3

Three major components:
› user agents (clients)
› mail servers
› simple mail transfer protocol:

SMTP

User Agent
› a.k.a. “mail reader”
› composing, editing, reading mail

messages

› e.g., Outlook, Thunderbird,
iPhone mail client

user mailbox

outgoing
message queue

mail
server

SMTP

user
agent

Electronic mail

mail servers:
› mailbox contains incoming

messages for user

› message queue of outgoing (to
be sent) mail messages

› SMTP protocol to send email
messages between mail servers

– client: sending mail to server
– “server”: receiving mail from

server

mail
server

SMTP

user
agent

Electronic mail: mail servers

› uses TCP to reliably transfer email message from client to server, port
25

› direct transfer: sending server to receiving server
› three phases of transfer
– handshaking (greeting)
– transfer of messages
– closure

› command/response interaction (like HTTP, FTP)
– commands:ASCII text
– response: status code and phrase

› messages must be in 7-bit ASCII
› Q: is SMTP stateful or stateless?

– Stateful

Electronic Mail: SMTP [RFC 2821]

user
agent

1) Alice uses UA to compose
message “to”

2) Alice’s UA sends message to
her mail server; message
placed in message queue

3) client side of SMTP opens
TCP connection with Bob’s
mail server

4) SMTP client sends Alice’s
message over the TCP
connection

5) Bob’s mail server places the
message in Bob’s mailbox

6) Bob invokes his user agent to
read message

mail
server

2 3 4
5

Alice’s mail server Bob’s mail server

user
agent

Scenario: Alice sends message to Bob

S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: < >
S: 250 … Sender ok
C: RCPT TO: < >
S: 250 … Recipient ok
C: DATA
S: 354 Enter mail, end with “.” on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection

Sample SMTP interaction

› SMTP uses persistent
connections

› SMTP requires message
(header & body) to be in 7-
bit ASCII

› SMTP server uses
CRLF.CRLF to determine
end of message
– Carriage return

– Line feed

comparison with HTTP:

› HTTP: pull
› SMTP: push

› both have ASCII
command/response
interaction, status codes

› HTTP: each object
encapsulated in its own
response msg

› SMTP: multiple objects sent in
one msg

SMTP: final words

SMTP: protocol for exchanging
email msgs

RFC 822: standard for text
message format:

› header lines, e.g.,
– To:
– From:
– Subject:

different from SMTP MAIL FROM,
RCPT TO: commands!

› Body: the “message”
– ASCII characters only

header

body

blank
line

Mail message format

› SMTP: delivery/storage to receiver’s server
› mail access protocol: retrieval from server
– POP: Post Office Protocol [RFC 1939]: authorization, download
– IMAP: Internet Mail Access Protocol [RFC 1730]: more features, including

manipulation of stored msgs on server
– HTTP: Using a browser to access a webmail https://webmail.sydney.edu.au

sender’s mail
server

SMTP SMTP
mail access

protocol

receiver’s mail
server

(e.g., POP,
IMAP)

user
agent

Mail access protocols

authorization phase
› client commands:
– user: declare username
– pass: password
› server responses
– +OK
– -ERR

transaction phase, client:
› list: list message numbers
› retr: retrieve message by number
› dele: delete
› quit

C: list
S: 1 498
S: 2 912
S: .
C: retr 1
S:
S: .
C: dele 1
C: retr 2
S:
S: .
C: dele 2
C: quit
S: +OK POP3 server signing off

S: +OK POP3 server ready
C: user bob
S: +OK
C: pass hungry
S: +OK user successfully logged on

POP3 Protocol

more about POP3
› previous example uses

POP3 “download and
delete” mode
– Bob cannot re-read e-mail

if he changes client

› POP3 “download-and-keep”:
copies of messages on
different clients

› POP3 is stateless across
sessions

IMAP
› keeps all messages in one

place: at server
› allows user to organize

messages in folders

› keeps user state across
sessions:
– names of folders and

mappings between
message IDs and folder
name

POP3 (more) and IMAP

DNS

Internet hosts, routers:
– IP address (32 bit) – used

for addressing datagrams

– “name”, e.g.,
www.yahoo.com – used by
humans

people: many identifiers:

– name, passport #
Q: how to map between IP

address and name, and vice
versa ?

Domain Name System:
› distributed database implemented

in hierarchy of many name
servers

› application-layer protocol: hosts,
name servers communicate to
resolve names (address/name
translation)

DNS: domain name system

DNS: services, structure

why not centralize DNS?
› single point of failure
› distant centralized database
› scalability

DNS services
› hostname to IP address

translation

› host aliasing
– canonical, alias names

› mail server aliasing
› load distribution

– replicated Web servers:
many IP addresses
correspond to one name

Root DNS Servers

com DNS servers org DNS servers edu DNS servers

poly.edu
DNS servers

umass.edu
DNS servers

yahoo.com
DNS servers

amazon.com
DNS servers

pbs.org
DNS servers

client wants IP for www.amazon.com; 1st approx:
› client queries root server to find com DNS server
› client queries .com DNS server to get amazon.com DNS server
› client queries amazon.com DNS server to get IP address for

www.amazon.com

… …

DNS: a distributed, hierarchical database

DNS: root name servers

13 root name
“servers” worldwide

a. Verisign, Los Angeles CA
(5 other sites)

b. USC-ISI Marina del Rey, CA
l. ICANN Los Angeles, CA

(41 other sites)

e. NASA Mt View, CA
f. Internet Software C.
Palo Alto, CA (and 48 other
sites)

i. Netnod, Stockholm (37 other sites)

k. RIPE London (17 other sites)

m. WIDE Tokyo
(5 other sites)

c. Cogent, Herndon, VA (5 other sites)
d. U Maryland College Park, MD
h. ARL Aberdeen, MD
j. Verisign, Dulles VA (69 other sites )

g. US DoD Columbus,
OH (5 other sites)

top-level domain (TLD) servers:

– responsible for com, org, net, edu, aero, jobs, museums, and all top-level country
domains, e.g.: uk, fr, ca, jp

– Network Solutions maintains servers for .com TLD

– Educause for .edu TLD

authoritative DNS servers:

– organization’s own DNS server(s), providing authoritative hostname to IP mappings
for organization’s named hosts

– can be maintained by organization or service provider

TLD, authoritative servers

› does not strictly belong to hierarchy
› each ISP (residential ISP, company, university) has one

– also called “default name server”

› when host makes DNS query, query is sent to its local DNS server
– has local cache of recent name-to-address translation pairs (but may be out of date!)

– acts as proxy, forwards query into hierarchy

Local DNS name server

requesting host
cis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS server
dns.poly.edu

2
3

authoritative DNS server
dns.cs.umass.edu

7
8

TLD DNS server
› host at cis.poly.edu wants

IP address for
gaia.cs.umass.edu

iterated query:
v contacted server

replies with name of
server to contact

v “I don’t know this
name, but ask this
server”

DNS name resolution example

.edu DNS server

3recursive query:
v puts burden of name

resolution on
contacted name
server

v heavy load at upper
levels of hierarchy?

requesting host
cis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS server
dns.poly.edu

2
7

authoritative DNS server
dns.cs.umass.edu

DNS name resolution example (cont’d)

TLD DNS server
.edu DNS server

›once (any) name server learns mapping, it caches
mapping
– cache entries timeout (disappear) after some time (TTL)

› cached entries may be out-of-date (best effort name-
to-address translation!)
– if name host changes IP address, may not be known

Internet-wide until all TTLs expire

› update/notify mechanisms proposed IETF standard
– RFC 2136

DNS caching, updating records

DNS: distributed db storing resource records (RR)

type=NS
– name is domain (e.g., foo.com)
– value is hostname of

authoritative name server for
this domain

RR format: (name, value, type, ttl)

type=A
§ name is hostname
§ value is IP address

type=CNAME
§ name is alias name for some

“canonical” (the real) name
§ www.ibm.com is really
servereast.backup2.ibm.com

§ value is canonical name

type=MX
§ value is name of mailserver

associated with name

DNS records

› example: new startup “Network Utopia”
› register name networkuptopia.com at DNS registrar (e.g., Network Solutions)

– provide names, IP addresses of authoritative name server

– registrar inserts two RRs into .com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)

(dns1.networkutopia.com, 212.212.212.1, A)

› create at authoritative server
type A record for www.networkuptopia.com;

(www.networkutopia.com, 212.212.212.22, A)

(www.home.networkutopia.com, www.networkutopia.com, CNAME)

Inserting records into DNS

Socket Programming

Socket programming

goal: learn how to build client/server applications that communicate using
sockets

socket: door between application process and end-end-transport protocol

Internet

controlled
by OS

controlled by
app developer

transport

application

physical
link

network

process

transport

application

physical
link

network

process
socket

Socket programming

Two socket types for two transport services:
– UDP: unreliable datagram
– TCP: reliable, byte stream-oriented

Application Example:
1. Client reads a line of characters (data) from its

keyboard and sends the data to the server.
2. The server receives the data and converts

characters to uppercase.
3. The server sends the modified data to the client.
4. The client receives the modified data and displays

the line on its screen.

Socket programming with UDP

UDP: no “connection” between client & server
› no handshaking before sending data
› sender explicitly attaches IP destination address and port #

to each packet
› receiver extracts sender IP address and port # from received

packet

UDP: transmitted data may be lost or received out-
of-order

Application viewpoint:
› UDP provides unreliable transfer of groups of bytes

(“datagrams”) between client and server

close
clientSocket

read datagram from
clientSocket

create socket:
clientSocket =
socket(AF_INET,SOCK_DGRAM)

Create datagram with server IP and
port=x; send datagram via
clientSocket

create socket, port= x:
serverSocket =
socket(AF_INET,SOCK_DGRAM)

read datagram from
serverSocket

write reply to
serverSocket
specifying
client address,
port number

server (running on serverIP) client

Client/server socket interaction: UDP

from socket import *
serverName = ‘hostname’
serverPort = 12000
clientSocket = socket(socket.AF_INET,

socket.SOCK_DGRAM)
message = input(’Input lowercase sentence:’)
message = message.encode(‘utf-8’)
clientSocket.sendto(message,(serverName, serverPort))
modifiedMessage, serverAddress =

clientSocket.recvfrom(2048)
print (modifiedMessage.decode(‘utf-8’))
clientSocket.close()

Python UDPClient
include Python’s socket
library

create UDP for server

get user keyboard
input

Attach server name, port to
message; send into socket

print out received string
and close socket

read reply characters from
socket into string

UDP client

convert from string to bytes
convert from bytes to string
New feature in Python 3

from socket import *
serverPort = 12000
serverSocket = socket(AF_INET, SOCK_DGRAM)
serverSocket.bind((”, serverPort))
print “The server is ready to receive”
while 1:

message, clientAddress = serverSocket.recvfrom(2048)
message=message.decode(‘utf-8’)
modifiedMessage = message.upper()
serverSocket.sendto(modifiedMessage.encode(‘utf-8’),

clientAddress)

Python UDPServer

create UDP socket

bind socket to local port
number 12000

loop forever

Read from UDP socket into
message, getting client’s
address (client IP and port)

send upper case string
back to this client

UDP server

Socket programming with TCP

client must contact server
› server process must first be

running

› server must have created socket
(door) that welcomes client’s
contact

client contacts server by:
› creating TCP socket, connecting

server by specifying IP address,
port number of server process

› client connects: client TCP
establishes connection to server
TCP

› when contacted by client, server TCP
creates new socket for server
process to communicate with that
particular client

– allows server to talk with multiple
clients

– source port numbers used to
distinguish clients

TCP provides reliable, in-order
byte-stream transfer (“pipe”)
between client and server

application viewpoint:

Accept client
connectionSocket =
serverSocket.accept()

create socket,
port=x,
serverSocket = socket()
wait for incoming
connection request
serverSocket.listen(1) create socket,

connect to hostid, port=x
clientSocket = socket()
clientSocket.connect((hostid, x))

server (running on hostid) client

send request using
clientSocket

read request from
connectionSocket

write reply to
connectionSocket

TCP
connection setup

close
connectionSocket

read reply from
clientSocket

close
clientSocket

Client-server socket interaction TCP

from socket import *
serverName = ’servername’
serverPort = 12000
clientSocket = socket(AF_INET, SOCK_STREAM)
clientSocket.connect((serverName,serverPort))
sentence = input(‘Input lowercase sentence:’)
clientSocket.send(sentence.encode(‘utf-8’))

## Do not specify serverName,serverPort

modifiedSentence = clientSocket.recv(1024)
print (‘From Server:’, modifiedSentence.decode(‘utf-8’))
clientSocket.close()

Python TCPClient

create TCP socket for
server, remote port 12000

No need to attach server
name, port

TCP client

from socket import *
serverPort = 12000
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.bind((‘’, serverPort))
serverSocket.listen(1)
print (‘The server is ready to receive’)
while 1:

connectionSocket, addr = serverSocket.accept()

sentence = connectionSocket.recv(1024)
capitalizedSentence = sentence.decode(‘utf-8’).upper().encode(‘utf-8’)

connectionSocket.send(capitalizedSentence)
connectionSocket.close()

Python TCPServer

create TCP welcoming
socket

server begins listening for
incoming TCP requests

loop forever

server waits on accept()
for incoming requests, new
socket created on return

read bytes from socket (but
not address as in UDP)

close connection to this
client (but not welcoming
socket)

TCP server

Peer-to-Peer

Pure peer-to-peer model architecture

› no always-on server
› arbitrary end systems directly

communicate

› peers are intermittently
connected and change IP
addresses

examples:

– file distribution (BitTorrent)

– Streaming (Zattoo, KanKan)

– VoIP (Skype)

File distribution: client-server vs. p2p

Question: how much time to distribute file (size F) from one server to N peers?

– peer upload/download capacity is limited resource

server

network (with abundant
bandwidth)

file, size F

us: server upload
capacity

ui: peer i upload
capacity

di: peer i download
capacityu2 d2

u1 d1

File distribution time: client-server

› server transmission: must
sequentially send (upload) N
file copies:
– time to send one copy: F/us
– time to send N copies: NF/us

increases linearly in N

time to distribute F
to N clients using

client-server approach Dc-s > max{NF/us,,F/dmin}

v client: each client must
download file copy
§ dmin = min client download rate
§ (worst case) client download time:

F/dmin

network
di

› server transmission: must upload
at least one copy
– time to send one copy: F/us

time to distribute F
to N clients using

P2P approach

network
di

DP2P > max{F/us,,F/dmin,,NF/(us + Sui)}

v client: each client must
download file copy
§ client download time: F/dmin

v clients: as aggregate must download NF bits = upload NF bits
§ Max upload rate us + Sui
§ NF/(us + Sui)

… but so does this, as each peer brings service capacity
increases linearly in N …

File distribution time: p2p

0.5

1.5

2.5

3.5

0 5 10 15 20 25 30 35

M
in

im
um

D
is

tr
ib

ut
io

n
T

im
e P2P

Client-Server

client upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us

Client-server vs. p2p

P2P file distribution: BitTorrent

› 20% of European internet traffic in 2012.

› Used for Linux distribution, software patches, distributing movies
› Goal: quickly replicate large files to large number of clients

› Web server hosts a .torrent file (w/ file length, hash, tracker’s URL…)
› A tracker tracks downloaders/owners of a file

› Files are divided into chunks (256kB-1MB)
› Downloaders download chunks from themselves (and owners)

› Tit-for-tat: the more one shares (server), the faster it can download (client)

BitTorrent, a file sharing application

tracker: tracks peers
participating in torrent

torrent: group of peers
exchanging chunks of a file

Alice arrives …

› file divided into 256KB chunks
› peers in torrent send/receive file chunks

… obtains list
of peers from tracker
… and begins exchanging
file chunks with peers in torrent

P2P file distribution: BitTorrent

› peer joining torrent:
– has no chunks, but will accumulate

them over time from other peers

– registers with tracker to get list of
peers, connects to subset of peers
(“neighbors”)

› while downloading, peer uploads chunks to other peers
› peer may change peers with whom it exchanges chunks
› churn: peers may come and go
› once peer has entire file, it may (selfishly) leave or

(altruistically) remain in torrent

BitTorrent: requesting, sending file chunks

requesting chunks:
› at any given time, different

peers have different subsets
of file chunks

› periodically, Alice asks each
peer for list of chunks that
they have

› Alice requests missing chunks
from peers, rarest first

sending chunks: tit-for-tat
› Alice sends chunks to those four

peers currently sending her
chunks at highest rate

› other peers are choked by Alice (do
not receive chunks from her)

› re-evaluate top 4 every10 secs

› every 30 secs: randomly select
another peer, starts sending
chunks

› “optimistically unchoke” this peer

› newly chosen peer may join top 4

(1) Alice “optimistically unchokes” Bob
(2) Alice becomes one of Bob’s top-four providers; Bob reciprocates
(3) Bob becomes one of Alice’s top-four providers

higher upload rate: find better
trading partners, get file faster !

BitTorrent: tit-for-tat

Related Posts