ECS781P
CLOUD COMPUTING
CLOUD NETWORKING
Lecturer: Dr. Sukhpal Singh Gill and Dr Ignacio Castro School of Electronic Engineering and Computer Science
Contents
• Energy Efficiency and Resource Management • Cloud applications
• Basics of Internet Addressing
• CDNs
• Reaching Cloud Services
2
3
Demand
3.4 MILLION EMAILS ARE EXCHANGED
6,000 TWEETS ARE SENT ON TWITTER
100,000 SEARCHES ARE MADE ON GOOGLE
`
600 ITEMS ARE SOLD ON AMAZON
250 SONGS
ARE DOWNLOADED VIA ITUNES
70,000 LIKES
ARE GENERATED ON FACEBOOK
Digital Transformation
5G and Smart Phones 6B in 2020 (Ericsson)
Machine Learning, Artificial Intelligence (AI)
Big Data, Data Analytics, Data Mining
Internet-of-Things (IoT) Devices 25B in 2020 (Gartner)
Digital Currencies
and Blockchain
Augmented Reality and
Virtual Reality (AR/VR)
Source: Cisco Global Cloud Index: Forecast and Methodology, 2015–2020
Cloud Computing
Cloud Computing
Computing provisioned as service (typically within datacenters)
Quality of Service (QoS) requirements
• Deadlines,responsetime,energy,budgetrestrictions, • Availability,reliability,security,etc.
Provisioned via Resource Management Schedulers, oversubscription, etc.
Sukhpal Singh and Inderveer Chana, “QoS-aware Autonomic Resource Management in Cloud Computing: A Systematic Review”, ACM Computing Surveys, vol. 48, no. 3, 46 pages, 2016,
Research Problem for Cloud Resource Management
Shortage of Infrastructure
Cost
Excess of Resources
Shortage and Excess of Resources
Time
Research Problem for Cloud Resource Management
Cost
Actual Infrastructure Demand
Required Resource Management Model
Time
1 minute of video stream =
0.0002kWh
PSY – Gangnam Style = 4 min 13s As of 2019, watched 3.2b times BCS estimates total of 312GWh…
Enough to power 67,826 UK homes… For a year!
BCS: British Computer Society
9
Carbon Footprints
• Data centers used for hosting cloud applications consume large amounts of electricity
• High operational cost for the cloud providers • High carbon footprint on the environment
• In 2014, US data centers alone consumed 70 billion kilowatt-hours of electricity
• = Two-year power consumption of all households in New York
• = The amount consumed by about 6.4 million average
American homes that year
• This is projected to be responsible for the emission of nearly
50 million tons of carbon pollution per annum in 2020.
• Source: US Natural Resources Defense Council (NRDC)
Sukhpal Singh Gill and Rajkumar Buyya, A Taxonomy and Future Directions for Sustainable Cloud Computing: 360 Degree View, ACM Computing Surveys, Volume 51, No. 5, Article No. 104, Pages: 1-33, ISSN 0360-0300, ACM Press, New York, USA, January 2019.
Open Challenges
• Intermittency and unpredictability of renewable energy sources (Wind and Solar):
• Powering data centers entirely with renewable energy sources is difficult
• Mixed sources of energy for data centers: • Grid power or brown energy
• Renewable energy sources or green energy
• Challenges:
• Minimizing brown energy usage
• Maximizing renewable energy utilization
Sukhpal Singh Gill and Rajkumar Buyya, Sustainable Cloud Computing Realization for Different Applications: A Manifesto, Digital Business: Business Algorithms, Cloud Computing and Data Engineering, S. Patnaik, X. Yang, M. Tavana, F. Popentiu-Vladicescu, F. Qiao (eds) 95-117pp, ISBN: 978-3-319-93939-1, Springer, Germany, July 2018.
12
Contents
• Energy Efficiency and Resource Management • Cloud applications
• Basics of Internet Addressing
• CDNs
• Reaching Cloud Services
13
Cloud applications
• Cloud computing is very attractive to the users:
• Economic reasons.
• low infrastructure investment.
• low cost – customers are only billed for resources used.
Cloud applications
• Convenience and performance.
• application developers enjoy the advantages of a just-in-time infrastructure; they are free to design an application without being concerned with the system where the application will run.
• the execution time of compute-intensive and data-intensive applications can, potentially, be reduced through parallelization. If an application can partition the workload in n segments and spawn n instances of itself, then the execution time could be reduced by a factor close to n.
• Cloud computing is also beneficial for the providers of computing cycles – it typically leads to a higher level of resource utilization.
Cloud applications (cont’d)
• Ideal applications for cloud computing:
• Web services.
• Database services.
• Transaction-based service. The resource requirements of transaction-oriented services benefit from an elastic environment where resources are available when needed and where one pays only for the resources it consumes.
Cloud applications (cont’d)
• Applications unlikely to perform well on a cloud:
• Applications with a complex workflow and multiple dependencies, as is often the case in high-performance computing.
• Applications which require intensive communication among concurrent instances.
• When the workload cannot be arbitrarily partitioned.
Challenges for cloud application development
• Performance isolation – nearly impossible to reach in a real system, especially when the system is heavily loaded.
• Reliability – major concern; server failures expected when a large number of servers cooperate for the computations.
• Cloud infrastructure exhibits latency and bandwidth fluctuations which affect the application performance.
• Performance considerations limit the amount of data logging; the ability to identify the source of unexpected results and errors is helped by frequent logging.
Existing and new application opportunities
• Three broad categories of existing applications:
• Processing pipelines.
• Batch processing systems. • Web applications.
Existing and new application opportunities
• Potentially new applications
• Batch processing for decision support systems and
business analytics.
• Mobile interactive applications which process large volumes of data from different types of sensors.
• Science and engineering could greatly benefit from cloud computing as many applications in these areas are compute-intensive and data-intensive.
Processing pipelines
• Indexing large datasets created by web crawler engines.
• Data mining – searching large collections of records to locate
items of interests. • Image processing .
• Image conversion, e.g., enlarge an image or create thumbnails.
• Compress or encrypt images.
Processing pipelines
• Video transcoding from one video format to another, e.g., from AVI to MPEG.
• Document processing.
• Convert large collections of documents from one
format to another, e.g., from Word to PDF.
• Encrypt documents.
• Use Optical Character Recognition to produce digital images of documents.
Batch processing applications
• Generation of daily, weekly, monthly, and annual activity reports for retail, manufacturing, other economical sectors.
• Processing, aggregation, and summaries of daily transactions for financial institutions, insurance companies, and healthcare organizations.
• Processing billing and payroll records.
• Management of the software development, e.g., nightly updates of
software repositories.
• Automatic testing and verification of software and hardware systems.
Web access
• Sites for online commerce.
• Sites with a periodic or temporary presence.
• Conferences or other events.
• Active during a particular season (e.g., the Holidays Season) or income tax reporting.
• Sites for promotional activities.
• Sites that “sleep” during the night and auto-scale during the day.
Autoscaling
• Computations Power • Storage
Autoscaling
• Auto-scaling monitors the performance of cloud applications and automatically adjusts the capacity to maintain steady, predictable performance and to ensure required computational power and storage space is available.
Workflows
• Process description – structure describing the tasks to be executed and the order of their execution. Resembles a flowchart.
• Case – an instance of a process description.
• State of a case at time t – defined in terms of tasks already completed at
that time.
• Events – cause transitions between states.
• The life cycle of a workflow – creation, definition, verification, and enactment; similar to the life cycle of a traditional program (creation, compilation, and execution).
Workflows
Elasticity and load distribution
• Elasticityability to use as many servers as necessary to optimally respond to cost and timing constraints of an application.
• How to divide the load
• Transaction processing systemsa front-end distributes the incoming transactions to a number of back-end systems. As the workload increases new back-end systems are added to the pool.
• For data-intensive batch applications two types of divisible workloads are possible:
• modularly divisibleload is a prior divided into pre-
defined modules.
• arbitrarily divisiblethe workload can be partitioned into an arbitrarily large number of smaller workloads of equal, or very close size.
• Many applications in physics, biology, and other areas of computational science and engineering obey the arbitrarily divisible load sharing model.
MapReduce philosophy
1. An application starts a master instance, M worker instances for the Map phase and later R worker instances for the Reduce phase.
2. The master instance partitions the input data in M segments.
3. Each map instance reads its input data segment and processes the data.
4. The results of the processing are stored on the local disks of the servers where the map instances run.
5. When all map instances have finished processing their data, the R reduce instances read the results of the first phase and merge the partial results.
6. The final results are written by the reduce instances to a shared storage server.
7. The master instance monitors the reduce instances and when all of them report task completion the application is terminated.
Application
1
Master instance
2
11
Local disk Local disk Local disk
7
Segment 1
Segment 1 Segment 2
Segment 3
Map instance 1
Map instance 2
Shared storage
Shared storage
Map instance 3
Segment M
Input data
3456 Local disk
Map phase
Reduce phase
Reduce instance 1
Reduce instance 2
Reduce instance R
Map instance M
Clouds for science and engineering
• The generic problems in virtually all areas of science are:
• Collection of experimental data.
• Management of very large volumes of data.
• Building and execution of models.
• Integration of data and literature.
• Documentation of the experiments.
• Sharing the data with others; data preservation for a long periods of time.
• All these activities require “big” data storage and systems capable to deliver abundant computing cycles.
Computing clouds are able to provide such resources and support collaborative environments.
Legacy applications on the cloud
• Is it feasible to run legacy applications on a cloud?
• BLAST – a biology code which finds regions of local similarity between sequences; it compares protein sequences to sequence databases and calculates the statistical significance of matches; used to infer functional and evolutionary relationships between sequences and identify members of gene families.
Networked applications
Programs that:
• run on (different) end systems • communicate over network
• e.g., web server software communicates with mobile browser software
• applications only on end systems • allows for rapid app development,
propagation
• two main interaction types • client – server
• peer to peer
application
transport
network
data link
physical
application
transport
network
application
transport
network
data link physical
data link
physical
35
Some cloud applications
36
Geo-distributed applications
J. Deng et al, Internet scale user-generated live video streaming: The Twitch case. Proc. Of PAM 2017
37
Agri-Info: Cloud Based Autonomic Technique for Delivering Agriculture as a Service
Sukhpal Singh Gill, Inderveer Chana and Rajkumar Buyya, “IoT based Agriculture as a Cloud and Big Data Service: The Beginning of Digital India”, Journal of Organizational and End User Computing (JOEUC), [IGI Global], vol. 29, no. 4, 2017
38
Agri-Info Architecture
39
Android based Cloud Application for Agri-Info
Find Productivity Crop-Information
Sukhpal Singh, Inderveer Chana and Rajkumar Buyya, Agri-Info: Cloud Based Autonomic System for Delivering Agriculture as a Service, Internet of Things: Engineering Cyber Physical Human Systems, Volume 9, Pages: 1-16, ISSN 2542-6605, Elsevier Press, Amsterdam, The Netherlands, March 2020.
5 steps to building a cloud-ready application
• Design the application as a collection of services • Data and Cloud Service
• Decouple the data
• Store on Public Cloud
• Consider communications between application components • Combine communications into a single stream of data
• Model and design for performance and scaling
• Application (Amazon/ebay) will scale under an increasing load
• Make security systemic within the application • Data Protection for healthcare application
41
42
Contents
• Energy Efficiency and Resource Management • Cloud applications
• Basics of Internet Addressing
• CDNs
• Reaching Cloud Services
43
What is the Internet?
PC • servers
wireless laptop
smartphone
wireless links
wired links
router
billions of connected computing devices:
– hosts = end systems
– running network apps
mobile network
global ISP
• communication links
– fiber, copper,
radio, satellite
– transmission rate: bandwidth
home network
institutional network
regional ISP
• packet switches: forward packets (chunks of data)
– routers and switches
44
How does the Internet look like?
access net
access net
ISP A
access net
access net
access net
IXP
access net
access net
access net
access net
access net
Content provider network
IXP
ISP C
regional net
access access net
access net
access net
ISP B
access net
access net
net
45
It’s a Network of networks
Tier 1 ISP
IXP
Tier 1 ISP
Google
IXP
Regional ISP
access access ISP ISP
IXP
access access ISP ISP
Regional ISP
access access ISP ISP
access access ISP ISP
at center: small # of well-connected large networks
• “tier-1” commercial ISPs (e.g., Level 3, AT&T), national & international coverage
• content provider network (e.g, Google, Akamai): private network that connects its data centers or racks to Internet, often bypassing tier-1, regional ISPs
46
The problem of addressing
• We need to be able to send requests to a service from anywhere on the Internet
• How can we give a unique address to every possible service?
• … in a way that the Internet can automatically find its way until that address …
• Service addresses have multiple parts
Host name IP Address Port
47
•
IP addressing
IP address: 32-bit identifier for host, router interface
223.1.1.1
223.1.1.2
223.1.2.1
223.1.2.9
223.1.1.4
• Internet-wide identifiers
223.1.1.3
223.1.3.27
•
Routers know how to get packet to destination IP
223.1.2.2
223.1.3.2
223.1.3.1
•
Dynamic: depend on the subnetwork the interface is connected
223.1.1.1 = 11011111 00000001 00000001 00000001 223 1 1 1
48
Subnets
• • •
•
• –
–
The whole space of IP addresses is allocated to ISPs and other Internet entities (e.g. public clouds)
Subnet: Set of interfaces that can physically reach each other without an intervening router
IP addresses belong to a subnet
Dynamic Host Configuration Protocol (DHCP) assigns free IP
addresses when a machine joins a subnetwork IP Addresses have two parts:
subnet part – high order bits host part – low order bits
<--Network address--><--Subnet address--><--Host address--> 130.5 . 5 . 25
49
Sockets
Socket: door between application process and end-end-transport protocol
The API for applications to communicate across the network
application
process
transport
network
link
physical
application
process
transport
network
link
physical
socket
Internet
controlled by app developer
controlled by OS
50
Sockets and ports
• Logical resources managed by the operating system
• Sockets are always assigned a free port when created
• Each process on a networked host can be addressed remotely by the port number it is listening to
• TCP and UDP ports are independent
• For server-side applications, default port numbers are
defined
• HTTP -> 80
• HTTPS -> 443 • SMTP -> 25
51
DNS: domain name system
• Distributed database implemented in a hierarchy of many name servers
• Maps hostnames to IP addresses
• Application-layer protocol: hosts, name servers communicate to resolve names (address/name translation)
52
DNS: a distributed, hierarchical database
Client wants IP for www.amazon.com:
• client queries root serverto find .com DNS server
DNS: a distributed, hierarchical database
Client wants IP for www.amazon.com:
• client queries root serverto find .com DNS server
DNS: a distributed, hierarchical database
Client wants IP for www.amazon.com:
• client queries root serverto find .com DNS server
• client queries .com DNS server to get amazon.com DNS server
• client queries amazon.com DNS server to get IP address for www.amazon.com
56
Contents
• Energy Efficiency and Resource Management • Cloud applications
• Basics of Internet Addressing
• CDNs
• Reaching Cloud Services
57
Web caches
• Cache servers aim to improve efficiency by reducing
– response time
– traffic to network
• Browser sends all HTTP requests to cache – object in cache:
Web cache
cache returns object
– else cache requests object from origin server, then returns object to client
client
origin server
client
origin
server
58
Content Delivery Network
• Infrastructure: large distributed system of servers deployed in multiple data centers across the Internet
• Goal: serve content to end-users with high availability and high performance
• Business:
– Media companies and e-commerce vendors pay CDN
operators to deliver their content
– CDN pays ISPs, carriers, and network operators for hosting its servers in their data centres
59
CDN vs Non-CDN model
Non-CDN model CDN model
60
CDN deployments
• Servers deployed around the world, often deep inside ISP networks
• Serve the same content from multiple (all) locations
• Examples:
– Akamai
– Limelight
– Amazon CloudFront
– Level 3
– Windows Azure
– ChinaCache
61
CDN DNS Redirectioning
DNS
HTTP
Where is Google?
Source: Calder, Matt, et al. “Mapping the expansion of Google’s serving infrastructure.” Proceedings ACM IMC, 2013.
The Netflix server map
Source: Boettger et al, “Open connect everywhere: A glimpse at the internet ecosystem through the lens of the netflix cdn.“ SIGCOMM CCR
64
AWS CloudFront
Example: qmplus.qmul.ac.uk
• Which IP?
• https://www.whatismyip.com/
• https://www.whatismyipaddress.com/
• Which port?
66
67
Contents
• Energy Efficiency and Resource Management • Cloud applications
• Basics of Internet Addressing
• CDNs
• Reaching Cloud Services
68
Reaching Virtual Machines
• VMs and Containers are created at a host, by allocating resources.
• Connectivity is provided by the host
• Messages have
to physically go through the host Network Interface Card (NIC)
69
VM to VM Communications
• VMs usually have private IP addresses
• Reserved ranges that only work locally
• E.g. 10.0.0.x, 192.168.0.x
• Interfaces are bridged by
hosts
• Also across multiple hosts
70
Reachability to the rest of the Internet
• NAT (Network Address Translation)
• Middlebox translating external ip + port into a private ip + port
• DNS is a high-level solution in public clouds
• AWS Router 53
71
AWS Router 53
• Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. It is designed to give developers and businesses an extremely reliable and cost effective way
to route end users to Internet applications by translating names like www.example.com into the numeric IP addresses like 192.0.
72
73
Coursework Choice ?
• Group Lab Project
• Individual Lab Project
• https://forms.gle/Pu3p5DzGasTTLmGPA
74