Cloud Computing INFS3208
Re-cap
1. CourseOrientation
2. HistoryandDefinitionofCloudComputing
3. BusinessDriversforcreationofCloudComputing
– Capacity Planning, Cost Reduction, Organisational Agility 4. TechnologiesthatimpactCloudComputing
– Clustering, Grid Computing, Virtualisation 5. CloudCharacteristics
– On-demand usage, Ubiquitous access, Multitenancy, Elastic, Measurable, Resilient.
CRICOS code 00025B 2
Outline
• CloudDeliveryModels
• CloudDeployModels
• Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld • Summary
CRICOS code 00025B 3
Cloud Delivery Models
A cloud delivery model represents a specific, pre-packaged combination of IT resources offered by a cloud provider.
Less
Three common cloud delivery models:
• Infrastructure-as-a-Service (IaaS)
• Platform-as-a-Service (PaaS)
• Software-as-a-Service (SaaS)
SaaS
PaaS
IaaS
CRICOS code 00025B 4
More
control
Cloud Delivery Models — IaaS
Infrastructure-as-a-Service (IaaS):
• a self-contained virtual environment consists of infrastructure-centric IT
resources
• IT resources can be accessed and managed via cloud service-based
interfaces and tools
• can include hardware, network, connectivity, operating systems, and
other “raw” IT resources
• provides a high level of control and responsibility over its configuration
and utilization
• needs cloud consumers’ administrative responsibility
• can be different by different cloud providers (different specs: CPU cores,
RAM, storage, etc)
• is generally offered as freshly initialized virtual machines
• users: system admins
• examples: Amazon EC2, Google Compute Engine (GCE), Microsoft
Azure
IaaS
CRICOS code 00025B
5
Cloud Delivery Models — PaaS
Platform-as-a-Service (PaaS):
• a pre-defined “ready-to-use” environment typically consists of already deployed and configured IT resources
• can include a programming language execution environment, an operating system, a web server, and a database.
• encapsulates an environment where users can build, compile, and run program without worrying about the infrastructure.
• no need to take administrative and maintaining responsibility
• needs users to manage their own data (e.g. with SQL databases)
• Lower level of control over the underlying IT resources
• users: developers
• examples:
• AWS Elastic Beanstalk
• Google app engine (GAE)
PaaS
CRICOS code 00025B
6
Cloud Delivery Models — SaaS
Software-as-a-Service (SaaS):
• a shared cloud service and made available as a “product”
• provides on-demand services
• no installation of the software on users’ PCs
• assessible via a web browser or lightweight client apps.
• run a single instance of the software
• can be available for multiple users
• has very limited administrative control over a SaaS implementation.
• users: end-users
• examples:
• Google ecosystem docs/sheets/mails/calendar/etc
• Microsoft Office 365
SaaS
CRICOS code 00025B
7
Model Comparisons
Delivery Model
Control Level
Functionality
Consumer Activities
Provider Activities
SaaS
Usage and usage-related configuration
Access to front-end user-interface
Uses and configures cloud services
Implements, manages, and maintains cloud service Monitors usage by cloud consumers
PaaS
Limited administrative
Moderate level of administrative control over IT resources relevant to cloud consumer’s usage of platform
Develops, tests, deploys, and manages cloud services and cloud-based solutions
Pre-configures platform and provisions underlying infrastructure, middleware, and other needed IT resources, as necessary Monitors usage by cloud consumers
IaaS
Full administrative
Full access to virtualized infrastructure-related IT resources and possibly to underlying physical IT resources
Sets up and configures bare infrastructure, and installs, manages, and monitors any needed software
Provisions and manages the physical processing, storage, networking, and hosting required Monitors usage by cloud consumers
CRICOS code 00025B
8
Model Comparisons
On Premises
Infrastructure
Platform
Software
(as a Service)
(as a Service)
(as a Service)
Application
Application
Application
Application
Data
Data
Data
Data
O/S
O/S
O/S
O/S
Virtualizatio n
Virtualization
Virtualization
Virtualization
Servers
Servers
Servers
Servers
Storage
Storage
Storage
Storage
Networking
Networking
Networking
Networking
Network Architects Application System Admins Developers
End Users
CRICOS code 00025B
9
Managed by Providers
Managed by Users Managed by Providers
Managed by Managed by Users Providers
Managed by Users
New Adventures in the Cloud
https://www.cnbc.com/2020/08/04/microsoft-reveals-more-details-about-its-xcloud-game-streaming-service.html
https://www.xbox.com/en-US/xbox-game-pass/cloud-gaming https://stadia.google.com/
CRICOS code 00025B 10
Examples of Cloud Computing Usage
Amazon’s AWS:
• leading company in Cloud Computing
• provides IaaS and PaaS
• famous for EC2 Google Cloud:
• offers IaaS, PaaS (GAE), and SaaS (Google docs/sheets/calendar/gmail)
Microsoft Azure:
• provides IaaS, PaaS, and SaaS (Office 365)
iCloud:
• majorly for Apple products (Macbook, iPad, iPhone, etc)
• store and backup users’ documents online.
• Storage-as-a-service (aka STaaS)
Netflix
VM on GCE
CRICOS code 00025B 11
SaaS
AWS Elastic Beanstalk
PaaS
Google Doc
Google App Engine
IaaS
Dropbox
Outline
• CloudDeliveryModels
• CloudDeployModels
• Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld
CRICOS code 00025B 12
Cloud Deployment Models
A cloud deployment model represents a specific type of cloud environment
In terms of ownership, size, and access, models can be divided into four common groups:
•
– –
– –
•
– – –
–
Public cloud
a publicly accessible cloud environment owned by a third-party cloud provider usually supplied via the delivery models and offered to consumers at a cost
is created and on-going maintained by the cloud provider.
typical examples: GCP, AWS, AZURE, etc.
Community cloud
is similar to a public cloud except that its access is limited to a community of cloud consumers.
may be jointly owned by the community members or by a third-party cloud provider
cloud consumers of the community typically share the responsibility for defining and evolving the
community cloud
Typical examples: Cloud for multiple governmental departments.
CRICOS code 00025B 13
Cloud Deployment Models
•
–
– –
–
•
– – –
Private cloud
is owned by a single organization and enables an organization to use CC technology to access to IT resources by different parts, locations, or departments.
actual administration may be carried out by internal or outsourced staff.
within a private cloud, the same organization is technically both the cloud consumer and
provider
Typical examples: UQCloud (VMs) and UQRDM cloud (STaaS)
Hybrid cloud
is a cloud environment comprised of two or more different cloud deployment models. Example: private cloud (sensitive data) + public cloud (less sensitive cloud services)
can be complex and challenging to create and maintain due to the potential disparity in cloud environments
CRICOS code 00025B 14
Outline
• CloudDeliveryModels
• CloudDeployModels
• Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld • Summary
CRICOS code 00025B 15
Cloud-Enabling Technology (CET)
Modern-day clouds are underpinned by a set of primary technology components that collectively enable key features and characteristics associated with contemporary cloud computing:
• BroadbandNetworksandInternetArchitecture • VirtualisationTechnology
• Data Centre Technology
• WebTechnology
• MultitenantTechnology
CRICOS code 00025B 16
CET I – Broadband Networks and Internet Architecture
• All clouds must be connected to a network (an inherent dependency on WWW, most clouds are Internet-enabled).
• Worldwide connectivity is enabled through a hierarchical topology composed of Tiers 1, 2, and 3.
• Cloud consumers and cloud providers typically use the Internet to communicate.
• Easily configuration of IT resources (external and internal users via WWW) and superior connectivity
• The potential of cloud platforms therefore generally grows in parallel with advancements in Internet connectivity and service quality (bandwidth, latency, protocols, etc.)
CRICOS code 00025B 17
CET II: Virtualisation Technology (VT)
• •
VT is the process of converting a physical IT resource into a virtual IT resource.
Most types of IT resources can be virtualised, including:
– Servers – A physical server can be abstracted into a virtual server.
– Storage – A physical storage device can be abstracted into a virtual storage device or a virtual disk.
– Network – Physical routers and switches can be abstracted into logical network fabrics, such as VLANs.
– Power – A physical UPS and power distribution units can be abstracted into what are commonly referred to as virtual UPSs.
A physical server is called a host or physical host.
A software that manages VMs and hardware is called as Virtual Machine
Monitor (VMM), also known as hypervisor in cloud computing context. An operating system in a virtual machine is called as guest OS.
• •
•
CRICOS code 00025B 18
CET II: Virtualisation Technology (VT)
• Steps of creating a new virtual server through virtualisation software:
I. the allocation of physical IT resources (e.g. specify #CPU, Mem, Storage in VirtualBox by Oracle);
II. followed by the installation of an operating system (e.g. Install Ubuntu or Windows systems in VirtualBox).
• Virtual servers use their own guest operating systems, which are independent of the operating system in which they were created.
• Guest OS and the application software on the virtual server are unaware of the virtualisation process.
MySQL
VM1 VM2 VM3 VM4
CRICOS code 00025B 19
CET II: Virtualisation Technology (VT)
Hardware Independence
• VT can convert and translate IT hardware into emulated and standardised software-based copies.
• Due to HI, virtual servers can easily be moved to another virtualisation host
• Thus, cloning and manipulating virtual IT resources is much easier than duplicating physical hardware.
Server Consolidation
• VT enables different virtual servers to share one physical server, which is called server consolidation
• SC is commonly used to increase hardware utilisation, load balancing, and optimisation of available IT resources.
• The resulting flexibility: different virtual servers can run different guest operating systems on the same host.
• supports common cloud features, e.g. on-demand usage, resource pooling, elasticity, scalability, and resiliency.
CRICOS code 00025B 20
CET II: Virtualisation Technology (VT)
Resource Replication
• Virtual servers are created as virtual disk images that contain binary file copies of hard disk content.
• Host’s OS can access these disk images e.g. copy, move, and paste (replicate, migrate, and back up the virtual server).
• In this way, it enables:
– Standard virtual machine creations with common configurations
– Increased agility in the migration and deployment of a virtual machine’s new instances – Backup & Roll back abilities
copy
copy
One Host Host A
Host
BCRICOS code 00025B 21
CET II: Virtualisation Technology (VT)
Operating System-Based Virtualisation
• virtualisation is the installation of virtualisation software in a pre-
existing operating system (called the host operating system)
• example: Install ubuntu on Windows with VMware/VirtualBox
• processing overhead: virtualisation software and host OS.
Hardware-Based Virtualisation
• the installation of virtualisation software directly on the physical host hardware bypassing the host OS
• example: Oracle VM Server for x86 (up to 384 CPUs and 6TB RAM)
• VMM is also named as Hypervisor
• more efficient (no hosting OS), but compatible issues.
operating system-based virtualisation
Hardware-based virtualisation
CRICOS code 00025B 22
CET III – Data Centre Technology
A data centre is a specialised IT infrastructure that houses centralised IT resources
• Servers (rack in cabinet);
• Databases and software systems;
• Networking and telecommunication devices.
Typical technologies and components • Virtualisation:
– –
Data centres consist of both physical and virtualised IT resources.
The physical IT resource layer refers to the facility infrastructure that houses:
computing/networking systems and equipment, hardware systems and their operating systems.
CRICOS code 00025B 23
CET III – Data Centre Technology
Typical technologies and components of Data Centre: • Standardisation and Modularity:
– DCs are built upon standardised commodity hardware and designed with modular architectures.
– reduce investment and operational costs. • Automation:
– DCs have specialised platforms that automate general management tasks such as provisioning, configuration, patching, and monitoring without supervision.
• Remote Operation and Management:
– Remotely access via consoles and management systems: most of the tasks in DCs (e.g.
operational and administrative tasks).
– On-site jobs: highly specific tasks – equipment handling and cabling or hardware-level installation and maintenance.
CRICOS code 00025B 24
CET III – Data Centre Technology
Typical technologies and components of Data Centre: • High Availability:
– Aiming to high-level availability, DCs usually have redundant, uninterruptable power supplies, cabling, and environmental control subsystems in anticipation of system failure, along with communication links and clustered hardware for load balancing.
• Security-Aware Design, Operation and Management:
– Security requirements (e.g. physical and logical access controls and data recovery strategies)
need to be comprehensive for DCs. • Facilities:
– Site: custom-designed locations that are outfitted with specialised computing, storage, and network equipment.
– Layout: multiple functional areas
– various power supplies, cabling, and environmental control stations that regulate heating,
ventilation, air conditioning, fire protection, and other related subsystems.
CRICOS code 00025B 25
CET III – Data Centre Technology
Hardware of Data Centres:
• Computing Hardware:
– rackmount form factor server (multiple racks in a cabinet);
– a power-efficient multi-core CPU architecture (many cores but low frequency, e.g. Xeon/EPYC CPUs);
– redundant and hot-swappable components, such as hard disks, power supplies, network interfaces, and storage controller cards.
• Storage Hardware:
– specialised storage systems that maintain enormous amounts of digital information in order to fulfill considerable storage capacity needs by using arrays of disks;
– frequently used storage technologies: RAID, Hot-Swappable, Virtualisation, and Fast Data Replication Mechanisms.
• Network Hardware:
– LAN fabric, high-performance switches & adaptors (up to 10 G/s), etc.
CRICOS code 00025B 26
CET III – World-class Data Centres
5 Largest Data Centres in the world 1 (as of date 22 May 2018)
1. Digital Reality Main Data Centre (San Francisco, US)
2. Global Switch (Singapore)
3. Du Technology data center in Virginia (Virginia, US)
4. CyrusOne’s Data Center (Arizona, US)
5. China Telecom’s Inner Mongolia data center (China) (No 1 in 2020)
Largest Data Centre in Australia: EQUINIX SY3 Data Centre (Sydney) 2
1. https://www.avalon.host/blog/5-largest-data-centers-in-the-world/ 2. https://cloudscene.com/market/data-centers-in-australia/all
CRICOS code 00025B 27
CET III – Data Centre in Brisbane
We’re here!
NEXTDC Brisbane Data Centre https://www.nextdc.com/ CRICOS code 00025B 28
CET IV – Web Technology
• Web technology is very commonly used for cloud service implementations and for front-ends used to remotely manage cloud-based IT resources.
• Fundamental technologies of Web architecture:
– Uniform Resource Locator (URL) – A standard syntax used for creating identifiers that point to Web- based resources, the URL is often structured using a logical network location.
– Hypertext Transfer Protocol (HTTP) – This is the primary communications protocol used to exchange content and data throughout the World Wide Web. URLs are typically transmitted via HTTP.
– Markup Languages (HTML, XML) – Markup languages provide a lightweight means of expressing Web- centric data and metadata: HTML (webpages) and XML (data).
• Example: a web browser can request to execute an action like read, write, update, or delete on a web resource on the Internet, and proceed to identify and locate the Web resource through its URL. The request is sent using HTTP to the resource host, which is also identified by a URL. The Web server locates the Web resource and performs the requested operation, which is followed by a response being sent back to the client. The response may be comprised of content that includes HTML and XML statements.
CRICOS code 00025B 29
CET V – Multitenant Technology
• Multitenant application enables multiple users (tenants) to access the same application logic simultaneously.
• Each tenant has its own view of the application that it uses, administers, and customises as a dedicated instance of the software while remaining unaware of other tenants that are using the same application.
• Multitenant applications ensure that tenants do not have access to data and configuration information that is not their own.
• Tenants can individually customise features of the application:
– User Interface – Tenants can define a specialised “look and feel” for their application interface.
– Business Process – Tenants can customise the rules, logic, and workflows of the business processes that are implemented in the application.
– Data Model – Tenants can extend the data schema of the application to include, exclude, or rename fields in the application data structures.
– Access Control – Tenants can independently control the access rights for users and groups.
CRICOS code 00025B 30
CET V – Multitenant Technology
• Multitenant application architecture is significantly more complex than that of single-tenant applications.
• Common characteristics of multitenant applications include:
– Usage Isolation – individual behaviour does NOT affect the other tenants’ behaviours.
– Data Security – Tenants cannot access data that belongs to other tenants.
– Recovery – Backup and restore procedures are separately executed for the data of each tenant.
– Application Upgrades – Tenants are not negatively affected by the synchronous upgrading of shared software artifacts.
– Scalability – The application can scale to accommodate increases in usage by existing tenants and/or increases in the number of tenants.
– Metered Usage – Tenants are charged only for the application processing and features that are actually consumed.
– Data Tier Isolation – Tenants can have individual databases, tables, and/or schemas isolated from other tenants. Alternatively, databases, tables, and/or schemas can be designed to be intentionally shared by tenants.
CRICOS code 00025B 31
CET V – Multitenant Technology
Multitenancy vs. is sometimes mistaken for virtualisation because the concept of multiple tenants is similar to the concept of virtualised instances.
The differences lie in what is multiplied within a physical server acting as a host:
• With virtualisation:
– Multiple virtual copies of the server environment can be hosted by a single physical server.
– Each copy can be provided to different users, can be configured independently, and can contain its own operating systems and applications.
• With multitenancy:
– A physical or virtual server hosting an application is designed to allow
usage by multiple different users.
– Each user feels as though they have exclusive usage of the application.
U1 U2 U3
VM1 VM2 VM3 VMM
Hardware
U1 U2 U3
CRICOS code 00025B 32
Multitenant application
Hardware
Outline
• CloudDeliveryModels
• CloudDeployModels
• Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld • Summary
CRICOS code 00025B 33
Goals and Benefits
Reduced Investments and Proportional
Costs
• Capital reduced by cloud provider (mass-inquisition, hardware/software sharing, and data centre deployments)
• Elimination or minimization of IT investments (focus on core business)
• Common measurable benefits, e.g. on-demand access to pay-as- you-go computing resources (CPU by hr)
Increased Scalability
• can instantly and dynamically allocate IT resources
• always meet and fulfil unpredictable demands avoids potential loss
Increased Availability and Reliability
• resilient IT resources
• failover support (recovery)
changing demand for an IT resource over a day
CRICOS code 00025B 34
Cloud Computing is for whom?
Who should adopt cloud computing
• Users/Business who need Collaboration work
• Users/Business want reduce operation cost
• Users/Business with increasing and changing needs
Who shouldn’t consider cloud computing
• Offline users/business
• The security-conscious
• Anyone tied to existing applications and without alternative
CRICOS code 00025B 35
Outline
• CloudDeliveryModels
• CloudDeployModels
• Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld • Summary
CRICOS code 00025B 36
Risks and Challenges
• Increased Security Vulnerabilities
• Reduced Operational Governance Control
• Limited Portability Between Cloud Providers
• Multi-regional Compliance and Legal Issues
CRICOS code 00025B 37
Outline
• CloudDeliveryModels
• CloudDeployModels
• Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld • Summary
CRICOS code 00025B 38
Revisit: Importance of Cloud Computing
• •
•
• •
•
It is projected that there will be 24 billion devices on the Internet by 2020.
The cloud will become more important as a controller of and resource provider for the Internet of
Things (IoT).
Gartner predicts that the cloud services marketplace will be worth $206.2 billion in 2019, and will grow to
$278.3 billion in 2021.
IDC predicts a similar market size of $210 billion in 2019, rising to $370 billion in 2022.
Ten years ago, many people claimed that cloud computing was a fad that would never catch on. But today there can be no doubt that cloud computing is a very significant and growing computing trend.
In the future, “ubiquitous cities” and “smart homes” can be built on cloud computing (cloud supported/controlled robotics).
– Cloud robotics is an emerging field of robotics ingrained in cloud computing.
– Cloud robotics provides a shared knowledge database.
– Cloud robotics has the abilities about powerful computational, storage, and communications resources with cloud.
– Cloud robotics offloads heavy computing tasks to the cloud.
https://www.explainingcomputers.com/cloud.html
https://www.gartner.com/en/newsroom/press-releases/2018-09-12-gartner-forecasts-worldwide-public- cloud-revenue-to-grow-17-percent-in-2019
https://www.idc.com/getdoc.jsp?containerId=prUS44891519
CRICOS code 00025B 39
Who are using Google Cloud Platform (GCP)?
https://cloud.google.com/customers/#/
CRICOS code 00025B 40
Examples of Cloud-based Applications – I
E-Health: Analysing breast cancer images faster and better with machine learning by ACS
Background:
• Cancer is the second most common cause of death in the United States (nearly 1⁄4 deaths).
• breast cancer is the most commonly diagnosed type of cancer and the second leading cause of cancer death in the United States.
• If detected early, breast cancer is one of the most survivable cancers: the five- and ten-year relative survival rates for women with invasive breast cancer are 90 percent and 83 percent, respectively.
• However, some molecular subtypes of breast cancer have a poor prognosis and there is limited understanding of these subtypes.
• Since 1992, the American Cancer Society has conducted the Cancer Prevention Study-II (CPS-II) Nutrition cohort, a prospective study of more than 188,000 American men and women.
– CPS-II provides valuable factors: height, weight, demographic characteristics, family history, use of medicines, etc.
– CPS-II provides medical records and surgical tissue samples for approximately 1,700 CPS-II participants diagnosed with
breast cancer
Aim to answer:
• •
What lifestyle, medical, and genetic factors are related to molecular subtypes of breast cancer? Do different features in the breast cancer tissue translate to a better survival rate?
https://cloud.google.com/customers/american-cancer-society/
https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and- figures/2018/cancer-facts-and-figures-2018.pdf
CRICOS code 00025B 41
Examples of Cloud-based Applications – I
Challenges:
• The computing power to analyse the high-resolution tissue;
• Effective and efficient detection by human experts;
• Labour costs of novel pattern detection in the image data. Solutions:
• Convert the tissue images into TIF format and store on Cloud Storage (scalable and data security)
• Pre-process Data (colour normalisation, etc)
• Run ML models on Cloud ML Engine (ease of use and latest): unsupervised deep learning models – allow algorithms to determine the accuracy of their predictions and make adjustments without an engineer stepping in.
Results:
• 12x faster image analysis with ML for improved patient outcomes
• Understandable outcomes guide clinicians for more effective treatments.
https://cloud.google.com/customers/american-cancer-society/
https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and- figures/2018/cancer-facts-and-figures-2018.pdf
high-resolution tissue images – uncompressed, proprietary format
Conversion
Compute Engine Cloud Storage
CRICOS code 00025B
Cloud ML Engine 42
Pre- processing
Clustering
Examples of Cloud-based Applications – II
Social Media Analysis: Queensland University of Technology (QUT)
Background:
• Social media has significantly change our daily life and style
• 500 mil tweets per day (5,787 tw/sec)
• Public opinions and sentiments about events are becoming more available and useful
• A group of world-leading researcher and academics in QUT Digital Media Research Centre are collecting, analysing, and visualising Aussie tweets in the QUT Digital Observatory project.
Aims:
• • •
analyses of the response to and interaction with particular events on digital platforms – short term tracking of public communication and consumption of content – long term
support to research projects that explore social media activities
https://cloud.google.com/customers/qut/
https://www.qut.edu.au/institute-for-future-environments/facilities/digital-observatory
CRICOS code 00025B 43
Examples of Cloud-based Applications – II
Challenges:
• Storage infrastructure to support such a huge and live dataset:
– A collection of tweets from all identified Australian accounts, collected since 2006.
– A live data storage: 2.4 billion tweets — growing at about 1.3 mil tw/day.
– User information: 3.7 mil Australian Twitter accounts & 140k daily active users.
• Processing capacity:
– Online queries, analysis and visualisation
Solutions:
• Google BigQuery: an analytics data warehouse to capture fast-growing datasets
• Google Reporting and Visualisation Tools Results:
• Support projects with industry partners, such as analyses of social media activity around natural crises.
• Help deliver insights about the sharing and consumption of news sources on Twitter
CRICOS code 00025B 44
Examples of Cloud-based Applications – III
QSearch: Creating marketing opportunities with social media analytics
Background:
• Opinions, thoughts, reviews, comments are usually posted and shared on social media
• Behaviours and demographics can be discovered by pattern recognition
• Social networks are a rich repository of user experiences and information for agencies, businesses, and government organisations.
Aims:
Analysing social networks for product, brand, or topic trends to
• identify opportunities,
• spend campaign dollars wisely,
• manage crises effectively.
https://cloud.google.com/customers/qsearch/
CRICOS code 00025B 45
Examples of Cloud-based Applications – III
Challenges:
• Big data storage and analytic power
• Data visualisation kits
• Infrastructure expense control for a start-up company with 10 employees
Solutions:
• BigQuery analytics data warehouse
• App Engine application development
• visualisation tools
Results:
• Records only one error per one million requests for short-URL customer tracking service
• Enables the business to connect to and integrate social media search and measurement tools
• Measures employee productivity and controls access to customer projects
• Analyses 43.2 billion records in 8 hours, not days
https://cloud.google.com/customers/qsearch/
CRICOS code 00025B 46
Examples of Cloud-based Applications – IV
E-Government: Improving the customer experience and worker productivity by Service NSW
Background:
• Service NSW is a whole of Wales Government Service access point that provides online, and in-person offices that handle more than 800 types of transactions including:
– Drivers licences, photo cards and vehicle and boat registration services
– Functions of NSW Registry of Births Deaths & Marriages
– Obtaining Opal cards for public transport, Seniors Card
– Etc.
• In 130 offices across the entire state, there are many customer- service kiosks serving 7.89 million people (as of 2018).
https://www.service.nsw.gov.au/service-centre
https://cloud.google.com/customers/service-nsw/ http://www.population.net.au/population-of-new-south-wales/
CRICOS code 00025B 47
Examples of Cloud-based Applications – IV
Challenges:
• Old kiosks’ operating system lacked remote service features and required time-consuming hard drive repairs
• Maintaining all the kiosks across the entire state is expensive
Solutions:
• Chrome devices for kiosks
• G Suite: Gmail, Hangouts, Calendar, and Google+ for communication; Drive for storage; Docs, Sheets, Slides, Forms, and Sites for collaboration.
Results:
• Chromebases require only 5 percent of support hours needed by Microsoft devices
• Cloud tools eliminate the need for costly private WAN networks
• Reduced annual IT operational costs by 46 percent
• Helps Service NSW meet goal to perform 70 percent of government transactions digitally by 2019
• Improves employee productivity, collaboration and data security
https://cloud.google.com/customers/service-nsw/
CRICOS code 00025B 48
Examples of Cloud-based Applications – V
HKTaxi: Using Google to deliver an intelligent, reliable taxi booking service
Background:
• Hongkong is one of most crowded cities in the world – 7.492 million (2019) and density is 7,134 p/km2 (3 for Australia)
• Hailing taxis in Hongkong can be a frustrating experience — particularly during demand peaks.
• Mobile phones and apps are making the experience easier and more transparent.
Aims:
• Highly efficient booking service: enable drivers and users locate each other
• Effective routes to destinations during peaking hours
• Less administration, more app development
https://cloud.google.com/customers/hktaxi/ https://hktaxiapp.com/
CRICOS code 00025B 49
Examples of Cloud-based Applications – V
Challenges:
• Support electronic payment services
• Service rating: enable users to rate drivers (better service)
• 24*7 connections of drivers and users
• Support staff to resolve any issues.
Solutions:
• Google Maps SDK for iOS/Android
• Cloud Machine Learning Engine: to predict the attractiveness of orders to drivers
Results:
• Stable service for both drivers and users
• More focus on app development, less administration cost
https://cloud.google.com/customers/hktaxi/ https://hktaxiapp.com/
CRICOS code 00025B 50
Examples of Cloud-based Applications – VI
GO-JEK: Using Machine Learning for forecasting and dynamic pricing
Background:
• Traffic congestion is a fact of life for most Indonesian residents.
• The nation’s roads and associated infrastructure strains to support the country’s 260 million people, about 10 million of whom reside in the capital, Jakarta.
• To minimise delays, Indonesians rely heavily on motorcycles, including motorcycle taxis, to travel to and from work or personal engagements.
Aims:
• Spatial and temporal data collection
• Customer behaviour analysis
• Routes optimisation for food delivery, taxi, etc.
• Estimations of arrivals and pricing
https://cloud.google.com/customers/go-jek/
CRICOS code 00025B 51
Examples of Cloud-based Applications – VI
Challenges:
• Manage over 1 mil drivers, or hundreds of thousands of active drivers concurrently online
• Manage over 300,000 merchants (restaurants, private sellers) for food deliver
• Ping each driver and customer every 10 seconds across the whole country – 6 million pings per minute and 8 billion pings per day
• Deal with customer interaction – generate about 4TB to 5TB of data every day
• Dynamically match the right driver with the right request (deliver person or food from A to B)
Solutions:
• Google Maps Platform: core components in the framework to find out optimised routes and estimated times of
arrival
• Big Data Package: Cloud Dataflow, Cloud Bigtable, and BigQuery form the basis of the company’s platform. Results:
• Supports 1 million motorcycle drivers with rapid access to riders and optimised routes
• Enables demand forecasting and pricing adjustments
• Positions business for international expansion
CRICOS code 00025B 52
Products and Services under Google Cloud
There are over 90 products in nine categories:
Compute
Storage & Databases
Networking
Big Data
Cloud AI
Manageme nt Tools
Identity & Security
API Platform
App Engine
Cloud Storage
VPC – Virtual Private Cloud
BigQuery
Cloud AutoML
Stackdriver
Cloud Identity
Maps Platform
Compute Engine
Cloud SQL
Cloud Load Balancing
Cloud Dataflow
Cloud TPU
Cloud Deployment Manager
Cloud IAM
Apigee API Platform
Kubernetes Engine (GKE)
Cloud BigTable
Cloud Armor
Cloud Dataproc
Cloud Machine Learning Engine
Cloud Console
Cloud Identity- Aware Proxy
API Monetizati on
https://cloud.google.com/products/
CRICOS code 00025B 53
Amazon Web Service v.s. Google Cloud Platform
Category
Compute
Network
IaaS
Containers
Service
Serverless Functions
Managed Batch Computing
Virtual Networks
Load Balancer
Dedicated Interconnect
Domains and DNS
CDN
AWS
Amazon Elastic Compute Cloud
PaaS
AWS Elastic Beanstalk
Amazon Elastic Container Service
AWS Lambda
AWS Batch
Amazon Virtual Private Cloud
Elastic Load Balancer
Direct Connect
Amazon Route 53
Amazon CloudFront
Compute Engine
App Engine
Google Kubernetes Engine
Cloud Functions
N/A
Virtual Private Cloud
Cloud Load Balancing
Cloud Interconnect
Google Domains, Cloud DNS
Cloud CDN
CRICOS code 00025B 54
GCP
Amazon Web Service v.s. Google Cloud Platform
Category
Storage
Database
Object Storage
Block Storage
Reduced
Storage
Archival Storage
File Storage
RDBMS
Service
-availability
Amazon S3 Standard
AWS
Amazon Simple Storage Service
Amazon Elastic Block Store
–
Infrequent
Cloud Storage
Persistent Disk
GCP
Access, Amazon S3 One Zone
Infrequent Access
Amazon Relational Database
–
Amazon Glacier
Amazon Elastic File System
Cloud Storage Nearline
Cloud Storage Coldline
Cloud Filestore (beta)
Service, Amazon Aurora
Cloud SQL,
Cloud
NoSQL: Key-value
NoSQL: Indexed
Block Storage
Amazon DynamoDB
Amazon SimpleDB
Amazon Elastic Block Store
Spanner
Cloud Firestore, Cloud Bigtable
Cloud Firestore
Persistent Disk
CRICOS code 00025B 55
Amazon Web Service v.s. Google Cloud Platform
Category
Big Data & Analytics
Application Services
Managemen t Services
Service
Batch Data Processing
Workflow Orchestration
Messaging
Monitoring
Logging
Deployment
Batch
AWS
Amazon Elastic MapReduce, AWS
Amazon Data Pipeline, AWS Glue
Amazon Simple Notification Service, Amazon Simple Queueing Service
Amazon CloudWatch
Amazon CloudWatch Logs
AWS CloudFormation
Cloud
Dataflow
BigQuery
GCP
Dataproc
Cloud Composer
Cloud Pub/Sub
Stackdriver Monitoring
Stackdriver Logging
Cloud Deployment Manager
, Cloud
Stream Data Processing
Stream Data Ingest
Analytics
Amazon Kinesis
Amazon Kinesis
Amazon Redshift, Amazon Athena
Cloud Dataflow
Cloud Pub/Sub
CRICOS code 00025B 56
Amazon Web Service v.s. Google Cloud Platform
Category
Machine Learning
Speech
Vision
Service
Conversational Interface
Video Intelligence
Auto-generated Models
Fully Managed ML
Amazon Transcribe
Amazon Rekognition
Amazon WS
Amazon Rekognition Video
N/A
Amazon SageMaker
GCP
Cloud Speech-to-Text
Cloud Vision
Natural Language Processing
Translation
Amazon Comprehend
Amazon Translate
Cloud Natural Language
Cloud Translation
Dialogflow Enterprise Edition
Cloud Video Intelligence
Cloud AutoML (beta)
Cloud Machine Learning
Engine
CRICOS code 00025B 57
Reading Materials
1. “Cloud computing: concepts, technology & architecture”. Erl, Thomas, , and . , 2013.
2. , 5 Things You Should Know About Hybrid Cloud, (https://www.informationweek.com/cloud/5-things- you-should-know-about-hybrid-cloud/d/d-id/1331818?) InformationWeek, 2018-05-17.
3. Hamdaqa, Mohammad. A Reference model for developing cloud applications (http://www.stargroup.uwaterloo.ca/~mhamdaqa/publications/A%20REFERENCEMODELFORDEVELOPINGCLOU D%20APPLICATIONS.pdf)
4. Amazon EC2 (http://aws.amazon.com/ec2/)
CRICOS code 00025B 58
Summary
• CloudDeliveryModels:SaaS,PaaS,andIaaS
• CloudDeployModels:Public/Private/Community/HybridCloud • Cloud-EnablingTechnologies
– Broadband Networks and Internet Architecture
– Virtualisation Technology (VT)
– Data Centre Technology
– Web Technology
– Multitenant Technology
• GoalsandBenefits
• RisksandChallenges
• Cloud-basedApplicationsintheWorld
CRICOS code 00025B 59
Tutorial & Practical for Week 2
Tutorial 1 (Week 2)
1. What is the NIST (National Institute of Standards and Technology) definition of Cloud Computing?
2. What are the six essential characteristics of Cloud Computing? Moreover, for each characteristic, please make a brief introduction.
3. What are the four deployment models in cloud computing? Please read these reading materials and answer this question in detail.
4. Discuss the differences between cloud delivery models and make examples of delivery models.
5. Play with GCP Pricing Calculator (https://cloud.google.com/products/calculator) and check how much the standard machine n1-standard-2 (no SSD, no static and no public IP address, running 24*7) costs you per month? Discuss how you can save credit on top of that VM specs.
Practical 1 (Week 2)
1. Redeem credit on GCP and install a VM on GCP
2. Install Web server on VM.
3. Linux Prac I
CRICOS code 00025B 60