Cluster and Cloud Computing – Lecture 5 Cloud Computing & Getting to Grips with the University of Melbourne Research Cloud
Professor . Sinnott &
Copyright By PowCoder代写 加微信 powcoder
• Cloud benefits
– Cloud marketing!?
• The various flavours of cloud computing – Introduction to #aaS?
– Demonstration of University of Melbourne Research Cloud
Source: http://dilbert.com/strips/comic/2012-05-25/ 2
Life before cloud computing
source: http://www.rackspace.com/knowledge_center/whitepaper/revolution-not-evolution-how-cloud-computing-differs-from-traditional-it-and-why-it
Life before cloud computing…ctd
source: Ambrust et al, doi:10.1145/1721654.1721672
Cloud-busting? (~2009)
• To buy or not to buy that is the question…?
– ScotGrid (www.scotgrid.ac.uk) • Cost £550k ~end 2006
– Offered ~1000 cores and 85Tb attached storage and 9.5Tb internal cluster storage
– Ran at ~90% for 3 years • Updated November 2008
– 3000 cores and 480Tb storage
Cloud-busting? (~2009)
1.7Gb memory/160GB disk
7.5Gb memory/850GB disk
15Gb memory/1690GB disk
• $1=£0.69 (back then!) – £0.30*6,700,676 CPU hours
= £2,010,202 for just compute on-demand + data + networking + …? • Now…???
– https://aws.amazon.com/ec2/pricing/
Cloud Computing: A Definition
• NIST definition: “Cloud computing is a model for
enabling ubiquitous, convenient, on-demand
network access to a shared pool of configurable
computing resources (e.g., networks, servers,
storage, applications, and services) that can be
rapidly provisioned and released with minimal
management effort or service provider interaction.” » National Institute of Standards and Technology
(http://dx.doi.org/10.6028/NIST.SP.800-145)
• Focus of today is to get you up and running on the Cloud and explore the technologies related to the underlined
– Later lecture will do compare/contrast with AWS
• …and then how do Clouds actually work (hypervisors etc)
Organizing the Clouds
Testing-as-a-Service Management/Governance-as-a-Service
Application-as-a-Service Process-as-a-Service
Information-as-a-Service Database-as-a-Service
Storage-as-a-Service Infrastructure-as-a-Service
Platform-as-a-Service
Integration-as-a-Service Security-as-a-Service
The Most Common Cloud Models
Deployment Models
Delivery Models
Essential Characteristics
Software as a Service (SaaS)
Platform as a Service (PaaS)
Infrastructure as a Service (IaaS)
• On-demand self-service
• Broad network access
• Resource pooling
• Rapid elasticity
• Measured service
Public Clouds
– Utility computing
– Can focus on core business
– Cost-effective
– “Right-sizing”
– Democratisation of computing
– Security
– Loss of control
– Possible lock-in
– Dependency of Cloud provider continued existence
Private Clouds
– Consolidation of resources – Easier to secure
– More trust
– Relevance to core business? • e.g. Netflix moved to Amazon
– Staff/management overheads
– Hardware obsolescence
– Over/under utilisation challenges
Hybrid Clouds
• Examples
– Eucalyptus, VMWare vCloud Hybrid Service
– Cloud-bursting
• Use private cloud, but burst into public cloud when needed
• How do you move data/resources when needed?
• How to decide (in real time?) what data can go to public cloud?
• Is the public cloud compliant with PCI-DSS (Payment Card Industry – Data Security Standard)?
Delivery Models
source: http://www.businessinsider.com.au/10-most-important-in-cloud-computing-2013-4?op=1#a-word-about-clouds-1
Public SaaS examples
• Sharepoint
• Salesforce.com CRM
• Microsoft Office 365
• Some definitions include those that do not require payment, e.g. ad-supported sites
Public PaaS Examples
Cloud Name
Language and Developer Tools
Programming Models Supported by Provider
Target Applications and Storage Options
Google App Engine
Python, Java, Go, PHP + JVM languages (scala, groovy, jruby)
MapReduce, Web, DataStore, Storage and other APIs
Web applications and BigTable storage
Salesforce.com’s Force.com
Apex, Eclipsed- based IDE, web- based wizard
Workflow, excel- like formula, web programming
Business applications such as CRM
Microsoft Azure
.NET, Visual Studio, Azure tools
Unrestricted model
Enterprise and web apps
Amazon Elastic MapReduce
Hive, Pig, Java, Ruby etc.
Data processing and e-commerce
.NET, stand-alone SDK
Threads, task, MapReduce
.NET enterprise applications, HPC
Infrastructure As A Service (IaaS)
• Primary focus of this course…
• Many providers
– Amazon Web Services (Market leader)
• http://aws.amazon.com
– Oracle Public Cloud
• https://cloud.oracle.com/
– Rackspace Cloud
• www.rackspace.com
– CenturyLink, CloudSigma, DigitalOcean, DimensionData, GoGrid, Helio, Internap, Joyent, ProfitBricks, Verizon, …
– NeCTAR/Openstack Research Cloud • www.nectar.org.au
Cost Comparison
http://connect.cloudspectator.com/iaas-price-comparison-2015
NeCTAR Research Cloud
• National eResearch Collaboration Tools and Resources (NeCTAR – www.nectar.org.au)
– $50m+$10m+$10m+$72m… federal funding
– Lead by University of Melbourne – Had four key strands
• National Servers Program
• Research Cloud Program
– OpenStack IaaS
– 4Gb-64Gb (mostly Linux flavours)
– 30,000 physical servers available across different
availability zones
» Being upgraded continually!
• eResearch Tools Program
• Virtual Laboratories Program – Astro,
– Genomics,
– Humanities,
– Climate,
– …endocrine genomics
Data Infrastructures
• Research Data Services (RDS)
• $50m+$10m+$10m+$72m project to establish data storage resources across Australia
• ~100 Petabytes national data storage
– (VicNode)
• UniMelb, UniMonash for Vic-wide “nationally significant data sets”
– Used by many diverse communities
NeCTAR Research Cloud
• Based on OpenStack
– Open source cloud technology (more later lecture)
• Many associated/underpinning services
– Compute Service (code-named Nova)
– Image Service (code-named Glance)
– Block Storage Service (code named Cinder)
– Object Storage Service (code-named Swift)
– Security Management (code-named Keystone) – Orchestration Service (code-named Heat)
– Network Service (code-named Neutron)
– Metering Service (code-named Ceilometer) –…
https://docs.openstack.org/rocky/
NeCTAR Research Cloud Instance Types
Instance Type
Memory (GB)
Instance Storage (GB)
10GB + 1 x 30
10GB + 1 x 60
10GB + 1 x 120
10GB + 1 x 240
m1.xxlarge
10GB + 1 x 480
+250Gb Volume storage (more if required)
UniMelb/NeCTAR Demo (Crib Sheets LMS)
• NeCTAR (https://dashboard.rc.nectar.org.au)
• UniMelb (https://dashboard.cloud.unimelb.edu.au)
• Note that there are some differences – e.g. limited number of IP addresses
UniMelb/NeCTAR Demo (Crib Sheets LMS)
• Dashboard
– Overview (Resource allocation)
– Instances (Create, terminate and configure instances)
– Volumes (Create, terminate, attach and backup)
– Images (Create image, list image)
– Access & Security (Security groups, key pairs, API access) – Object store (Store data as an object)
• Launching a new VM
– Flavor (defines the compute, memory, and storage capacity) – Ephemeral disk (not persistence storage!!!)
– Create key pair
• ssh-keygen -t rsa -f cloud.key (Unix/Linux/MacOS, see Putty for Windows) • chmod 600 cloud.key (or get UNPROTECTED PRIVATE KEY FILE! error)
NeCTAR Demo (Crib Sheets LMS)
• Launching a new VM (continue) – Copy pub key (cloud.pub)
– Select key pair
– Select security group
– Availability zone
• Connecting to VM via SSH
– Private key (your secret key to keep secure)
– ssh -i
• Create a Volume
– Must be in the same availability zone as the instance
Note: you can’t create volumes until you have some (=assignment 2)!
NeCTAR Demo (Crib Sheets LMS)
• Attach a volume
– Check the device name: sudo fdisk -l
• sudo = runs commands with security privileges of another user – (by default: superuser)
– Create the mounting point: sudo mkdir /mnt/demo – Format the volume: sudo mkfs.ext4 /dev/vdb
– Mount the volume: sudo /dev/vdb /mnt/demo
– Check the result: df -h
• Installing an application – sudo apt-get install vim
• Create snapshots
– Snapshot for an instance – Snapshot for a volume
Note: if you don’t know Linux or what commands like df –h do, then google is your friend! See later too…
NeCTAR Demo (Crib Sheets LMS)
• Restore a snapshot
– Create an instance from an instance snapshot – Create a volume from a volume snapshot
• Setting up security groups
– Security groups act as a virtual firewall that controls the traffic for one or more instances
– It contains a set of security rules
– Default security group only allows SSH (from anywhere)
– Create a security group
– Create a rule
Note: be careful with security. Only open ports that are needed!
• CIDR (https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) – 128.250.0.0/16 (All IPs from UoM)
– x.x.x.x/32 (x.x.x.x only, use CIDR calculate for IP range calculation)
• Security Groups
Basic recipes to follow…
– ssh-keygen -t rsa -f cloud.key (Unix/Linux/MacOS, see Putty for Windows)
– chmod 600 cloud.key (or get UNPROTECTED PRIVATE KEY FILE! error)
– ssh -i cloud.key (Ubuntu)
– ssh -i cloud.key (Amazon Linux, RHEL)
– sudo fdisk -l
• sudo = runs commands with security privileges of another user (by default: superuser)
– sudo mkdir /mnt/demo
– sudo mkfs.ext4 /dev/vdb
• mkfs = make file system
• ext4 = type of file system (ext2, ext3, ext4)
– See http://www.thegeekstuff.com/2011/05/ext2-ext3-ext4/ for details on differences
– sudo mount /dev/vdb /mnt/demo
Basic recipes to follow…
• Install software
– sudo apt-get install vim
• apt-get installs/removes packages on Ubuntu installations • Common Shell Commands
– http://www.dummies.com/computers/operating-systems/linux/common- linux-commands/
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com