Introduction
The University of Leeds School of Computing
Deadline: Monday 9 March 2020, 10:00
COMP5850M Coursework 1 – 2019/2020
Congratulations! You have been offered a job at Clouds4U, a company specialising in providing IaaS. Its customers are interested in running Virtual Machines (VM) on the cloud.
This exercise aims to give you some practical experience of using a Cloud Virtual Infrastructure Manager (Open- Nebula) and Java OpenNebula Cloud API (OCA). It will also give you the opportunity to research on the cloud computing domain of trade-offs between performance and energy efficiency.
You should review the lecture notes on these topics carefully before you start work.
Useful Resources
Useful resources are available on COMP5850M module area on Minerva: https://minerva.leeds.ac.uk
Part 1: Java OpenNebula Cloud API (OCA)
You are provided with the bulk of the code that demonstrates a basic Open- Nebula Cloud API functionality for a VM. The client program will provide some basic information such as a VM template and get various bits of tasks performed, e.g. instantiate a VM. The source code supplied to you is not complete and therefore needs completion. The work required is broken down into a series of small steps that you should follow, in the order shown, to complete the exercise.
Examine the file VMachineSample.java and try to get an overall feel for what the software is doing (or supposed to do). You should focus on the OpenNebula/client interaction aspects. By looking at the file, you should be able to see that the implementation is incomplete.
Complete the implementation of this file. You should focus on the following aspects:
• VM template: specify a template of your choice. Note: Feel free to use the template provided in the code. • the information OpenNebula provides about the VM, e.g. name of host where it is running;
• the time is takes to instantiate the VM;
• the time it takes to delete the VM.
Compile the program. Run it and record its output.
Note: You are expected to run the experiments n times (e.g. n = 5) because of the inherent dynamic nature of the cloud infrastructure. A statistical analysis (average, standard deviation) is therefore expected. Compare the results of the various executions and discuss any discrepancies in performance.
Part 2: VM Migration
You are told VM migration in clouds brings multiple benefits such as higher performance, improved manageability and fault tolerance. There are a number of VM migration mechanisms you may consider, e.g. heuristics, power-aware, performance-aware and network-aware.
Your next task is to write a program that migrates an existing VM to a target host. Propose a design and discuss your assumptions and requirements to implement your solution. Record the time it takes to migrate the VM and discuss the results.
Part 3: Resource Scaling and Performance/Energy Consumption Trade-Off
You are told the company’s customers are interested in running various types of jobs such as data parallel processing (e.g. Hadoop) and compute intensive jobs (e.g. Message Passing Interface (MPI)) on their VMs. The company seeks to understand the trade-off between performance (job runtime) and power/energy consumption. The assumption is that while a service is running (Hadoop or MPI), the resources (CPU, memory, power and network) are monitored, e.g. thanks to a Zabbix-based monitoring infrastructure.
Your task is to evaluate the performance and power/energy consumption trade-off on the school of computing cloud testbed, by performing a range of experiments. The experiments will consider the scalability of Virtual Machines (VMs) and Physical Hosts (PHs) when consuming a resource intensive application. Attention should be given to the following VM allocation scenarios when considering a service running:
(a) the application runs on 1, 2 or up to 4 VMs
(b) the VMs can be allocated to 1, 2 or up to 4 PHs.
The experiments should reflect the various VMs/PHs allocation possibilities and be implemented on the cloud testbed where the monitoring infrastructure is provided. Feel free to consider using alternative monitoring solutions, e.g. collectd http://www.collectd.org.
The application. You should choose one of the following:
(a) The stress tool introduced in the lab. session. The CPU-bound process, the I/O bound process as well as the
memory are stressed on each VM instance. This is less challenging and marked out of 10.
(b) A Hadoop data parallel processing job and its associated input data requirements. This is more challenging
and marked out of 25.
(c) An MPI compute intensive job. This is more challenging and marked out of 25. You should consider one of
the following:
(i) a simple MPI example, e.g. QuickSort MPI version
(ii) a mini-app, e.g. Tealeaf, Hydro, Cloverleaf or NBody from the mini-app consortium, see: https://github.com/UK-MAC
(iii) other, e.g. Gromacs to perform molecular dynamics, see http://www.gromacs.org/.
Submission
• Use the document Report available in the coursework folder to report your written answers to the ques- tions. Submit the document through the link Report Submission. Microsoft Word and pdf file formats are acceptable (pdf is encouraged).
• To submit your code for the exercise, create a Zip or tar archive of the files which make up your system and submit the file through the link Code Submission. Remember to include a descriptive README file as well as any additional material which you think are useful. Ensure you include all files needed to work as part of your submission and that the code in relation to the various questions is in a separate folder.
Marking Scheme
Part 1 10 Part 2 15 Part 3 25
50
A more detailed marking scheme is available in the document Report. Weight: this coursework accounts for 20% of the assessment.