COMP90015 Distributed Systems
Introduction
Aaron of Computing and Information Systems © The University of Melbourne
2022 Semester II
Copyright By PowCoder代写 加微信 powcoder
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II 1/31
1 Subject administration
2 A Computer System Basis Physical Model
Process Model
3 Distributed Systems Definition
Motivation
Consequences
Case Studies
Commercial distributed systems
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II 2/31
Subject administration
• , Lecturer and Subject Coordinator,
• , Head Tutor
• -Khoury, Tutor
• , Tutor
• , Tutor
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II 3/31
Subject administration
Assessment
Projects will be group work with groups of size 2. You may work alone if you wish however you must complete the entire project. The programming language is Java (11). Please start considering your partner now. Project 1 details will be provided in Week 4.
1 Project 1, 20%, software and written report, starting around Week 4 and due around Week 8
2 Project 2, 20%, software and written report, starting around Week 8 and due around Week 12
3 Final Exam, 60%, online
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 4/31
Subject administration
Academic Integrity
All University of Melbourne students are expected to uphold academic
integrity in all aspects of every piece of work that they submit for
assessment.
• https://academicintegrity.unimelb.edu.au/
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 5/31
Subject administration
Traditional Course Overview
This course was originally developed from Coulouris, Dollimore and Kindberg, Distributed Systems: Concepts and Design, Edition 5, © Addison-Wesley 2012, with emphasis on the following chapters:
Chapter 1 Characterization of Distributed Systems Chapter 2 System Models
Chapter 4 Interprocess Communication
Chapter 5 Remote Invocation
Chapter 6 Indirect Communication
Chapter 7 Operating System Support Chapter 11 Security
Chapter 12 Distributed File Systems Chapter 13 Name Services
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II 6/31
Subject administration
2021 Semester 2 Course Overview
A tentative plan for the semester:
25th Jul. 1st Aug. 8th Aug.
15th Aug. 22nd Aug.
29th Aug. 5th Sep.
12th Sep. 19th Sep.
26th Sep. 3rd Oct. 10th Oct. 17th Oct.
Motivation, Challenges
IPC, Data Representation Architectures, Fundamental Models Synchronous/Asynchronous Protocols
Middleware, RPC, RMI
Indirect Communication Paradigms
Encrypting Communication Digital Signatures, Certificates, SSL/TLS
Distributed File Systems
Tutorial 1 Tutorial 2
Tutorial 3
Tutorial 4 Tutorial 5
Tutorial 6 Tutorial 7
Tutorial 8
Name Services TBA
Non-teaching Week
Tutorial 9 Tutorial 10 Tutorial 11
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II 7/31
Subject administration
Learning Outcomes
• Familiarity with distributed system terminology and fundamental concepts
• Develop critical thinking, reasoning, modeling and analysis techniques
• Know a range of solutions to essential distributed system challenges
• Understand high-level distributed system paradigms and demonstrate their appropriate application
• Develop skills in distributed system design and programming implementation
• Develop skills in small group work and written communication
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 8/31
A Computer System Basis Physical Model
Computer Hardware
In this subject we are mainly interested in distributed systems based on computer systems, with a broad applicability:
• Machine/Host/Device/Node: terminology used somewhat synonomously depending on context; “machine room”, “server host”, “edge device”, “cluster node”, … even we use “thing” as in “Internet of Things”
• Physical location: typically stationary (desktop) or mobile (phone or tablet), but technically not much difference – we can move our desktop if we want.
• Hardware devices/specifications:
• CPUs/cores (clock rate, cache), RAM (capacity, latency)
• Connectivity: ethernet, wireless, cellular, bluetooth, USB (bitrate/bandwidth)
• Storage: SSD, HDD (capacity, latency, throughput, read/write iops)
• Graphics: GPU (resolution, refresh rate)
• Peripheral devices: keyboard, monitor, mouse, printer, webcam, microphone, etc
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 9/31
A Computer System Basis Physical Model
Computer Software
We are mainly interested in the higher layers of software, from the Application/User software down to the Operating System.
• Application/User software
• Middleware/Libraries – Application Programming Interfaces (APIs)
• Operating System (OS):
• Kernel (monolithic/micro kernel)
• Device drivers
• Modules/services
• Hypervisor (for virtualization)
• Bootloaders, EFI (Extensible Firmware Interface) BIOS (Basic Input Output
• ROM (Read Only Memory) and firmware, POST (Power On Self Test)
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 10/31
A Computer System Basis Physical Model
A platform is a layer of functionality or API across a number of machines, by de facto being a consistent hardware and OS specification combined, but sometimes being a middleware layer above the OS primarily to hide differences in OS and hardware on different machines.
• POSIX or Portable Operating System Interface is the standard OS API adopted by most Unix-based OSes. It was defined in 1988 by the IEEE Computer Society.
• The macOS from . adopted POSIX around 2007 when their OS became UNIX-based.
• The Linux OS, Android and a range of other popular OSes for devices such as routers are “mostly” POSIX-compliant.
• Cygwin/MinGW provide a mostly POSIX-compliant development and runtime environment for Microsoft Windows. Conversely, Wine is a compatibility layer for UNIX-like operating systems to run programs written for Microsoft Windows.
• The Windows API or WinAPI is the API for the Microsoft Windows OS, from Microsoft Corporation.
• The Java Virtual Machine executes programs, usually written in Java, which compile to Java bytecode. The JVM provides a consistent API across many different Hardware/OS platforms, which is one aspect which lead to the popularity of the Java programming language.
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 11/31
A Computer System Basis Physical Model Virtualization and Emulation
Even commodity OS and computer hardware provides feature rich virtualization and emulation support, that can be of interest when studying distributed systems. However the details are often quite technical and OS specific.
• Virtualization: managing access to existing physical devices such that there appears to be more of them or greater capacity from a software perspective (i.e. logically), essentially time sharing:
• Virtual memory: paging to disk
• Virtual Cores: time sharing physical cores
• OS Virtualization: Jails, Containers, Zones
• Virtual Machine (VM): all devices are virtualized,
• Hypervisors
• runanOSintheVM
• Emulation: create the appearance of a device existing, even though it does not:
• Mobile device emulation for testing and debugging on hardware that is not physically available
• Network device emulation: switches and routers (Mininet)
• OS emulation
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 12/31
A Computer System Basis Physical Model
Computer Networks I
Besides the computer system, the computer network is an essential aspect for our study of distributed systems:
• We are almost exclusively interested in networks that support the TCP/UDP/IP protocol stack, although sometimes we use other network protocols specific to a given technology:
• Bluetooth: largely for removing peripheral cables, Service Discovery Protocol (SDP)
• Cellular Networks: mixing traditional telecommunications functionality like SMS with
Internet-based applications
• Relevant networking devices include:
• Wireless Base Stations: provide a way to connect to the network using radio waves anywhere from 900MHz to 60GHz, sometimes called WLAN or WiFi.
• Switches and Routers: provide a way to connect to the network using cables, e.g. Ethernet, at least 1Gbps but also at 10, 25 and 40Gbps.
• The well known Internet, arising in the 1980’s, has become the dominant global computer network and communications platform:
• Internet Service Providers (ISPs) provide access which among other things provides one or more unique Internet Addresses:
• represent end-points on the public Internet,
• devices with public Internet addresses can be directly communicated with by every other device
connected to the Internet.
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 13/31
A Computer System Basis Physical Model Computer Networks II
• No one country owns or controls the entire Internet, although some governing bodies such as ICANN (The Internet Corporation for Assigned Names and Numbers) have traditionally controlled some essential aspects, namely the Internet Addresses and Internet Domain Names.
• There are lessor known alternatives arising to ICANN’s governance, e.g. Yeti DNS (Chinese government sponsored) and National DNS (Russian government sponsored).
• Organizations can build private networks that operate in the same way as the Internet but that are entirely owned and governed by the organization:
• Private set of network addresses that are unique only to the organization’s private network.
• Local Area Networks (LANs): the smallest unit of network management where typically
all of the machines on the LAN are subject to the same network policies.
• Wide Area Networks (WANs): multiple LANs connected together, providing for differing
network policies across an organization.
• Network Address Translation (NAT): provide an ability for a private network to connect to
the public Internet via a machine (typically a router) that has both a public Internet
Address and a private network address.
• Virtual Private Network (VPN): connecting two or more private networks via a secure
connection over the public Internet .
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 14/31
A Computer System Basis Process Model Single process, single thread
• Smallest OS encapsulation:
• Every application requires at least one process
• Primarily encapusalates processor and memory resources
• System calls to access devices on the machine
• One machine • Single thread:
• one core
• one address space
• no concurrency control
Not a distributed system.
thread machine
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II
A Computer System Basis Process Model Single process, multiple threads
• One machine, one process • Multiple threads:
• multiple cores
• one address space shared among threads
• concurrency control required
Not a distributed system – limited to a single machine.
threads machine
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 16/31
A Computer System Basis Process Model Multiple processes, single/multi-threaded
• One or more machines – each process can run on a different machine
• Each process has its own address space
• Inter-process communication (IPC) is required:
• Shared memory, pipes and file-based communication if processes on the same machine
• TCP/UDP/IP for processes on the same or different machines • Each process may have one or more threads:
• One or more cores per process
• Concurrency control within processes and across processes
Now do we have a distributed system?
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II
Distributed Systems Definition What is a Distributed System?
Distributed System definitions – many and varying:
• “A system in which hardware or software components located at networked computers communicate and coordinate their actions only by passing messages” [Coulouris]
• “A collection of independent computers that appears to its users as a single coherent system” [Tanenbaum]
Key aspects:
• a number of components
• communication between the components (implied?)
• synergy; achieve more than the simple sum of individual components
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 18/31
Distributed Systems Definition
Discussion questions
Question (1): If we have a number of floor cleaning robots, that never communicate, but that independently go through a process of cleaning the same floor at the same time (with some smarts like not bumping in to each other, and detecting if some part of the floor needs to be cleaned or can be skipped over), in such a way that they end up collectively cleaning the floor faster than a single independent robot could do so, is this an example of a distributed system? Why or why not?
Question (2): Multiple threads can be seen as working together to solve a problem. So why don’t we call a multi-threaded, single process application as a distributed system?
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 19/31
Distributed Systems Motivation Why would we build a Distributed System? I
Lots of reasons, many of which you would take for granted in today’s world:
• Communication:
• users at different computers can communicate with each other
• multiple users can communicate with each other concurrently: online chat, video
conference
• Remote access:
• user does not need to travel to a given computer to use it, but can access it remotely
• can use resources on the other side of the Earth, can control a rover on Mars (if it lands
successfully)
• Resource sharing:
• single resource can be shared among multiple users: significant cost saving and increased
utilization
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 20/31
Distributed Systems Motivation Why would we build a Distributed System? II
• Reliability:
• increased mean-time-to-failure, redundancy in resources allows the system to continue to
operate in the presence of hardware and software failures
• Availability:
• less downtime, e.g. when upgrading some hardware the system can stay online on
remaining hardware
• Scalability:
• using more resources to provide greater capacity than any single resource can
• ideally having N resources provides N times the capacity of a single resource
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 21/31
Distributed Systems Motivation
API, protocols IPC
Application
logic & user interface
Why transform an existing single process application into a distributed application?
App Server logic
App Client user
what bene ts do you see?
remote access
redundancy
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne)
2022 Semester II 22/31
Distributed Systems Consequences Consequences of Distributed Systems I
The benefits that distributed systems bring are not free:
• Communication complexity. Communication is not free, and is largely an overhead. Processes may waste time waiting for communication to take place. Communication requires additional power consumption. Communication errors can occur, sometimes causing communication to be impossible for significant periods of time.
• Concurrency. In a Distributed System computers perform their tasks autonomously and communicate with other computers via message passing when needed. Services provided by distributed systems will be accessed by multiple users simultaneously. Distributed system design should take this into consideration and implement appropriate techniques for handling concurrency.
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 23/31
Distributed Systems Consequences Consequences of Distributed Systems II
• No global clock. Clocks on individual computers operate independently. Since computers communicate by message passing there are limits to the accuracy with which the computers in a network can synchronize their clocks. Therefore, distributed systems have to deal with the fact of not having a global clock and implement time synchronization techniques when needed.
• Independent failures. Some components of the system may fail while the others are still running. Failures of participating computers of the system is not known to others immediately.
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 24/31
Distributed Systems Consequences Interprocess Communication
APIs, protocols and semantics
IPC significantly increases the design and implementation effort over a single process application. IPC generally requires:
• an API which defines the available operations supported by the communicating processes, e.g. a distributed Calendar App:
• GetRemoteCalendar(address:InetAddress) : RemoteCalendarRef
• RetrieveCalendarData(cal:RemoteCalendarRef, date:Date) : Event[] • StoreCalendarData(cal:RemoteCalendarRef,event:Event)
• communication semantics that define what the programmer can expect, e.g.:
• “all API calls block until the operation has successfully completed exactly once, or else an
exception is thrown in which case the operation outcome is undefined”
• a communication protocol which defines how the processes will initiate communication and exchange data to implement the API and guarantee the semantics, e.g.:
• Transmission Control Protocol (TCP) connections and JSON data objects
(School of Computing and InformaCtiOonMSPy9s0te0m15s ©DistTrihbeutUednivSeyrsstietymosf Melbourne) 2022 Semester II 25/31
Distributed Systems Consequences Distributed system challenges I
• Heterogeneity – the parts of the system are not consistent (homogenous)
• Hardware, OS and networking can differ across different components.
• Processes can be written in different languages, and by different developers.
• Openness – the system can be built upon or accessed by third-party developers, through public APIs and protocols
• When APIs and protocols need to change they can break third-party systems.
• APIs should be simple to use, provide all the
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com