OS Support for Building Distributed Applications: Multithreaded Programming using Java Threads
Dr. Rajkumar Buyya
Cloud Computing and Distributed Systems (CLOUDS) Laboratory School of Computing and Information Systems
The University of Melbourne, Australia http://www.buyya.com
1
Outline
◼ Introduction to Middleware ◼ Thread Applications
◼ Defining Threads
◼ Java Threads and States
◼ Architecture of Multithreaded servers ◼ Threads Synchronization
◼ Summary
2
Introduction
◼ Middleware is a layer of software (system) between Applications and Operating System (OS) powering the nodes of a distributed system.
◼ The OS facilitates:
◼ Encapsulation and protection of
resources inside servers;
◼ Invocation of mechanisms required to access those resources including concurrent access/processing.
Applications
Middleware
Distributed Nodes with OS
3
Middleware and Network Operating System (NOS)
◼ Many DOS (Distributed OS) have been investigated, but there are none in general/wide use. But NOS are in wide use for various reasons both technical and non- technical.
◼ Users have much invested in their application software; they will not adopt a new OS that will not run their applications.
◼ Users tend to prefer to have a degree of autonomy of their machines, even in a closely knit organisation.
◼ A combination of middleware and NOSs provides an acceptable balance between the requirement of autonomy and network transparency.
◼ NOS allows users to run their favorite word processor.
◼ Middleware enables users to take advantage of services that become available in their distributed systems.
4
Introducing a middleware
◼ Building Distributed Systems
◼ DOS or NOS are not enough to build a DS!
◼ NOS are a good starting point but ….
◼ … we need an additional layer “gluing” all together
Middleware NOS
Building Distributed Systems
◼ Middleware
◼ High-level features for DS ◼ Communication
◼ Management
◼ Application specific
◼ Uniform layer where to build DS services ◼ Runtime environment of applications
◼ Operating System
◼ Low / medium level (core) features ◼ Process / threads management
◼ Local hardware (CPU, disk, memory)
◼ Security (users, groups, domain, ACLs) ◼ Basic networking
Operating system layers and Middleware
Applications, services
Middleware
OS1 Processes, threads,
communication, …
OS2 Processes, threads,
communication, …
Computer & network hardware
Computer & network hardware
OS:kernel, libraries& servers
Pl at fo rm
Node 1
Node 2
◼ Unix and Windows are two examples of Network Operating Systems – have a networking capability built into them and so can be used to access remote resources using basic services such as rlogin, telnet.
7
Core OS components and functionality
Process manager
Thread manager
Communication manager
Memory manager
Supervisor
8
Threaded Applications
◼ Modern Applications and Systems ◼ Operating System Level
◼ Multitasking: multiple applications running at once
◼ Application Level
◼ Multithreading: multiple operations performed at
the same time within an application.
◼ Bottom Line:
◼ Illusion of concurrency
Threaded Applications
◼ Modern Systems
◼ Multiple applications run concurrently!
◼ This means that… there are multiple processes on your computer
web & email
games office automation
multimedia
pictures
Multitasking
A single threaded program
class ABC
{
….
}
}
public void main(..) {
…
..
begin
body end
11
Threaded Applications
◼ Modern Systems
◼ Applications perform many tasks at once!
◼ This means that… there are multiple threads within a single process.
Background printing
GUI rendering
Application core logic
A Multithreaded Program
Main Thread
start
start
start
Thread A
Thread B
Thread C
Threads may switch or exchange data/results
13
Single and Multithreaded Processes
threads are light-weight processes within a process
Single-threaded Process
Execution
Multi-threaded Process Threads of
Single instruction stream
Multiple instruction stream Address Space
Common
14
Multithreaded Server: For Serving Multiple Clients Concurrently
◼ Modern Applications
◼ Example: Multithreaded Web Server
Web/FTP server
Client 1
Client 2
Process Request Client 1
Process Request Client 2
Process Request Client N
Client N
Threaded Applications
◼ Modern Applications
◼ Example: Internet Browser + Youtube
Video Streaming
Favorities, Share, Comments Posting
Modern Applications need Threads (ex1): Editing and Printing documents in background.
Editing Thread
Printing Thread
17
Multithreaded/Parallel File Copy
reader() {
——— –
lock(buff[i]);
read(src,buff[i]);
unlock(buff[i]);
——— –
}
writer() {
———- lock(buff[i]); write(src,buff[i]); unlock(buff[i]); ———-
}
buff[0]
buff[1]
Cooperative Parallel Synchronized Threads
18
Defining Threads
◼ Applications – Threads are used to perform:
◼ Parallelism and concurrent execution of independent tasks / operations.
◼ Implementation of reactive user interfaces.
◼ Non blocking I/O operations.
◼ Asynchronous behavior.
◼ Timer and alarms implementation.
Defining Threads
◼ Example: Web/FTP Server
while
{
Main Thread
Worker Thread
Worker Thread
}
Web/FTP server
Worker Thread
Execution Timeline
Defining Threads
◼ A Thread is a piece of code that runs in concurrent with other threads.
◼ Each thread is a statically ordered sequence of instructions.
◼ Threads are used to express concurrency on both single and multiprocessors machines.
◼ Programming a task having multiple threads of control – Multithreading or Multithreaded Programming.
Java Threads
◼ Java has built in support for Multithreading
◼ Synchronization
◼ Thread Scheduling
◼ Inter-Thread Communication: ◼ currentThread start
◼ yield run
◼ sleep stop
◼ resume
setPriority getPriority suspend
◼ Java Garbage Collector is a low-priority thread.
22
Threading Mechanisms…
◼ Create a class that extends the Thread class ◼ Create a class that implements the Runnable
interface
Thread
(objects are threads)
[a] [b]
Runnable
(objects with run() body)
Thread
MyThread
MyClass
23
1st method: Extending Thread class
run() method:
class MyThread extends Thread
{
public void run()
{
// thread body of execution
} }
◼ Create a thread:
MyThread thr1 = new MyThread();
◼ Start Execution of threads: thr1.start();
◼ Create and Execute:
new MyThread().start();
◼ Create a class by extending Thread class and override
24
An example
class MyThread extends Thread { public void run() {
System.out.println(” this thread is running … “); }
}
class ThreadEx1 {
public static void main(String [] args ) {
MyThread t = new MyThread(); t.start();
} }
25
2nd method: Threads by implementing Runnable interface
◼ Create a class that implements the interface Runnable and override run() method:
class MyThread implements Runnable {
…..
public void run()
{
// thread body of execution
}
}
◼ Creating Object:
MyThread myObject = new MyThread();
◼ CreatingThreadObject:
Thread thr1 = new Thread( myObject );
◼ StartExecution: thr1.start();
26
An example
class MyThread implements Runnable { public void run() {
System.out.println(” this thread is running … “); }
}
class ThreadEx2 {
public static void main(String [] args ) {
Thread t = new Thread(new MyThread()); t.start();
} }
27
Life Cycle of Thread
new
start()
ready
I/O completed
resume()
sleeping
notify()
waiting
wait()
Time expired/ interrupted
dispatch
blocked
sleep()
suspend()
Block on I/O
dead
running
completion
stop()
28
A Program with Three Java Threads
◼ Write a program that creates 3 threads
29
Three threads example
◼ class A extends Thread ◼{
◼ public void run() ◼{
◼ for(int i=1;i<=5;i++)
◼{
◼ System.out.println("\t From ThreadA: i= "+i); ◼}
◼ System.out.println("Exit from A");
◼}
◼}
◼ class B extends Thread ◼{
◼ public void run() ◼{
◼ for(int j=1;j<=5;j++)
◼{
◼ System.out.println("\t From ThreadB: j= "+j); ◼}
◼ System.out.println("Exit from B");
◼}
◼}
30
Three threads example
◼ class C extends Thread ◼{
◼ public void run() ◼{
◼ for(int k=1;k<=5;k++)
◼{
◼ System.out.println("\t From ThreadC: k= "+k); ◼}
◼ System.out.println("Exit from C"); ◼}
◼}
◼ class ThreadTest
◼{
◼ public static void main(String args[]) ◼{
◼ new A().start();
◼ new B().start();
◼ new C().start(); ◼}
◼}
31
Run 1
◼ [raj@mundroo] threads [1:76] java ThreadTest From ThreadA: i= 1
From ThreadA: i= 2 From ThreadA: i= 3 From ThreadA: i= 4 From ThreadA: i= 5
Exit from A
From ThreadC: k= 1
From ThreadC: k= 2 From ThreadC: k= 3 From ThreadC: k= 4 From ThreadC: k= 5
Exit from C
From ThreadB: j= 1
From ThreadB: j= 2 From ThreadB: j= 3 From ThreadB: j= 4 From ThreadB: j= 5
Exit from B
32
Run 2
◼ [raj@mundroo] threads [1:77] java ThreadTest From ThreadA: i= 1
From ThreadA: i= 2 From ThreadA: i= 3 From ThreadA: i= 4 From ThreadA: i= 5 From ThreadC: k= 1 From ThreadC: k= 2 From ThreadC: k= 3 From ThreadC: k= 4 From ThreadC: k= 5
Exit from C
From ThreadB: j= 1
From ThreadB: j= 2 From ThreadB: j= 3 From ThreadB: j= 4 From ThreadB: j= 5
Exit from B Exit from A
33
Thread Priority
◼ In Java, each thread is assigned priority, which affects the order in which it is scheduled for running. The threads so far had same default priority (NORM_PRIORITY) and they are served using FCFS policy.
◼ Java allows users to change priority:
◼ ThreadName.setPriority(intNumber) ◼ MIN_PRIORITY = 1
◼ NORM_PRIORITY=5
◼ MAX_PRIORITY=10
34
Thread Priority Example
class A extends Thread
{
public void run()
{
System.out.println("Thread A started"); for(int i=1;i<=4;i++)
{
System.out.println("\t From ThreadA: i= "+i);
}
System.out.println("Exit from A");
} }
class B extends Thread
{
public void run()
{
System.out.println("Thread B started"); for(int j=1;j<=4;j++)
{
System.out.println("\t From ThreadB: j= "+j);
}
System.out.println("Exit from B");
} }
35
Thread Priority Example
class C extends Thread
{
public void run()
{
System.out.println("Thread C started"); for(int k=1;k<=4;k++)
{
System.out.println("\t From ThreadC: k= "+k);
}
System.out.println("Exit from C");
} }
class ThreadPriority
{
public static void main(String args[])
{
A threadA=new A();
B threadB=new B();
C threadC=new C();
threadC.setPriority(Thread.MAX_PRIORITY); threadB.setPriority(threadA.getPriority()+1); threadA.setPriority(Thread.MIN_PRIORITY); System.out.println("Started Thread A");
threadA.start(); System.out.println("Started Thread B");
threadB.start(); System.out.println("Started Thread C");
threadC.start();
System.out.println("End of main thread"); }
}
36
Assignment 1 at a Glance: Multithreaded Dictionary Server – Using Sockets and Threads
A Client Program Meaning(“guru”)?
Meaning (“guru”) “master or teacher”
Multithreaded Dictionary Server
A Client Program Meaning(“love”)?
A Client Program in “C”
Meaning(“java”)?
A Client Program in “C++”
Meaning(“channel”)?
37
Accessing Shared Resources
◼ Applications access to shared resources need to be coordinated.
◼ Printer (two person jobs cannot be printed at the same time)
◼ Simultaneous operations on your bank account.
◼ Can the following operations be done at the same time on the same account?
◼ Deposit()
◼ Withdraw() ◼ Enquire()
39
Online Bank: Serving Many Customers and Operations
Internet Bank Server
Client 1
Bank Operator 1 Bank Local Area Network
Client 2
Bank Operator M
Client N
Bank Database
Shared Resources
◼ If one thread tries to read the data and other thread tries to update the same data, it leads to inconsistent state.
◼ This can be prevented by synchronising access to the data.
◼ Use “synchronized” method:
◼ public synchronized void update()
◼{ ◼...
◼}
41
the driver: 3 Threads sharing the same object
class InternetBankingSystem {
public static void main(String [] args ) {
Account accountObject = new Account ();
Thread t1 = new Thread(new MyThread(accountObject)); Thread t2 = new Thread(new YourThread(accountObject)); Thread t3 = new Thread(new HerThread(accountObject));
t1.start(); t2.start(); t3.start();
// DO some other operation } // end main()
}
42
Shared account object between 3 threads
class MyThread implements Runnable { Account account;
public MyThread (Account s) { account = s;}
public void run() { account.deposit(); } } // end class MyThread
class YourThread implements Runnable { Account account;
public YourThread (Account s) { account = s;}
public void run() { account.withdraw(); } } // end class YourThread
class HerThread implements Runnable { Account account;
public HerThread (Account s) { account = s; }
public void run() {account.enquire(); } } // end class HerThread
account (shared object)
43
Monitor (shared object access): serializes operation on shared objects
class Account { // the 'monitor' int balance;
// if 'synchronized' is removed, the outcome is unpredictable public synchronized void deposit( ) {
// METHOD BODY : balance += deposit_amount; }
public synchronized void withdraw( ) {
// METHOD BODY: balance -= deposit_amount; }
public synchronized void enquire( ) {
// METHOD BODY: display balance. }
}
44
Architecture for Multithread Servers
◼ Multithreading enables servers to maximize their throughput, measured as the number of requests processed per second.
◼ Threads may need to treat requests with varying priorities:
◼ A corporate server could prioritize request processing according to class of customers.
◼ Architectures:
◼ Worker pool
◼ Thread-per-request
◼ Thread-per-connection ◼ Thread-per-object
45
Client and server with threads (worker-pool architecture)
Thread 2 makes
requests to server
Receipt & queuing
Requests
Input-output
Thread 1
generates results
T1
Client
N threads Server
◼ In worker-pool architectures, the server creates a fixed pool of worker threads to process requests.
◼ The module “receipt and queuing” receives requests from sockets/ports and places them on a shared request queue for retrieval by the workers.
46
Alternative server threading architectures
workers
remote objects
per-connection threads
remote objects
per-object threads
I/O
I/O
remote objects
a. T hread-per-request
b. T hread-per-connection
c. Thread-per-object
Server associates a Thread with each connection and destroys when client closes the connection. Client may make many requests over the connection.
Associates Thread with each object. An IO thread receives request and queues them for workers, but this time there is a per-object queue.
IO Thread creates a new worker thread for each request and worker thread destroys itself after serving the request.
47
Invocations between address spaces
(a) System call
Thread
Control transfer via trap instruction
Control transfer via privileged instructions
User (b) RPC/RMI (wi thin one computer)
Thread 1
User 1
(c) RPC/RMI (between computers)
Thread 1 User 1
Kernel
Kernel
Protection domain boundary
Thread 2 User 2
Ne t w o rk
Thread 2 User 2
Kernel 1
Kernel 2
49
Summary
◼ Operating system provides various types of facilities to support middleware for distributed system:
◼ encapsulation, protection, and concurrent access and management of node resources.
◼ Multithreadingenablesserverstomaximizetheirthroughput, measured as the number of requests processed per second.
◼ Threads support treating of requests with varying priorities.
◼ Various types of architectures can be used in concurrent
processing:
◼ Workerpool
◼ Thread-per-request
◼ Thread-per-connection ◼ Thread-per-object
◼ Threadsneedtobesynchronizedwhenaccessingand manipulating shared resources.
◼ New OS designs provide flexibility in terms of separating mechanisms from policies.
52
References
◼ CDK Book (Text Book)
◼ Chapter 7 – “Operating System Support”
◼ Chapter 14: Multithread Programming
◼ R. Buyya, S. Selvi, X. Chu, “Object Oriented Programming with Java: Essentials and Applications”, McGraw Hill, New Delhi, India, 2009.
53