Independent thesis, 15 HE credits, for degree of Bachelor in
Computer Science
Spring Term 2016
Realization of multi-threaded model of
Node.JS on multi-core processors
Author： Ziteng You
School of Health and Society / School of Education and Environment [Arial 14p]
Author
Ziteng You
Title
Realization of multi-threaded model of Node.JS on multi-core processors
Supervisor
Nazila Hasanzade
Examiner
Dawit Mengistu
Abstract
The goal of research is to detect under multi-requests, multi-core multi-thread can improve
Node.JS performance and efficiency, reflected by response time and response per second. In
final testing result, we can find multi-core multi-thread system average response time less 50%
than single thread system and double rps(response per second) for it.
Node.JS is a single-threaded platform. It only supports single-core. As multi-core becomes
popular, the program should make use of multi-threaded computing to take full advantage of
multi-core CPU technology, letting Node.JS take full advantage of multi-core is imperative.
This research uses multi-core multi-thread design and normal single thread design respectively
to complete a same simulate taxi call system on Node.JS platform. The system simulated all
processes that user can call a taxi, take the taxi and paying when taxi arrived on the web site.
The server response for the user operation. The multi-core multi-thread model is main-sub
thread model, that is use single thread to build thousands of sub-threads. The main thread only
response to build sub-threads, sub-threads response for process system request. At last using
test program to calculate the response time and working efficiency with this two kinds of design
in setting of the special circumstances.
Keywords
Node.JS; single-threaded platform; multi-core; multi-threaded computing; taxi call system;
Table of Contents Page
Document page i
Abstract ii
Table of Contents iii
1 Introduction 1
1.1 Introduction of Node.JS 1
1.2 Asynchronous I/O 1
1.3 Motivation 2
2 Method 4
2.1 Program analysis 4
2.2 Program design 5
2.3 System design 6
2.4 Server API design 7
2.5 Client Front-end Design 8
3 Implementation 9
3.1 System framework 9
3.2 System implementation 9
3.3 Multi-thread design 10
4 Results 12
4.1 Performance Comparison 12
4.2 Result Analysis 13
5 Discussion and future work 15
5.1 Discussion 15
5.2 future work 15
6 Conclusion 17
7 Bibliography 18
1
1.Introduction
1.1 Introduction of Node.JS
Node.JS or Node is a platform can let JavaScript run on the server side. It can change
JavaScript from the browser bound to run in the general server environment.
Node.JS is born for real time Web application development platform, it is from the
beginning of full consideration in real-time response, large scale data requires scalable
framework. This makes it abandoned the traditional platform relying on multi-thread
to realize the design idea of high concurrency, and uses a single threaded
asynchronous. Type I/O, event driven programming model. These characteristics not
only brought a performance boost, but also reduces the complexity of multi-thread
programming, thus improving the development efficiency.
Node.JS is a JavaScript running environment. In fact, it is the GoogleV8 engine
package. The purpose is to execute and run JavaScript code on the server side.
JavaScript has long time been a Browser based client scripting language, As a server-
side language, because of its asynchronous non blocking characteristics, it is in a long
connection. Advantage of working under multi-request environment is very obviously.
Node.JS on the official website is defined as: “a built in Chrome JavaScript when
running on the platform to build high-speed, scalable. Node.JS network program
using event driven, non blocking I/O model, so it has light weight and high efficiency.
It use to build a perfect equipment running in a distributed data intensive real-time
program selection Web.” the site had not limited to the contents of the presentation of
many interactive and collaborative environment has been moved to the site, and this
demand is still increasing rapidly. This is the so-called data intensive real-time (data-
intensive real-time) applications, such as online collaborative whiteboard, multi-
player online games. The web application the need for a real-time response platform
for a large number of concurrent users request the support of them, this is what the
Node.JS areas of expertise.
1.2 Asynchronous I/O
Threads in the implementation of the disk read and write or network communications,
such as I/O operation usually takes a long time. At this time, the operating system will
deprive the thread of the CPU control, so that it will suspend its execution, while the
Source to other threads, such as the thread of the scheduling method called blocking.
When the I/O operation is completed, the operating system will be the blocking state
of the thread is released to restore its control over the I/O, so that it continues to
perform.
The formula is a synchronous I/O or blocking. Asynchronous I/O or non blocking I/O
will not take a blocking policy for all I/O operations.
When you encounter a I/O operation, will not wait for the completion of the I/O
operation or data return, but only to I/O Request to be sent to the operating system,
2
continue to execute the next statement. When the operating system to complete the
I/O operation, it is a matter of the system will handle the event at a specific time in the
form of a thread that executes the I/O operation. Asynchronous I/O, the thread must
have the event loop, and constantly check that there are no pending events, in turn to
be processed.
In blocking mode, a thread can only handle a task, to improve the throughput must be
through multiple lines. In the non blocking mode, a thread will always perform the
calculation, the CPU kernel used by this thread.
Node.JS is the biggest characteristic of the asynchronous I/O and event driven
architecture design. The solution for high concurrency, the traditional architecture is a
multi-thread model, also is to provide a system thread for each business logic, the
time overhead of.Node.JS system through the thread switch to compensate for the
synchronous I/O call when using a single thread model all the way, the asynchronous
request for all I/O, to avoid the frequent context switching.
The Node maintains a queue of events in the process of implementation, the program
enters the event loop in the execution waiting for the arrival of an event, each
asynchronous I/O request completion will be pushed to the event queue, waiting for
the process for processing.
Node.JS is a new development platform, the Node.JS user community is growing at a
furious pace, and attracts every top developers. Node.JS ecosystem in many of the
perfect.JavaScript is a very popular programming language, almost every computer
are mounted one or more JavaScript compiler, development of most the developer of
Web in its occupation career have learned JavaScript. so Node.JS technology will get
better.
1.3 Motivation
Node.JS provides net, Datagram, HTTP, HTTPS of the 4 modules, respectively for
the treatment of TCP (Transmission Control Protocol), UDP (User Datagram
Protocol), HTTP (Hyper Text Transfer Protocol), HTTPS (Hyper Text Transfer
Protocol over Secure Socket Layer), applicable to the server and the client.
The Node.JS server development of high integration, application of net module can
easily build server, and can handle a large number of concurrent requests for Node.JS
network and good parallel I/O intensive scene, ability to use the event loop can
effectively make use of resources, to provide better service.
According to the characteristics and advantages of Node.JS, this paper proposes a
Node.JS build AIS communication server based on AIS, improve the reliability of
communication between base station and communication center server stability,
timeliness, to meet the demand of AIS application, has application value.
In order to solve the problem of waste in the process of replication, multi-thread is
introduced into the service model, so that a thread service a request. The thread of the
3
process of the relatively low cost of many, and the data can be shared between threads,
memory. The cost of the problem can be solved, and the thread pool can be used to
reduce the overhead of creating and destroying threads.
4
2.METHOD
2.1 program analysis
The heart utilization rate is always 100% I/O to notify to the event. High system
throughput in the event, because a thread blocking when there are other threads in the
work. Multi-threads can make CPU Source is not blocked by the thread waste, and in
the non-blocking mode, the thread will not be blocked by I/O, always in the use of
CPU. Asynchronous I/O to reduce the number of threads created in the thread, the
allocation of memory, the inclusion of scheduling, switch threads, memory Page CPU,
cache and other aspects cost of event driven. And callback function event driven
programming is a programming style that determines the execution flow of a program
by an event. Processor (handler event) or event callback (callback event) to deal with
the event when the event is callback.
When an event born, you call a function that is defined in the event, rather than
simply return the result value. Callback function which is one of the most obvious
features of Node, and this programming model means that the current process is
executing I/O operation will not be blocked, therefore, multiple I/O operations can be
performed in parallel, when the operation is completed, the corresponding callback
event driven programming, the underlying dependency on the event loop (loop event).
Event loop is essentially an event detection. There are two functions, which are
triggered by the event handler and the event handler are constantly called by a
structure. Circular mechanism need to detect what happened, when the event occurs,
it found the corresponding callback function and tune use it.
The event loop is just a thread running within the process, when the event occurs, the
event handler is able to run and will not be interrupted. There is at most one event
callback function at a particular point in time. Any event handler will not be
interrupted at run time.
Node.JS program from the start of the event cycle, to the end of the event loop, all the
logic is the callback event Function, so Node.JS is always in the event loop, the
program portal is the first event loop callback Function. The callback function of an
event may issue a I/O request or a direct launch event during the execution of the
event.
After the line is completed and then return to the event loop, the event loop checks
that there are no pending events in the event queue to the end of the program.
single thread
Node.JS maintains the characteristics of the JavaScript in a single thread in the
browser. Like multi-threaded programming that everywhere concerned about the
status of the synchronization problem, there is no deadlock here, there is no upper and
lower switching performance of the paper is brought on by the overhead.
Here said the Node.JS is single thread, refers to the Node.JS does not give us the
ability to create a thread, all of our own code is executed by a single thread, at the
5
same time, we can only execute one. But the host environment Node.JS is not a single
thread, it will maintain an execution queue, loop detection, scheduling JavaScript
threads to perform asynchronous operations, in response to events, the callback
function is called by the main environment to perform, so the single thread execution
does not conflict with concurrent operation.
cross platform
Node.JS in addition to the use of V8 (Google) with open source JavaScript engine for
Chrome Google, as the JavaScript engine, but also the use of efficient libev and libeio
library support for event driven and asynchronous I/O Node.JS biggest feature is the
use of asynchronous I/O and event driven design framework. Node.JS use the single
threaded model, for all of the I/O are using asynchronous request mode, to avoid the
frequent context switch. Node.JS in the execution of the process will maintain an
event queue, the program in the implementation of the event loop.
Waiting for the next event, each asynchronous I/O request is completed and will be
pushed to the event queue, waiting for the event.
With the social networking era coming, various forms of social networking
applications are continuing to emerge, the mainstream social networking applications
including Facebook, Twitter and Google+, etc., each carrying hundreds of millions of
daily levels of online traffic. Facing such a huge number of users concurrent access,
how to continue to provide high-speed, stable, smooth high-quality user experience
will become one of biggest problems to be solved for network applications.
To overcome concurrent challenges, we need to strengthen and improve the overall
system design and architecture from different dimensions, from inside and outside to
improve application concurrency ability comprehensively Both strengthen internal
systems architecture and optimize key algorithms, but also with third-party
applications external auxiliary components to improve concurrency. Compared to
traditional mufti-threaded programming model and language, application service
platform based on incident response mechanisms, whether from memory utilization,
asynchronous I/O, and ease of programming all have an unparalleled advantage,
maximize effective use of existing resources with enhancing system application
concurrency.
This thesis is mainly based on Node.JS technology, combined with load balancing,
distributed storage and caching techniques, presents a innovative solution to solve
massive concurrent access by an efficient, stable, and scalable asynchronous and
concurrent architecture. Through technical research and real-time high concurrent
access performance tests, verify feasibility and innovation of network application
asynchronous concurrent architecture. Finally, problems and deficiencies arising in
the work are summarized, and prospected system future development.
2.2 program design
6
The system is a simulate taxi call system, it connect to Google map API, simulate
taxis and all actions on Google map.
There are two kinds of model, one is multi-core multi-thread model, other one is
single thread model. Both of them will using to complete same taxi call system.
Figure 1. two different thread model
The multi-core multi-thread model is main-sub thread model, that is use single thread
to build thousands of sub-threads. The main thread only response to build sub-threads,
it averagely build sub-threads on each cores. Sub-threads response for process system
request. The single thread model is a single thread process a request on the core, and
next request needs waiting before it complete last request.
Server uses the express module to define a plurality of different HTTP post request.
Client sends the location to server, server returns the nearest taxi to client, client
displayed those taxi on Google map. Client send the destination to server, server
calculate the travel distance, return to the number of kilometers and prices. Web client
through the acquisition position function of HTML5 to getting user location, users can
also click other place on the map to change the position. Client is responsible for
interaction with the user, send post request to the server side, and display the return
data.
2.3 System Design
The system design is based on client-server architecture. The server side makes use of
7
a popular Node.JS module Express to implement a RESTful API. The server
and client communicates through JSON formatted messages. JSON is a human
readable data format that is very easy for encoding and parsing by Javascript.
Since javascript is used as both server-side and client-side language, the JSON format
is very expedient. Since it is human readable, it is also appropriate for debugging
purpose.
2.4 Server API Design
Car Position Update API
Scenario: The cars will regularly send the server their current position at regular time.
Request Parameters:
car Id
Latitude
Longitude
Action:
Update Car position in the server
Response:
Whether updated successfully
Nearby Cars Search API
Scenario: The client wants to find the nearly cars with a certain distance from their
current position.
Request Parameters:
User Id
Latitude
Longitude
Action:
Search the records of cars positions to find out the nearby cars.
Response:
Send the list of nearby cars information.
Car Booking API
8
Scenario: The client wants to book a car.
Request Parameters:
User Id
Latitude
Longitude
Action:
Find the nearest available car.
Response:
Send the nearest available car information.
Price Computation API
Scenario: When the journey ends, the client wants to compute the price for the
journey.
Request Parameters:
Start Latitude
Start Longitude
End Latitude
End Longtitude
Action:
Compute the price.
Response:
Sending the price information.
2.5 Client Front-end Design
The front-end is the medium between users and servers. It is responsible for
communication on one hand and the interaction with user on another hand.
The front-end will accept user’s input through buttons and text fields, it then sends the
HTTP request to the server using jquery , and updating the UI when receiving the
response from server.
Through RESTful API, the client can be relatively independent of the server. The
only requirement is that client sends request according to the RESTful API
provided by the server.
9
3.Implementation
3.1 System framework
In the server side, I use the nodes module express to implement the car booking
RESTful API. express is one of the most popular framework for Node.JS web
application. It is flexible and easy to use. I use the get and post method of the
express module to define the API url, the parameters and the response.
To simulate the nearby cars, I create file generate.js to randomly generate a
certain number of cars around a given position. The system will find the nearest car
for the client request from these generated random positions.
For the Multi-thread car booking system, I make use of the cluster module. In the
main process, I create the same number of sub-processes as the number of CPUs. All
the worker process will share the same listening port. Other Application logic are the
same with the single-threaded system.
The front-end implementation makes use of Google MAP API which is the most
popular map provider in the world. It uses the marker with car icon to indicate car
positions. The client side makes use of jQuery library to send request to the server
and receive JSON encoded response from it.
3.2 System implementation
Users use computer or mobile phone browser open the web page, web browser
connect to Google map API, user can find taxis near their location on the map.
Then user can click “book”button, system will distribute a recently taxi and hiding
other taxis on the map.
User can click anywhere on the map for his destination address, and the taxi will
come to take him.
After arriving at the destination, the user click the “leave” button to finish this trip on
web page, and the system will show the mileage and price according to the distance
on website.
10
step 1 step 2
step 3 step 4
Figure 2. Program running
The test program is testing when system get 100 client parallel requests and each
client send request 10 times continuously, show rps and response time of two different
systems when they process each task (getposition, book, getprice, carbooking.html).
3.3 Multi-thread design
Node.JS is itself single-threaded. Although it can be efficient when doing IO intensive
stuff through event-driven and non-blocking I/O model, it can’t take advantage of the
multi core when doing CPU intensive computation.
Since an instance of Node.JS runs on a single thread, the natural solution is to create
multiple Node.JS processes to run on multiple cores.
There is a main process that can create multiple sub-processes. The main process
communicates with sub-process through IPC(inter process communication).
The main process continues listening on a port accepting the client’s connections.
When a new connection comes, the main process is responsible to allocation the
connection to one of the sub-process to do the actual work. A scheduling algorithm
such as round robin is implemented to distribute the work as balanced as possible.
Balance means each sub-process will take largely equal size work.
Another approach is that main creates the listening socket and pass the socket to the
sub-process, the sub-process then accepting the connections itself. In this approach,
the scheduling of tasks is handled to the operating system’s process scheduler which
may not make a balanced scheduling.
The sub-process created by main process is also called worker process, because they
do the actual work, whereas main process is responsible for scheduling. The worker
processes are independent of each other. If one worker process fails, others are not
11
affected. As long as there is at least one worker process, the server can continue to
accept new connections.
Designed in this way, we not only can run on multiple cores with multiple processes,
it can also give us more failure-tolerance at the same time.
12
4.Results
4.1 Performance Comparison
To evaluate the performance of Node.JS in single threaded mode and in multi-
threaded mode, I make use of the module siege to test the system response time under
the two different modes. siege is a benchmark tool that measures server’s
performance under duress. It can spawn multiple concurrent client processes to send
multiple HTTP requests at the same time to the server. The siege tool will output the
minimum, maximum, average response time and RPS (request per second). The
HTTP request tested for my system are getposition, book, getPrice and
carbooking.html.
The load testing is conducted in 3.1 GHz Intel Core i5 CPU with 4 cores
and 8 GB memory. Using the siege , I create 100 concurrent client processes to
send the 4 API request each 10 times. The results are plated in following figures.
13
Figure 4. Testing result
4.2 Result Analysis
From above figures, we can see that except in getPosition , the Multi-thread
system has much less response time than the single-thread system and can handle
much more requests per second than the single-thread system.
The reason for this is that, when using single-thread version, only 1 core is used, the
other 3 cores are wasted. But when in Multi-thread system, we have 4 nodes
instance running on 4 cores, fully taking advantage of all cores. Although the Multi-
thread does incur certain overhead such as sub-process spawning and tasks scheduling
14
among worker threads, in most cases, the time saved by using multiple cores
surpassed the time overhead and resulting in quicker server response overall. The
Multi-thread surpasses the single-thread version most when doing tasks that needs
large ratio of CPU computing.
15
5.Discussion and future
5.1 Discussion
In this project, I create a web application for car booking adopting RESTful API
design in client-server architecture with user friendly client side using Google Map.
I extend the system to Multi-thread version overcoming the limitation of single-
threaded nodes. I use siege to test the response time of the implemented Multi-
thread system with the single-threaded version. The result shows that Multi-thread
indeed gives us advantage. The test is conducted in 1 machine with 4 CPUs.
Nowadays, the server are more and more powerful. Companies typically use
multiserver clusters and each with dozens CPUs, we can expect that our Multi-thread
system will unleash more power in such machines.
As previously described, we take advantage of the multi-cores through creating
multiple nodes instances – worker process to distribute the tasks. Is there other ways
to use the multi-cpus at the same time? One answer is to change the implementation
of nodes itself, making it no longer single threaded, but multithreaded. This
approach is possible in theory, but it will change the nodes fundamentally and
require too much more work which certainly beyond the scope of this project. There
is also a third way. Instead of changing Node.JS , we can create new thread when
needed using c++ add-on. Such approach is easier than previous one, but may be
complex for programmer using them because it is much harder to code using add-on
than using pure javascript.
In conclusion, although there are ways to take advantage of the multi-cpus, the
approach in this project is the most sensible way. Because it doesn’t need to change
the Node.JS in fundamentally way, and we can still use pure js to implement it. It
is also very easy to change code from single-threaded version to Multi-thread version
in this way.
5.2 future work
The solution for high concurrency, the traditional architecture is a multi-thread model,
is also a business logic with a system thread, thread switch through the system to
compensate for the synchronous I/O call time. This method will bring the server
hardware resource consumption.
Node.JS method for solving multiple request, lies in the way that the connection to the
server for example, in the Node.JS of each HTTP connection will launch an operation
in the Node.JS engine in the process of the event, and not for each connection to
generate a new OS process. This approach is based on event driven can better save the
server memory resources.
In addition, for those simple server, realized by Node.JS. Node.JS will be easy than
use of the C ten + to solve the long connection request, many problems caused by the
cost, so in some projects, such as online games, online chat, real-time message push
16
function, SNS real-time communication, real-time monitoring system, can consider
the application development of Node.JS.
In recent years, with the rapid development of the domestic shipping industry, the
quantity of ships in navigable waters has grown rapidly. However, at the same time,
the increasing quantity of ships and traffic density has a serious impact on the
ecological environment and traffic safety of the navigable waters. In order to
guarantee the safety of navigation, strengthen the management of the ship, The AIS
base station network system covered river and coastal areas waters has been
established in China. The signal has covered the important navigable waters and ports
of coastal and inland countrywide.
AIS server is designed to receive AIS data with the feature of large quantity and real
time. Nevertheless, in order to avoid occurrences of data congestion, missing, etc.
During communication, it requires the communication links with the AIS base
stations stable, reliable, and able to meet the needs of real-time communication.
Currently, the most of AIS communication servers are developed with VC++
technology. They receive a variety of information via the network port or the serial
interface of the PC linking with AIS base station, which is sent by sailing vessels
through the network.
Node.JS is a platform which allows JavaScript to run on the server side. It has
abandoned the design ideas of the traditional platform which rely on mufti-threading
to achieve high concurrency. However, it adopted single-threaded, asynchronous I/O,
event-driven programming model. It has turned out to be suitable for server design
with the characteristic of high concurrency, real-time response and ultra-large-scale
data.
17
6.Conclusion
This degree work is finding a suitable multi-core multi-thread system for Node.JS
platform to letting Node.JS take full advantage of multi-core. From research result we
can see when client have huge multi-request, multi-core multi-thread system is more
high performance and efficiently than single thread system. That means if multi-core
multi-thread Node.JS can be using effectively, Node.JS can be using on
more broader field. Hope to see that Node.JS can get a wider range of using.
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
file:///F:/Youdao/Dict/6.3.69.8341/resultui/frame/javascript:void(0);
18
7.Bibliography
[1] Karagz,Mehmet Fatih.Design and implementation of RESTful wireless sensor
network
gateways using Node. js framework:proceeding of the 20th European Wireless
Conference,
Turgut, Cevahir, 2014: 517-522[C].
[2] Marak Squires. Ten node applications that need to exist[J].Nodejitsu Blog,
2011, (02):38-41.
[3] Stefan Tilkov, Steve Vinoski. Node. js:Using JavaScript to Build High
Performance
Network Programs[J].IEEE Internet computing,2010, 14(06):80-83.
[4] Liu Feng. Site selection and networking scheme of AIS base station for inland
waterway J]. China Science and technology information, 2011, (11): 97-104.
[5] Liu Chang. Research on Key Technologies of automatic identification system
(AIS) for ships [D]. Dalian: Dalian Maritime University, 2013
[6] Douglas Crockford.JavaScript:The World’s Most Misunderstood
Language[J].Douglas Crockford, 2001, (10):6-15.
[7] Tao Limin, Xuchang, such as automatic identification system (AIS) of the base
station network research (J]. navigation technology, 2004, (04): 31-33.
[8] http://zh.wikipedia.org/wiki/V8_(JavaScript)#cite-note-comic-2.
[9] http:/