untitled
INV ITED
P A P E R
Real-Time Wireless
Sensor-Actuator Networks
for Industrial Cyber-Physical
Systems
Despite their success in industrial monitoring applications, wireless sensor actuator
network (WSAN) technologies face significant challenges in supporting control systems.
This paper surveys recent advances in real-time WSANs for industrial control systems.
By Chenyang Lu, Senior Member IEEE, Abusayeed Saifullah, Member IEEE, Bo Li,
Mo Sha, Member IEEE, Humberto Gonzalez, Member IEEE, Dolvara Gunatilaka,
Chengjie Wu, Lanshun Nie, and Yixin Chen, Senior Member IEEE
ABSTRACT | With recent adoption of wireless sensor-actuator
networks (WSANs) in industrial automation, industrial wireless
control systems have emerged as a frontier of cyber-physical
systems. Despite their success in industrial monitoring
applications, existing WSAN technologies face significant
challenges in supporting control systems due to their lack of
real-time performance and dynamic wireless conditions in
industrial plants. This article reviews a series of recent
advances in real-time WSANs for industrial control systems:
1) real-time scheduling algorithms and analyses for WSANs;
2) implementation and experimentation of industrial WSAN
protocols; 3) cyber-physical codesign of wireless control
systems that integrate wireless and control designs; and
4) a wireless cyber-physical simulator for codesign and
evaluation of wireless control systems. This article concludes
by highlighting research directions in industrial cyber-physical
systems.
KEYWORDS | Cyber-physical systems; industrial wireless net-
works; real-time systems; wireless control systems; wireless
sensor-actuator networks
I . INTRODUCTION
Wireless sensor-actuator networks (WSANs) are gaining
rapid adoption in process industries due to their advantage
in lowering deployment effort in harsh industrial environ-
ments. Industrial standard organizations such as ISA [1],
HART [2], WINA [3], and ZigBee [4] have been actively
pushing the application of wireless technologies in
industrial automation and manufacturing. While early
success of industrial WSANs focused on monitoring
applications, there is significant value in exploring WSANs
for process control applications to take full advantage of
wireless technology in industrial plants.
However, wireless control systems face unique chal-
lenges that distinguish them from traditional control
systems. First, it is challenging to meet the stringent
latency requirements of feedback control in WSANs. IEEE
802.15.4 radios commonly used in WSANs have a
maximum bandwidth of only 250 kbps. Multihop commu-
nication over mesh networks further increases communi-
cation delays. Furthermore, channel conditions in
industrial environments can change dynamically due to
Manuscript received June 6, 2015; revised September 17, 2015; accepted October 23,
2015. Date of publication December 7, 2015; date of current version April 19, 2016. This
work was supported in part by the U.S. National Science Foundation under Grants
1320921 (NeTS), 1035773 (CPS), 1017701 (NeTS), and 1144552 (NeTS), and by the
National Natural Science Foundation of China (NSFC) under Grant 61273038.
C. Lu, B. Li, D. Gunatilaka, and Y. Chen are with the Department of Computer Science
and Engineering, Washington University in St. Louis, Saint Louis, MO 63130 USA
(e-mail: lu@wustl.edu; b.li@wustl.edu; dgunatilaka@wustl.edu; ychen25@wustl.edu).
A. Saifullah is with the Department of Computer Science, Missouri University of
Science and Technology, Rolla, MO 65409 USA (e-mail: saifullaha@mst.edu).
M. Sha is with the Department of Computer Science, State University of New York at
Binghamton, Binghamton, NY 13902 USA (e-mail: msha@binghamton.edu).
H. Gonzalez is with the Department of Electrical and System Engineering, Washington
University in St. Louis, Saint Louis, MO 63130 USA (e-mail: hgonzale@wustl.edu).
C. Wu is with Yahoo!, Sunnyvale, CA 94089 USA (e-mail: wuchjie@gmail.com).
L. Nie is with the Department of Computer Science and Technology, Harbin Institute of
Technology, Harbin 150001, China (e-mail: nls@hit.edu.cn).
Digital Object Identifier: 10.1109/JPROC.2015.2497161
0018-9219 � 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Vol. 104, No. 5, May 2016 | Proceedings of the IEEE 1013
external interference, moving obstacles and weather
conditions. As a result, WSANs demand real-time sched-
uling algorithms and analysis techniques that are both
effective and efficient. In contrast to wired industrial
networks (e.g., control area networks [5]) with well
established real-time scheduling techniques, there have
been limited results on real-time scheduling for WSANs.
The lack of analytical methods for achieving real-time
performance in WSANs hinders their adoption in control
systems. Wireless control demands a new real-time
scheduling theory for WSANs [6].
Second, design of wireless control systems must deal
with interdependencies between control and communica-
tion. For example, while it is well known in digital control
that a low sampling rate usually degrades control
performance [7], a high sampling rate may increase
resource contention in bandwidth-constrained WSANs
leading to long communication delays, which again may
lead to degraded control performance [8]. The coupling
between wireless communication and control therefore
motivates a cyber-physical codesign approach that integrates
wireless networks and control designs.
This article provides a review of recent advances in real-
time WSANs for industrial control systems, an increasingly
important class of cyber-physical systems (CPS) in the dawn
of Industrial Internet [9] and Industry 4.0 [10].
Section II describes real-time scheduling for industrial
WSANs. We first introduce WirelessHART, an industrial
standard for wireless sensing and actuation. We then
present a suite of real-time scheduling algorithms, delay
analyses and protocol implementation to achieve real-time
performance in WirelessHART networks. Section III
introduces our recent efforts to design industrial wireless
control systems following a cyber-physical codesign
approach. We first study the problem of selecting sampling
rates to optimize control performance. We then investigate
how to incorporate emergency alarms in wireless control
systems. To support the design and evaluation of wireless
control systems, we introduce the wireless cyber-physical
simulator (WCPS) for holistic studies of wireless control
systems. We conclude this article by highlighting prom-
ising research directions.
II . REAL-TIME INDUSTRIAL WSANs
We first provide an overview of the WirelessHART
standard and then describe our implementation and
experimentation of a WirelessHART protocol stack.
Finally, we summarize our real-time scheduling algorithms
and analyses for WirelessHART.
A. WirelessHART
A wireless control system comprises feedback control
loops connecting sensors, controllers and actuators
through a wireless mesh network. Sensors measure
variables of the plant and send the measurements to a
controller over the wireless mesh network. The controller
then sends control commands to the actuators in order to
control the physical processes. Industry plants pose harsh
environments for wireless communication due to signif-
icant channel noise, physical obstacles, multipath fading,
and interference from coexisting wireless devices [11].
These harsh and dynamic environments make it difficult
for WSANs to meet the stringent reliability and real-time
requirements of industrial control applications.
There exist multiple industrial WSAN standards, e.g.,
WirelessHART [2], ISA100 [1], WINA [3], and ZigBee [4].
In this article we will focus on the WirelessHART
standard, although the design principles may be extended
for the other standard technologies. WirelessHART
devices are now adopted worldwide for industrial process
management and control [13], [14]. Unique features that
make WirelessHART particularly suitable for industrial
process control are described below.
A WirelessHART network is managed by a centralized
network manager. The network manager is responsible for
the management, scheduling, creating the routes, and
optimizing the network. Network devices include a
Gateway, a set of field devices, and several access points
as shown in Fig. 1. The network manager and the
controllers are installed or connected to the Gateway.
The field devices are wireless sensors and actuators. Each
field device is equipped with a half-duplex omnidirectional
radio transceiver compliant with the IEEE 802.15.4
standard. Multiple access points are wired to the Gateway
to provide redundant paths between the wireless network
and the Gateway.
A WirelessHART network adopts various mechanisms
to ensure reliable communication in unreliable industrial
environments. Time is globally synchronized and slotted
Fig. 1. Architecture of a WirelessHART network (Credit: HART
Communication Foundation [12]).
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
1014 Proceedings of the IEEE | Vol. 104, No. 5, May 2016
using a built-in time synchronization protocol. Each time
slot is 10 ms, which is sufficient to send or receive one
packet and an corresponding acknowledgement. Transmis-
sions are scheduled based on a multichannel time division
multiple access (TDMA) protocol. A time slot can be either
dedicated or shared. In a dedicated slot, only one sender is
allowed to transmit to the receiver. In a shared slot, more
than one sender can attempt to transmit to the same
receiver. Since shared slots are assigned to multiple
senders, collisions may occur within a shared slot. To
reduce collisions, senders adopt carrier sense multiple
access with collision avoidance (CSMA/CA) to contend in a
shared slot. CSMA/CA allows multiple nodes to share a
common channel through carrier sensing, where nodes
attempt to avoid collisions by transmitting only when the
channel is idle.
The network uses 16 channels defined in IEEE 802.15.4,
and adopts channel hopping in every time slot. Channel
hopping provides frequency diversity to mitigate interfer-
ences and reduce multipath fading effects. Any excessively
noisy channel is blacklisted.
A WirelessHART network supports two types of routing
approaches: source routing and graph routing. Source
routing provides a single directed path for routing from a
source to a destination device. Graph routing involves a
routing graph consisting of a directed list of paths between
the two devices, and is adopted for enhanced end-to-end
reliability. In graph routing, packets from field devices are
routed to the Gateway through the uplink graph. To every
field device, there is a downlink graph from the Gateway.
The end-to-end communication between a source (sensor)
and destination (actuator) happens in two phases. In the
sensing phase, on one path from the source to the Gateway
in the uplink graph, the scheduler allocates a dedicated slot
for each device starting from the source, followed by
allocating a second dedicated slot on the same path to
handle a retransmission. Then, to offset failure of both
transmissions along a primary link, the scheduler allocates
a third shared slot on a separate path to handle another
retry. Then, in the control phase, using the same method,
the dedicated links and shared links are scheduled in the
downlink graph of the destination.
B. Implementation and Experimentation
To study and evaluate WSAN protocols, we have
developed an experimental WSAN testbed [15]. The system
comprises a network manager on a server and a network
protocol stack implementation on TinyOS 2.1.2 [16] and
TelosB motes [17]. Each mote is equipped with a TI
MSP430 microcontroller and a TI CC2420 radio compat-
ible with the IEEE 802.15.4 standard. Fig. 2 shows the
motes deployment in our campus buildings. In an
experiment, our motes can be designated as access points
and field devices, which form a multihop wireless mesh
network running WSAN protocols. To support experi-
mentation and measurements, all motes in our system are
physically connected to a wired backplane network that
can be used for managing wireless experiments and
measurements without interfering with wireless com-
munication.
Our network manager implements a route generator
and a schedule generator. The route generator is
responsible for generating source routes or graph routes
based on the collected network topology, while the
schedule generator is responsible for generating packet
transmission schedules. Our protocol stack adopts the
CC2420x radio driver [19] as the radio core, which is
responsible for transmitting and receiving packets. A
multichannel TDMA MAC protocol, RT-MAC, is imple-
mented on top of the radio core. As shown in Fig. 3, RT-
MAC divides time into 10 ms slots based on the
WirelessHART standard. The clocks of all the motes in
the network are synchronized using Flooding Time
Synchronization Protocol (FTSP) [20] during a Sync
window. The network then transmits packets based on
recurring superframes (transmission schedules). A 2 ms
guard time is reserved in the beginning of each slot to
accommodate the clock synchronization error and channel
switching delay. Both dedicated and shared slots specified
in the WirelessHART standard are supported by RT-MAC.
Only one sender is allowed to transmit and the packet
transmission occurs immediately after the guard time in a
dedicated slot, while more than one sender can contend
for the channel in a shared slot.
Fig. 2. WSAN testbed in Bryan Hall and Jolley Hall of Washington
University in St. Louis [18].
Fig. 3. Time frame format of RT-MAC.
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
Vol. 104, No. 5, May 2016 | Proceedings of the IEEE 1015
We have performed a series of experiments on our
WSAN testbed [15]. For instance, we have conducted a
comparative study of the two alternative routing ap-
proaches adopted by WirelessHART, namely source
routing and graph routing, and investigated the tradeoff
among reliability, latency, and energy consumption under
the different routing approaches.
We first run our experiments in a clean environment
where we blacklist the four 802.15.4 channels overlapping
with our campus Wi-Fi network and run the experiments
on the remaining 802.15.4 channels. We then repeat our
experiments in a noisy environment where we configure
the network to use channels overlapping with our campus
Wi-Fi network and in a stress-testing environment where
we generate controlled Wi-Fi interference. In our study,
we use reliable links with PRR higher than 80%, where
PRR stands for Packet Reception Ratio, i.e., the fraction of
transmissions that are successful over a wireless link. We
set up 8 data flows, and run our experiments long enough
such that each flow delivers at least 500 packets from its
source to its destination. We measure the packet delivery
rate (PDR) of a flow, defined as the percentage of packets
that are successfully delivered to their destination. Table 1
shows the minimum, median and maximum PDR among
all the flows in the clean, noisy and stress-testing
environments. Graph routing improves the median PDR
by 15.9%, and 21.4% compared to source routing in noisy
and stress testing. Moreover, graph routing drastically
improves the minimum PDR, with 35.5% and 63.5%
increase in noisy and stress testing environments. This
result shows graph routing can effectively enhance the
predictability of real-time performance, especially under
significant interference. However, our study also shows
that route diversity incurs costs for latency and energy
consumption, with graph routing suffering from an
average of 80% increase in end-to-end latency and
consuming an average of 130% more energy over source
routing.
Our study concludes that it is important to employ
graph routing algorithms specifically designed to optimize
latency and energy efficiency. This problem was recently
explored in [21] and [22], which proposed a set of real-time
and energy-efficient routing algorithms for WirelessHART
networks. Both source routing and graph routing have
been implemented in our WSAN testbed and the Wireless
Cyber-physical Simulator (WCPS) (to be discussed in
Section III-B), which enable systematic comparison between
the alternative routing approaches.
C. Real-Time Scheduling
To save cost and enhance flexibility, it is desirable to
support many control loops in a same network. The
stability and control performance of these systems heavily
depends on real-time communication over the shared
wireless network. Feedback control loops in a WSAN
therefore impose strict requirements on reliability and
real-time guarantees in wireless communication in order
to avoid plant shutdowns and/or accidents. For example, in
oil refineries, spilling of oil tanks has to be avoided by
controlling oil level in real-time. However, industrial
plants pose a harsh environment for wireless communica-
tion due to unpredictable channel conditions, limited
bandwidth, physical obstacles, multipath fading, and
interference from coexisting wireless devices [11]. With
the adoption of industrial wireless standards such as
WirelessHART [2], process monitoring and control
applications have seen the feasibility of achieving reliabil-
ity and real-time wireless communication through spatial
and spectrum diversity. Real-time communication in these
wireless networks pose new and important challenges.
Unlike wired networks, there have been limited results on
real-time scheduling theory for wireless networks. Real-
time transmission scheduling and analysis for WSANs
require new methodologies to deal with unique wireless
characteristics.
1) Overview of State of the Art: Real-time communication
in wireless networks has been explored in many earlier
efforts [23]–[34], [34]–[40]. The survey in [41] provides a
comprehensive review of these works. However, these are
not suitable for industrial applications that usually need
multichannel communication, multipath routing and real-
time performance analysis results. Earlier works [42]–[46]
on real-time performance analysis for wireless sensor
networks focused on data collection through a routing tree
[42], [45].
Real-time WSAN for control purposes has been studied
for single hop networks in [47]–[52]. For multihop
wireless networks, a mathematical framework to model
and analyze schedules for WirelessHART networks has
been proposed in [53]. Real-time scheduling for Wireless-
HART networks has received attention in recent works [6],
[54]–[60]. These scheduling policies can be broadly
classified into two categories: fixed priority scheduling
and dynamic priority scheduling [61]. In a fixed priority
scheduling policy, each data flow has a fixed priority, and
every transmission has the priority of its flow. Due to its
simplicity and low scheduling overhead, fixed priority
scheduling is a common class of real-time scheduling
policies in practice [61]. On the other hand, dynamic
priority scheduling policy refers to the policy where there is
no fixed priority, i.e., priorities of transmissions during the
scheduling change dynamically according to some chosen
criterion. While such policies have higher scheduling
overhead, they provide better schedulability since
TABLE 1 Minimum, Median, and Maximum pdr Among All Flows
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
1016 Proceedings of the IEEE | Vol. 104, No. 5, May 2016
priorities are changed dynamically to enhance the
schedulability.
In the following we summarize our recent results on
real-time scheduling for industrial WSANs under fixed
priority and dynamic priority scheduling, respectively.
2) Fixed Priority Scheduling: Delay analysis can be used to
determine whether real-time data flows can meet their
deadlines. When used offline, the analysis helps users
design a WSAN to meet the real-time performance
requirements. For example, we used schedulability analysis
to select the sampling rates of wireless control loops to
optimize control performance under communication dead-
lines [62]. In [57], we used schedulability analysis to assign
priorities to real-time flows in order to meet their
deadlines. When used online, the network manager can
adjust the workloads in response to network dynamics.
Industrial environments can cause frequent changes in
network topologies and channel conditions. If a WSAN can
no longer guarantee the deadlines for all the flows based on
delay analysis, the network manager may stop or adjust the
data rates of noncritical flows in order to maintain real-time
guarantees to the remaining flows. The Wireless Cyber-
Physical Simulator (WCPS) introduced in Section III-B can
be used to assess the pessimism of delay analysis and
wireless control performance under realistic settings.
For fixed priority scheduling in WirelessHART net-
works, we proposed a suite of delay analyses for real-time
flows [55], [56]. These analyses determine upper bounds
on the end-to-end communication delays and provide
sufficient conditions for schedulability. Two factors con-
tribute to the communication delay of a control loop. A
lower priority flow can be delayed by higher priority flows:
a) due to channel contention (when all channels are assigned
to transmissions of higher priority flows in a time slot); and
b) due to transmission conflicts (when a transmission of the
flow and a transmission of a higher priority flow involve a
common node). A key insight underlying this analysis is to
map the real-time transmission scheduling in Wireless-
HART networks to real-time multiprocessor scheduling. By
incorporating the unique characteristics of WirelessHART
networks into the state-of-the-art worst case response time
analysis for multiprocessor scheduling, the analysis can
efficiently compute a safe and tight upper bound of the end-
to-end delay of every flow.
Priority assignment is an important problem in fixed-
priority scheduling as it has a significant impact on the
schedulability of real-time flows. An ideal priority assign-
ment should not only enable real-time flows to meet their
deadlines, but also work synergistically with real-time
schedulability tests to support effective network capacity
planning and efficient online admission control and
adaptation. In [57] we studied optimal and near optimal
fixed priority assignment using local search. A salient
feature of this local search approach is that it exploits the
delay bounds provided by a schedulability analysis to
efficiently search for feasible priority assignments by
discarding unnecessary branches in the search space.
3) Dynamic Priority Scheduling: Dynamic priority sched-
uling of transmissions was studied in [58] and [60] for
WirelessHART networks of with tree topologies. Our work
in [54] and [59] studied dynamic priority scheduling of
transmissions for general WirelessHART networks. The
work in [54] has shown that the real-time transmission
scheduling for WirelessHART networks is NP-hard. We
observed that transmission conflicts play a major role in the
communication delays and the schedulability of the control
loops, which makes the traditional real-time scheduling
policies such as least laxity first (LLF) less effective. We
propose an optimal local search scheduling algorithm that
exploits the necessary condition to effectively discard
infeasible branches in the search space. We also proposed a
faster heuristic called conflict-aware least laxity first
(C-LLF) for dynamic priority scheduling. The algorithm
identifies the critical time windows in which too many
conflicting transmissions have to be scheduled, thereby
determining the criticality of each transmission. Criticality
of a transmission is quantified by its conflict-aware laxity,
which is its laxity after discarding the (estimated) time slots
that can be wasted through waiting to avoid transmission
conflict. Transmissions exhibiting the lowest conflict-
aware laxity are assessed to be more critical. C-LLF gives
the highest priority to the transmissions exhibiting lower
conflict-aware laxity. Thus C-LLF integrates LLF and the
degree of conflicts associated with a transmission, and
outperforms traditional real-time scheduling policies.
Our work in [59] has provided a schedulability analysis
for earliest deadline first (EDF), a common dynamic
priority scheduling in WirelessHART networks.
We evaluated our schedulability analyses against
experimental results on our WSAN testbed [18] as well
as in simulations. In our evaluation, both fixed-priority
scheduling [57] and dynamic priority scheduling [54]
policies outperformed the traditional real-time scheduling
policies by significant margins. All experimental results
and simulations showed that our analyses provide safe
upper bounds of communication delays in the network and
enable effective schedulability tests [56]. For tighter upper
bounds under graph routing, we established probabilistic
delay bounds in [6] that represent upper bounds with
probability � 0.90. The worst-case and probabilistic
bounds can be used in different application scenarios
depending on the level of predictability required. Our
analysis hence can be used for an effective schedulability
test and admission control of real-time flows in Wireless-
HART networks.
III . WIRELESS CONTROL CODESIGN
Due to limited resources in a WSAN shared by multiple
control loops, it is critical to optimize the overall control
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
Vol. 104, No. 5, May 2016 | Proceedings of the IEEE 1017
performance. However, in a wireless control system, the
control performance not only depends on the control
algorithms, but also relies on real-time communication
over the shared wireless network. The coupling between
real-time communication and control requires a cyber-
physical codesign approach for a holistic optimization of
control performance. This section summarizes our recent
efforts on cyber-physical codesign for industrial wireless
control systems. We first study the problem of selecting
sampling rates to optimize the control performance of
multiple feedback control loops sharing a WSAN. To
support cyber-physical codesign of wireless control
systems, we present wireless cyber-physical simulator
(WCPS), an integrated simulation environment for holistic
studies of wireless control systems. We then investigate
how to incorporate emergency alarms in wireless process
control systems. We wrap up this section with a case study
of wireless process control for coupled water tanks using
both regular and emergency control loops.
A. Rate Selection
In control systems, the impacts of sampling rate have
been studied for robot control [63], disturbance rejection
[64], and various control performance indices [65]–[69].
The choice of sampling rates needs to balance control and
communication. While a high sampling rate is desirable
from a pure digital control perspective, it also has the
undesirable effect of heavier network load and longer
communication delay, which can also degrade the control
performance.
The control performance can be quantified as a
function of sampling rates using the formulation proposed
by Seto et al. [65], which characterizes the performance
difference between the continuous-time control system
and that of its digital implementation.
In [62] and [70] we apply these ideas to optimize the
performance of a set of feedback control loops over a
WSAN, where the control cost of the i-th control loop at the
sampling rate fi is approximated as �i e
��i fi , where
�i; �i > 0 are the magnitude and decay rate coefficients,
respectively. Given the sampling rates of all of the control
loops ff1; f2; . . . ; fng, the total control cost of the system is
defined by
Xn
i¼1
wi �i e
��i fi (1)
where wi is the weight of the i-th control loop. We then use
the total control cost as the performance index of the
system and as the objective of an optimization problem to
determine the best sampling rates of a multiloop control
system sharing a WirelessHART network under stability
and delay constraints. In particular, the objective is to
minimize the cost in (1) subject to two constraints. First,
the delay bound Ri of the i-th loop, which can be computed
based on the sampling rate in [55] and [56], must be
smaller than its predetermined deadline Di. Second, each
control loop must have a sampling rate fi within its
minimum and maximum rates, denoted f mini and f
max
i
respectively, to ensure stability. Thus, we mathematically
formulate this optimization problem as follows:
min
ffigni¼1
Xn
i¼1
wi �i e
��i fi
subject to : Riðf1; . . . ; fnÞ � Di; 8 i 2 f1; . . . ; ng;
f mini � fi � f
max
i ; 8 i 2 f1; . . . ; ng: (2)
The resulting constrained optimization problem is
challenging since the delay Ri is nondifferentiable,
nonconvex, and not in closed-form. We explored four
methods to solve this problem: a) a subgradient-descent
algorithm; b) a greedy heuristic; c) a penalty method using
simulated annealing (SA); and d) a convex relaxation
method based on a new delay bound that is convex and
smooth but more pessimistic than previous analyses [55],
[56] using a different approach. Notably, the convex
relaxation greatly simplifies the optimization problem,
since the overall problem becomes convex and differen-
tiable by simplifying the delay bound analysis.
In [62], we evaluate the methods through simulations
based on the topology of our WSAN testbed [18]. The
results demonstrate that SA achieves the lowest control
cost but requires the longest execution time. Both the
greedy heuristic and the subgradient method lead to high
control costs because the optimization problem is highly
nonlinear with a large number of local extrema. For
example, for 30 control loops, the greedy heuristic incurs a
control cost up to 2.67 times that of SA. In contrast, the
convex relaxation approach using an interior point method
incurs a control cost no more than 35% higher than that of
SA, while incurring a much shorter execution time. The
convex relaxation approach therefore achieves the desir-
able balance between control performance and run-time
efficiency. This result demonstrates the significant
advantage of cyber-physical codesign that integrates
control optimization and schedulability analysis in
wireless control systems.
B. WCPS
Existing wireless control system research often relies
on lab-scale equipment and simulations. However, lab-
scale equipment usually suffers from limited physical size,
which cannot capture delays and data loss in real WSANs.
Simulation tools for control systems often lack realistic
models of WSANs that exhibit complex and stochastic
behavior in real-world environments. The lack of tools that
can capture both the cyber wireless network and physical
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
1018 Proceedings of the IEEE | Vol. 104, No. 5, May 2016
aspects of control systems has been a hurdle to wireless
control research.
Early experimental work on wireless control [71]–[76]
usually relied on a lab testbed where wireless sensors are
within a single hop and experience no data loss due to the
physical proximity of the devices. The challenge in realistic
experimentation with wireless control systems motivated
the development of simulation tools for wireless control
systems. For example, NCSWT [77] and Gisso [78] are two
simulators designed for wireless control systems. Truetime
[7] is a well established control system simulator capable of
holistic studies of CPU scheduling, communication, and
control algorithms. Unfortunately, none of these simula-
tors provides a realistic wireless radio model or a state-of-
the-art WSAN protocol stack.
Wireless cyber-physical simulator (WCPS) [79] is
designed to provide a holistic and realistic simulation of
wireless control systems. WCPS employs a federated
architecture that integrates: a) simulink for simulating the
physical system dynamics and controllers; and b) TOSSIM
for simulating WSANs. Simulink is commonly used by
control engineers to design and study control systems, while
TOSSIM [80] has been widely used in the sensor network
community to simulate WSANs based on wireless link
models that have been validated in diverse real-world
environments [81]. WCPS provides an open-source middle-
ware to orchestrate simulations in Simulink and in TOSSIM.
WCPS 2.0 implements the WirelessHART network
protocol stack at the routing and MAC layers [82]. To
support WirelessHART we also extended TOSSIM to
simulate wireless communication over multiple channels.
We have implemented both Source Routing and Graph
Routing as specified in the WirelessHART standard. To our
knowledge, WCPS 2.0 is the first simulator that supports
WirelessHART protocols based on a realistic wireless link
model.
Thanks to WCPS we have been able to develop case
studies that simulate wireless control systems for civil
infrastructure and process plants. In [83] we simulated two
wireless structural control systems. Wireless structural
control systems offer an attractive approach to protect civil
infrastructures from natural hazards such as earthquakes
and other natural disasters. In the first simulation we
studied a benchmark building model, where the wireless
traces were collected in a multistory building. In the
second simulation we studied the structural model of the
Cape Girardeau bridge over the Mississippi River, where
the wireless traces were collected from a similar bridge in
South Korea. These case studies shed light on the
limitations of traditional structural control approaches
under realistic wireless conditions. They further allowed
us to validate the advantages of cyber-physical codesign
algorithms for wireless communication protocols [84],
[85]. Based on our experience with WCPS, we have
recently enhanced the wireless building control study,
transforming it into a benchmark that can be used by the
structural control community to explore and evaluate
different wireless control approaches [86], allowing
practitioners to easily generate and configure realistic
nuisances such as network induced delay, data loss,
measurement noise, and control constraints.
C. Emergency Communication
In [82], we considered the case of WirelessHART
control networks comprised of: a) regular flows, which are
periodically generated and typically stabilize a desired part
of a physical system; and b) emergency flows, which are
infrequent and typically signal that an unsafe situation is
about to occur. Both regular and emergency flows have
predetermined deadlines to transmit a packet. Since
emergency flows carry information related to unsafe
situations, these flows have a higher criticality than
regular flows. Therefore, we proposed the following
approach to transmit packets through the network: a) in
regular mode, i.e., when there is no emergency, all regular
flows should meet their deadlines; and b) in emergency
mode, i.e., when an emergency occurs, all existing
emergency flows should meet their deadlines, while
regular flows are delivered on a best-effort basis.
Periodic scheduling (PS) is a baseline scheduling
approach that reserves periodic slots dedicated to emergen-
cy packets in the transmission schedule. Emergency packets
can be transmitted only in the reserved slots. When there is
no emergency, the reserved slots are left unused. We take a
two-level priority assignment approach for scheduling,
where we first schedule emergency flows to meet their
deadlines, and then we schedule the regular flows using the
remaining capacity. Within each group, flows are scheduled
using a rate monotonic policy. This basic approach is simple
in design, yet it wastes (a potentially large amount of)
network bandwidth, reserving time slots for emergency
flows even when there is no emergency.
To avoid wasting resources when no emergencies
occur, we designed a slot stealing (SS) method that allows
emergency flows to ‘‘steal’’ slots from regular schedules
when emergencies arise, removing the need to allocate
exclusive time slots for emergency flows. The ‘‘stealing’’
process is implemented by adding a random backoff and a
clear channel assessment (CCA) to the transmission of all
regular flow packets, and allowing emergency packets to
be transmitted immediately at the beginning of the time
slot. Hence, if an emergency packet is available, then it
will take the slot, and any regular flow packet trying to use
that slot will fail the CCA, forcing it to wait for the next
available time slot or drop the packet. By allowing the
overlapping emergency and regular schedules, SS is able to
schedule the same number of flows in a shorter time frame
when compared to PS, while ensuring timely delivery of
emergency packets in case of an emergency.
We also explored two alternative approaches to send
emergency signals. For systems that need to periodically
monitor and control the emergency state, an emergency
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
Vol. 104, No. 5, May 2016 | Proceedings of the IEEE 1019
flow is activated whenever emergencies occur, and the
emergency flow then periodically generates data until the
emergency is over. For systems that do not need to
periodically monitor and control the emergency state, the
system can adopt an event-based approach to communicate
the emergency alarms, i.e., an emergency sensor only
sends one alarm-start packet and one alarm-stop packet at
the beginning and end of the emergency, respectively.
While the event-based communication results in the same
transmission schedule, it significantly reduces the number
of regular transmissions that are dropped or delayed by
emergency transmissions, leading to better control
performance. In [82], simulation results showed the
combination of slot stealing and event-based emergency
communication produced the best results in terms of
emergency handling and control performance.
D. Emergency Control Case Study
We tested our emergency communication protocol on a
set of two identical water tank systems sharing a common
wireless network simulated using WCPS 2.0. A diagram of
each of water tank systems is shown in Fig. 4. Our choice is
based on the simple, yet representative, dynamics of water
tanks, their hybrid dynamical nature (since the evolution
of the system changes when the water tanks are either full
or empty), and more importantly, its similarity to systems
commonly used in industrial applications. Also, we
simulated two identical water tank systems sharing a
wireless network to study the effect of one system’s
emergency over the second system’s regular flow.
Following the software architecture in WCPS 2.0, the
sensor data generated by Simulink is fed into the WSAN
simulated using TOSSIM. TOSSIM then returns the
packets delayed or dropped according to the behavior of
the network, which are then fed to the controller
implemented in Simulink. Controller commands are
then fed again into TOSSIM, which delays or drops the
packets and sends the outputs to the actuators in the water
tanks, closing the loop.
For this study we collect wireless traces from 21 nodes
in a WSAN testbed [18]. We then use the wireless traces as
inputs to the TOSSIM simulator to generate realistic
wireless characteristics in a simulated WSAN. The route in
the simulated WSAN is 6 hops. We used further adjusted
received signal strength (RSS) to test the system perfor-
mance with different wireless signal strengths.
On the control side, we used a PID controller to
regulate the water level of Tank 2, denoted L2, by actuating
the Pump taking water from the Basin, as shown in Fig. 4.
If no emergencies occur, then the Valve stays closed. We
also defined a discrete emergency controller setting the
values of the Pump and Valve, with the objective of
avoiding water spillage. Hence, if any of the emergency
level sensors, denoted LH1 , L
H
2 , L
L
b, and L
H
b , detect a
dangerous water level, then an emergency signal is sent
and the controller switches from the PID controller to the
discrete emergency controller, switching back when the
emergency is cleared. We defined a system failure using
two principles: a) if the system cannot stabilize at a regular
configuration (i.e., within nonemergency water levels) in a
fixed 100 second interval; and b) if any of the water levels
exceed the height of its associated water tank
Our simulations show that the combination of the slot-
stealing and event-based emergency communication is
highly effective in avoiding system failures, since it
produces a tighter schedule that results in a faster update
frequency of the regular control flows. It also reduces
communication load and mitigates the impact of emer-
gency communication on regular control flows. Our case
study demonstrates that even for a 6-hop lossy wireless
network (with 5.8% median packet loss), successful
system control can still be achieved through a combination
of regular and emergency control.
IV. RESEARCH DIRECTIONS
A. Enhancing Scalability
A major limitation of current industrial WSANs is their
limited scalability due to the centralized network archi-
tecture. In WirelessHART, when a node or link fails, the
centralized network manager must generate a new global
TDMA schedule, which requires the network manager to
collect the current topology of the network, create a new
schedule, and distribute the schedule among all field
devices. As WSANs are subject to frequent changes in
channel quality and link condition, global changes to the
network schedule create excessive communication
overhead in a large network. As a result, while the
Fig. 4. Diagram of the coupled water tank system. The water levels
of Tank 1, Tank 2, and Basin are denoted L1, L2, and Lb, respectively.
The water emergency levels are denoted LH1 , L
H
2 , L
L
b, and L
H
b . The water
flows between tanks are denoted u, v12, v1b, and v2b. The state of the
valve is denoted d 2 f0; 1g, where d ¼ 1 if the valve is open.
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
1020 Proceedings of the IEEE | Vol. 104, No. 5, May 2016
centralized architecture has proven to be sufficient for
small-scale installations, it can become a significant
limitation as WSANs start to be deployed over large
geographic areas (e.g., thousands of devices over an oil
field). A key research direction is to make WSANs scalable
while achieving end-to-end real-time communication.
A promising approach to enhance scalability is through
a hierarchical network architecture, where a large WSAN
is divided into multiple subnetworks. Each subnetwork
employs its own manager to manage local operations, and a
global manager coordinates with the subnetwork managers
to manage the entire network in a hierarchical fashion. An
advantage of this architecture is that it can meet industrial
needs for both network visibility and scalability. As a
submanager may deal with the wireless dynamics within
its subnetwork, the hierarchical architecture can scale
effectively. A challenge in designing a hierarchical
architecture is to deal with the interdependencies among
the subnetworks that share the wireless spectrum and need
to support flows traversing multiple subnetworks subject
to end-to-end deadlines.
B. Exploring White Spaces
Another limiting factor of today’s WSANs stems from
the short communication range of the IEEE 802.15.4
radios adopted by industrial standards such as Wireless-
HART. To overcome the short communication range,
many WSANs form multihop mesh networks resulting in
long communication delays and limited scalability. An
opportunity to enhance the scalability of real-time WSANs
arises from the opening of a new spectrum resulting from
the transition to digital TV broadcasting globally and
freeing up the VHF/UHF spectrum. White spaces refer to
the allocated but locally unused TV spectrum. Since TV
transmissions are in lower frequencies, white space
transmissions have excellent propagation characteristics
over long distance. They can easily penetrate walls and
other obstacles, and hence hold enormous potential for
industrial applications that need real-time communication
over large geographic areas. Thanks to its long communi-
cation range and wall-penetration capability, a white space
network will have small hop counts and will drastically
reduce communication delays and protocol complexities.
While white spaces are mostly being tapped into for
wireless broadband access to date, they also open up the
opportunity to support highly scalable real-time industrial
applications that have been challenging under existing
WSAN technologies [87]. The characteristics and the
application demands in industrial sensing and actuation
pose unique challenges in adopting white spaces for
industrial WSANs. Instead of high-throughput traffic, a
WSAN should exploit the wide spectrum in white spaces to
support low-data-rate communication from numerous field
devices. Furthermore, a WSAN over white spaces must
achieve high degrees of energy efficiency that is compara-
ble to today’s WSAN technologies. It also needs to handle
changes and variations in spectrum availability in white
spaces while maintaining desired real-time performance.
C. Cyber-Physical Codesign
While earlier work has shown the promise of cyber-
physical codesign in wireless control systems through
point designs and case studies, there exist opportunities to
develop a theory and practice of cyber-physical codesign
through a broader exploration of the interaction between
the network protocol stack and control design. For
example, if a controller employs a state observer to
estimate system states based on intermittent observations,
the controller can tolerate a certain degree of data loss
from the sensors. A WSAN can exploit the controller’s
resiliency against sensor data loss by reducing the route
redundancy from sensors to controllers, while dedicating
more network resources to enhance the reliability of
communication to actuators. That is, the allocation of
network resources should be dependent on the control
design. Conversely, an actuator may buffer the control
inputs from a model predictive controller and use
previously received control inputs when the network fails
to deliver the new control inputs. When the wireless
condition deteriorates causing more data loss, the
controller may increase its sampling rates or control
horizon to increase its tolerance to data loss to the
actuators. That is, the controller should adapt to network
conditions. Note that the adaptation of network resources
and controller sampling rates can be designed using
approximated mathematical models, or using distributed
extensions of the data-driven methods in [88]. Therefore, a
wireless control system will need to codesign wireless
networks and control in an interdependent fashion. To
establish a unified cyber-physical codesign approach to
industrial wireless control, it will be crucial to develop
interfaces between the WSAN and the control components
to maintain optimal control performance by adapting to
changing network and physical dynamics in the wireless
control systems. We need to further develop a suite of
algorithms and analytical techniques to assure the safety of
the control system under dynamic conditions. Finally, the
research community will need long-term collaboration
among computer science, control theory and domain
experts for this inherently interdisciplinary research.
D. Cyber-Physical Testbed
The advancement and adoption of new WSAN
technologies have been hindered by the lack of testbeds
under realistic industrial settings. While cyber-physical
simulators such as WCPS are useful for studying wireless
control systems, they require realistic industrial plant
models and wireless traces collected from real-world
industrial settings. Despite best efforts on modeling
techniques, simulations cannot replace physical compo-
nents and networks with complex and sometimes unex-
pected dynamics. Unfortunately, full-scale industrial
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
Vol. 104, No. 5, May 2016 | Proceedings of the IEEE 1021
plants are often difficult to replicate for research due to
space and safety constraints. A step beyond simulations
might be hybrid testing technology that combines physical
components and simulations for hardware-in-the-loop
experiments. For example, a physical WSAN testbed may
be carefully integrated with simulations of an industrial
plant to study the impact of real wireless dynamics on
control systems. Conversely, an industrial plant may be
integrated with a WSAN simulator to evaluate potential
wireless control designs without a physical WSAN
deployment. This would require extending tools such as
WCPS to support hardware-in-the-loop testing where both
the WSAN and the plant can be replaced by physical
implementations. Close partnership between industry and
researchers will be critical to develop realistic cyber-
physical testbeds for industrial wireless control systems.
V. CONCLUSION
Real-time WSANs are poised to play a key role in industrial
automation in the era of Industry 4.0 [10] and Industrial
Internet [9]. Recent research and industrial developments
have demonstrated the promise of supporting real-time
communication and control over WSANs. This article
reviews recent advances in two related fronts: a) real-time
scheduling and analytical techniques for achieving real-
time performance in industrial WSANs and b) cyber-
physical codesign of wireless control systems. We highlight
the significant challenges and opportunities in cyber-
physical systems research that crosscut wireless and
control domains. Interdisciplinary collaboration and part-
nership among wireless and control researchers and
industry communities will be crucial in realizing the
vision of wireless industrial control. h
REF ERENCE S
[1] ISA100. [Online]. Available: https://www.isa.
org/isa100/.
[2] WirelessHART. [Online]. Available: http://
en.hartcomm.org/.
[3] Wireless Industrial Networking Alliance.
[Online]. Available: http://www.wina.org.
[4] ZigBee Alliance. [Online]. Available: http://
www.zigbee.org.
[5] R. I. Davis, A. Burns, R. J. Bril, and
J. J. Lukkien, ‘‘Controller area network (can)
schedulability analysis: Refuted, revisited and
revised,’’ Real-Time Syst., vol. 35, no. 3,
pp. 239–272, 2007.
[6] A. Saifullah, D. Gunatilaka, P. Tiwari,
M. Sha, C. Lu, B. Li, C. Wu, and Y. Chen,
‘‘Schedulability analysis under graph routing
for wirelesshart networks,’’ presented at the
RTSS, , 2015.
[7] A. Cervin, D. Henriksson, B. Lincoln, J. Eker,
and K.-E. Årzén, ‘‘How does control timing
affect performance? Analysis and simulation
of timing using Jitterbug and TrueTime,’’
IEEE Control Syst., vol. 23, no. 3, pp. 16–30,
Jun. 2003.
[8] F.-L. Lian, J. Moyne, and D. Tilbury,
‘‘Network design consideration for distributed
control systems,’’ IEEE Control Syst. Technol.,
vol. 10, no. 2, pp. 297–307, Mar. 2002.
[9] P. Evans and M. Annunziata, ‘‘Industrial
Internet: Pushing the boundaries of minds
and machines, General Electric (GE),
Nov. 26, 2012. [Online]. Available:
http://www.industrialinternetconsortium.
org/white-papers.htm.
[10] H. Kagermann, W. Wahlster, and J. Helbig,
‘‘Recommendations for implementing the
strategic initiative industrie 4.0: Final report
of the industrie 4.0 working group,’’
Apr. 2013, Tech. Rep.
[11] J. Song, A. K. Mok, D. Chen, and M. Nixon,
‘‘Challenges of wireless control in process
industry,’’ presented at the CRTES, 2006.
[12] WirelessHART Architecture. [Online].
Available: http://en.hartcomm.org/hcp/tech/
wihart/images/Expanded_WirelessHART_
Mesh_Network.jpg.
[13] Siemens, ‘‘Process instrumentation and
weighing technology.’’ [Online]. Available:
http://www.industry.usa.siemens.com/
automation/us/en/process-instrumentation-
and-analytics/process-instrumentation/
brochures/Documents/PIBR-0A940-1014-
one-stop-shop.pdf.
[14] HART Communication Foundation,
‘‘Mitsubishi chemical connects HART
technology to control room savings up to
/20–30,000 a day.’’ [Online]. Available:
http://en.hartcomm.org/hcp/tech/
applications/applications_success_
mitsubishi_chemical.html.
[15] M. Sha, D. Gunatilaka, C. Wu, and C. Lu,
‘‘Implementation and experimentation of
industrial wireless sensor-actuator network
protocols,’’ presented at the EWSN, 2015.
[16] TinyOS. [Online]. Available: http://www.
tinyos.net/.
[17] Memsic, TelosB Mote Platform. [Online].
Available: http://www.memsic.com/userfiles/
files/Datasheets/WSN/telosb_datasheet.pdf.
[18] The WUSTL Wireless Sensor Network
Testbed. [Online]. Available: http://wsn.cse.
wustl.edu/index.php?title=Testbed.
[19] CC2420x Radio Driver. [Online]. Available:
https://github.com/tinyos/tinyos-main/tree/
master/tos/chips/cc2420x.
[20] M. Maróti, B. Kusy, G. Simon, and Á. Lédeczi,
‘‘The flooding time synchronization protocol,’’
in Proc. 2nd Int. Conf. Embed. Netw. Sensor
Syst., 2004, pp. 39–49.
[21] C. Wu, D. Gunatilaka, A. Saifullah, M. Sha,
P. Tiwari, C. Lu, and Y. Chen, ‘‘Maximizing
network lifetime of wireless sensor-actuator
networks under graph routing,’’ All Comput.
Sci. Eng. Res., Tech. Rep. WUCSE-2015-004,
(2015). [Online]. Available: http://openscho-
larship.wustl.edu/cse_research/508/
[22] C. Wu, D. Gunatilaka, M. Sha, C. Lu, and
Y. Chen, ‘‘Conflict-aware real-time routing for
industrial wireless sensor-actuator
networks,’’ All Comput. Sci. Eng. Res., Tech.
Rep. WUCSE-2015-005, (2015). [Online].
Available: http://openscholarship.wustl.edu/
cse_research/507/
[23] H. Li, P. Shenoy, and K. Ramamritham,
‘‘Scheduling messages with deadlines
in multi-hop real-time sensor networks,’’
presented at the RTAS, 2005.
[24] X. Wang, X. Wang, X. Fu, G. Xing, and
N. Jha, ‘‘Flow-based real-time communication
in multichannel wireless sensor networks,’’
presented at the EWSN, 2009.
[25] V. Kanodia, C. Li, A. Sabharwal, B. Sadeghi,
and E. Knightly, ‘‘Distributed multi-hop
scheduling and medium access with delay and
throughput constraints,’’ presented at the
MobiCom, 2001.
[26] C. Lu, B. M. Blum, T. F. Abdelzaher,
J. A. Stankovic, and T. He, ‘‘RAP: A real-time
communication architecture for large-scale
wireless sensor networks,’’ presented at the
RTAS, 2002.
[27] K. Karenos, V. Kalogeraki, and
S. V. Krishnamurthy, ‘‘A rate control
framework for supporting multiple classes of
traffic in sensor networks,’’ presented at the
RTSS, 2005.
[28] K. Karenos and V. Kalogeraki, ‘‘Real-time
traffic management in sensor networks,’’ in
Proc. RTSS, 2006, pp. 422–434.
[29] T. He, B. M. Blum, Q. Cao, J. A. Stankovic,
S. H. Son, and T. F. Abdelzaher, ‘‘Robust and
timely communication over highly dynamic
sensor networks,’’ Real-Time Syst., vol. 37,
no. 3, pp. 261–289, 2007.
[30] G.-S. Ahn, A. T. Campbell, A. Veres, and
L.-H. Sun, ‘‘SWAN: Service differentiation in
stateless wireless ad hoc networks,’’ in Proc.
INFOCOM, 2002, vol. 2, pp. 457–466.
[31] T. He, J. A. Stankovic, C. Lu, and
T. Abdelzaher, ‘‘SPEED: A stateless protocol
for real-time communication in sensor
networks,’’ presented at the ICDCS, 2003.
[32] E. Felemban, C.-G. Lee, and E. Ekici,
‘‘MMSPEED: Multipath multi-SPEED
protocol for QoS guarantee of reliability
and timeliness in wireless sensor networks,’’
IEEE Trans. Mobile Comput., vol. 5, no. 6,
pp. 738–754, Jun. 2006.
[33] B. D. Bui, R. Pellizzoni, M. Caccamo,
C. F. Cheah, and A. Tzakis, ‘‘Soft real-time
chains for multi-hop wireless ad-hoc
networks,’’ presented at the RTAS, 2007.
[34] Y. Gu, T. He, M. Lin, and J. Xu,
‘‘Spatiotemporal delay control for
low-duty-cycle sensor networks,’’ presented at
the RTSS, 2009.
[35] A. Koubaa, M. Alves, and E. Tovar, ‘‘i-GAME:
An implicit GTS allocation mechanism in
IEEE 802.15.4 for time-sensitive wireless
sensor networks,’’ presented at the ECRTS,
2006.
[36] M. Caccamo, L. Y. Zhang, L. Sha, and
G. Buttazzo, ‘‘An implicit prioritized
access protocol for wireless sensor networks,’’
presented at the RTSS, 2002.
[37] K. Liu, N. Abu-Ghazaleh, and K.-D. Kang,
‘‘JiTS: Just-in-time scheduling for real-time
sensor data dissemination,’’ presented at the
PerCom, 2006.
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
1022 Proceedings of the IEEE | Vol. 104, No. 5, May 2016
[38] B. Li, D. Wang, F. Wang, and Y. Q. Ni,
‘‘High quality sensor placement for shm
systems: Refocusing on application demands,’’
presented at the INFOCOM, 2010.
[39] R. Mangharam, A. Rowe, R. Rajkumar, and
R. Suzuki, ‘‘Voice over sensor networks,’’
presented at the RTSS, 2006.
[40] T. Facchinetti, L. Almeida, G. C. Buttazzo,
and C. Marchini, ‘‘Real-time resource
reservation protocol for wireless mobile
ad hoc networks,’’ presented at the RTSS,
2004.
[41] J. A. Stankovic, T. F. Abdelzaher, C. Lu, L. Sha,
and J. C. Hou, ‘‘Real-time communication
and coordination in embedded sensor
networks,’’ Proc. IEEE, vol. 91, no. 7,
pp. 1002–1022, Jul. 2003.
[42] O. Chipara, C. Lu, and G.-C. Roman,
‘‘Real-time query scheduling for wireless
sensor networks,’’ presented at the RTSS,
2007.
[43] T. F. Abdelzaher, S. Prabh, and R. Kiran,
‘‘On real-time capacity limits of multihop
wireless sensor networks,’’ presented at the
RTSS, 2004.
[44] P. Jurcı́k, R. Severino, A. Koubâa, M. Alves,
and E. Tovar, ‘‘Real-time communications
over cluster-tree sensor networks with mobile
sink behaviour,’’ presented at the RTCSA,
2008.
[45] J. B. Schmitt and U. Roedig, ‘‘Sensor network
calculusVA framework for worst case
analysis,’’ presented at the DCOSS, 2005.
[46] N. Pereira, B. Andersson, E. Tovar, and
A. Rowe, ‘‘Static-priority scheduling over
wireless networks with multiple broadcast
domains,’’ presented at the RTSS, 2007.
[47] H.-J. Korber, H. Wattar, and G. Scholl,
‘‘Modular wireless real-time sensor/actuator
network for factory automation applications,’’
IEEE Trans. Ind. Inform., vol. 3, no. 2,
pp. 111–119, May 2007.
[48] X. Liu and A. Goldsmith, ‘‘Cross-layer design
of distributed control over wireless network,’’
in Advances in Control, Communication
Networks, and Transportation Systems,
E. H. Abed, Ed. Berlin, Germany: Springer-
Verlag, 2005, pp. 111–136.
[49] X. Liu and A. Goldsmith, ‘‘Wireless network
design for distributed control,’’ presented at
the CDC, 2004.
[50] L. Xiao, M. Johansson, H. Hindi, S. Boyd, and
A. Goldsmith, ‘‘Joint optimization of wireless
communication and networked control
systems,’’ in Switching and Learning in
Feedback Systems, R. Murray-Smith and
R. Shorten, Eds. Berlin, Germany:
Springer-Verlag, 2005, pp. 248–272.
[51] T. W. Carley, M. A. Ba, R. Barua, and
D. B. Stewart, ‘‘Contention-free periodic
message scheduler medium access control in
wireless sensor/actuator networks,’’ presented
at the RTSS, 2003.
[52] A. Saifullah, S. Sankar, J. Liu, C. Lu,
R. Chandra, and B. Priyantha, ‘‘Capnet:
A real-time wireless management network
for data center power capping,’’ presented at
the RTSS, 2014.
[53] R. Alur, A. D’Innocenzo, K. H. Johansson,
G. J. Pappas, and G. Weiss, ‘‘Modeling and
analysis of multi-hop control networks,’’
presented at the RTAS, 2009.
[54] A. Saifullah, Y. Xu, C. Lu, and Y. Chen,
‘‘Real-time scheduling for WirelessHART
networks,’’ presented at the RTSS, 2010.
[55] A. Saifullah, Y. Xu, C. Lu, and Y. Chen,
‘‘End-to-end delay analysis for fixed
priority scheduling in wirelesshart networks,’’
presented at the RTAS, 2011.
[56] A. Saifullah, Y. Xu, C. Lu, and Y. Chen,
‘‘End-to-end communication delay analysis in
industrial wireless networks,’’ IEEE Trans.
Comput., vol. 64, no. 5, pp. 1361–1374,
May 2013.
[57] A. Saifullah, Y. Xu, C. Lu, and Y. Chen,
‘‘Priority assignment for real-time flows in
WirelessHART networks,’’ presented at the
ECRTS, 2011.
[58] P. Soldati, H. Zhang, and M. Johansson,
‘‘Deadline-constrained transmission
scheduling and data evacuation in
wirelesshart networks,’’ presented at the
ECC, 2009.
[59] C. Wu, M. Sha, D. Gunatilaka, A. Saifullah,
C. Lu, and Y. Chen, ‘‘Analysis of EDF
scheduling for wireless sensor-actuator
networks,’’ presented at the IWQoS, 2014.
[60] H. Zhang, F. Osterlind, P. Soldati, T. Voigt,
and M. Johansson, ‘‘Rapid convergecast on
commodity hardware: Performance limits and
optimal policies,’’ presented at the SECON,
2010.
[61] G. C. Butazzo, Hard Real-Time Computing
Systems, 2nd ed. Berlin, Germany:
Springer-Verlag, 2005.
[62] A. Saifullah, C. Wu, P. B. Tiwari, Y. Xu,
Y. Fu, C. Lu, and Y. Chen, ‘‘Near optimal rate
selection for wireless control systems,’’
ACM TECS, vol. 13, no. 4s, 2014.
[63] P. Khosla, ‘‘Choosing sampling rates for
robot control,’’ in Proc. ICRA, 1987, vol. 4,
pp. 169–174.
[64] H. Hirata and J. Powell, ‘‘Sample rate effects
on disturbance rejection for digital control
systems,’’ in Proc. ACC, 1990, pp. 1137–1145.
[65] D. Seto, J. P. Lehoczky, L. Sha, and K. G. Shin,
‘‘Trade-off analysis of real-time control
performance and schedulability,’’ Real-Time
Syst., vol. 21, pp. 199–217, 2001.
[66] A. Horch and A. J. Isaksson, ‘‘Assessment of
the sampling rate in control systems,’’ Contr.
Eng. Practice, vol. 9, no. 5, pp. 533–544.
[67] E. Bini and A. Cervin, ‘‘Delay-aware period
assignment in control systems,’’ presented at
the RTSS, 2008.
[68] D. Henriksson and A. Cervin, ‘‘Optimal
on-line sampling period assignment for
real-time control tasks based on plant state
information,’’ in Proc. CDC-ECC, 2005,
pp. 4469–4474.
[69] J. Bai, E. P. Eyisi, F. Qiu, Y. Xue, and
X. D. Koutsoukos, ‘‘Optimal cross-layer design
of sampling rate adaptation and network
scheduling for wireless networked control
systems,’’ presented at the ICCPS, 2012.
[70] A. Saifullah, C. Wu, P. B. Tiwari, Y. Xu,
Y. Fu, C. Lu, and Y. Chen, ‘‘Near optimal
rate selection for wireless control systems,’’
presented at the RTAS, 2012.
[71] Y. Wang, A. R. Swartz, J. P. Lynch, K. H. Law,
and C.-H. Loh, ‘‘Performance evaluation of
decentralized wireless sensing and control in
civil structures,’’ in Proc. SPIE, Nondestructive
Characterization Composite Mater., Aerosp.
Eng., Civil Infrastructure, Homeland Security,
vol. 6531, 2007, pp. 653113–653113.
[72] Y. Wang and K. H. Law, ‘‘Structural control
with multi-subnet wireless sensing feedback:
Experimental validation of time-delayed
decentralized H1 control design,’’ Adv.
Structural Eng., vol. 14, no. 1, pp. 25–39, 2011.
[73] J. P. Lynch, Y. Wang, A. R. Swartz, K. Lu, and
C. Loh, ‘‘Implementation of a closed-loop
structural control system using wireless
sensor networks,’’ Structural Contr. Health
Monitoring, vol. 15, no. 4, pp. 518–539, 2008.
[74] P. Tabuada, ‘‘Event-triggered real-time
scheduling of stabilizing control tasks,’’ IEEE
Trans. Autom. Control, vol. 52, no. 9,
pp. 1680–1685, Sep. 2007.
[75] M. Pajic, S. Sundaram, J. Le Ny, G. J. Pappas,
and R. Mangharam, ‘‘Closing the loop: A
simple distributed method for control over
wireless networks,’’ presented at the IPSN,
2012.
[76] A. Bemporad, S. Di Cairano, E. Henriksson,
and K. H. Johansson, ‘‘Hybrid model
predictive control based on wireless sensor
feedback: An experimental study,’’ Int. J.
Robust Nonlinear Contr., vol. 20, no. 2,
pp. 209–225, 2010.
[77] E. Eyisi, J. Bai, D. Riley, J. Weng, W. Yan,
Y. Xue, X. Koutsoukos, and J. Sztipanovits,
‘‘Ncswt: An integrated modeling and
simulation tool for networked control
systems,’’ Simulation Modelling Practice
Theory, vol. 27, pp. 90–111, 2012.
[78] B. Aminian, J. Araújo, M. Johansson, and
K. H. Johansson, ‘‘GISOO: A virtual
testbed for wireless cyber-physical systems,’’
presented at the IECON, 2013.
[79] Wireless Cyber Physical Simulator. [Online].
Available: http://wcps.cse.wustl.edu.
[80] P. Levis, N. Lee, M. Welsh, and D. Culler,
‘‘TOSSIM: Accurate and scalable simulation
of entire TinyOS applications,’’ presented at
the Sensys, 2003.
[81] H. Lee, A. Cerpa, and P. Levis, ‘‘Improving
wireless simulation through noise modeling,’’
presented at the IPSN, 2007.
[82] B. Li, L. Nie, C. Wu, H. Gonzalez, and C. Lu,
‘‘Incorporating emergency alarms in reliable
wireless process control,’’ in ICCPS, 2015.
[83] B. Li, Z. Sun, K. Mechitov, G. Hackmann,
C. Lu, S. J. Dyke, G. Agha, and
B. F. Spencer, Jr., ‘‘Realistic case studies of
wireless structural control,’’ in Proc. ICCPS,
2013, pp. 179–188.
[84] Z. Sun, B. Li, S. Dyke, and C. Lu, ‘‘Evaluation
of performances of structural control
benchmark problem with time delays from
wireless sensor network,’’ presented at the
EMI/PMC, 2012.
[85] Z. Sun, B. Li, S. Dyke, and C. Lu, ‘‘A novel
data utilization and control strategy for
wireless structural control systems with tdma
network,’’ presented at the ASCE IWCCE,
2013.
[86] Z. Sun, B. Li, S. J. Dyke, C. Lu, and
L. Linderman, ‘‘Benchmark problem in active
structural control with wireless sensor
network,’’ Structural Contr. Health Monitoring,
2015. [Online]. Available: http://dx.doi.org/
10.1002/stc.1761.
[87] A. Saifullah, C. Lu, J. Liu, R. Chandra, and
S. Sankar, ‘‘Challenge: Wireless Sensor
Networking Over White Spaces,’’ Microsoft
Research, Tech. Rep. MSR-TR-2014-6, 2014.
[Online]. Available: http://research.
microsoft.com/pubs/208183/
whitespace_vision.pdf
[88] S. Yin, H. Luo, and S. X. Ding, ‘‘Real-time
implementation of fault-tolerant control
systems with performance optimization,’’
IEEE Trans. Ind. Inform., vol. 61, no. 5,
pp. 2402–2411, May 2014.
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
Vol. 104, No. 5, May 2016 | Proceedings of the IEEE 1023
ABOUT THE AUT HORS
Chenyang Lu (Senior Member, IEEE) received the
B.S. degree from the University of Science and
Technology of China, Hefei, China, the M.S. degree
from the Chinese Academy of Sciences, Beijing,
China, and the Ph.D. degree from University of
Virginia, Charlottesville, VA, USA, in 1995, 1997,
and 2001, respectively.
He is currently a Fullgraf Professor in the
Department of Computer Science and Engineering
at Washington University, St. Louis, MO, USA. His
research interests include real-time systems, wireless sensor networks,
cyber-physical systems, and Internet of Things. He is the author and
coauthor of over 150 research papers with over 14 000 citations and an
h-index of 52.
Dr. Lu is Editor-in-Chief of the ACM Transactions on Sensor Networks,
an Area Editor of the IEEE INTERNET OF THINGS JOURNAL, and an Associate
Editor of the new ACM Transactions on Cyber-Physical Systems and Real-
Time Systems. He also chaired premier conferences such as the IEEE
Real-Time Systems Symposium (RTSS), the ACM/IEEE International
Conference on Cyber-Physical Systems (ICCPS), and the ACM Conference
on Embedded Networked Sensor Systems (SenSys).
Abusayeed Saifullah (Member, IEEE) received
the Ph.D. degree from Washington University,
St. Louis, MO, USA, in 2014.
He is an Assistant Professor in Computer
Science at the Missouri University of Science and
Technology, Rolla, MO, USA. His research interests
include cyber-physical systems with contributions
spanning real-time systems, embedded systems,
wireless sensor networks, and parallel and
distributed computing.
Dr. Saifullah received the Turner Dissertation Award 2014 of
Washington University, the Best Paper Award at RTSS’14, and the Best
Student Paper Award at RTSS’11.
Bo Li received the bachelor’s and master’s
degrees in electrical engineering from the Harbin
Institute of Technology, Harbin, China, in 2006
and 2008, respectively. He is currently working
toward the Ph.D. degree in the Department of
Computer Science and Engineering at Washington
University, St. Louis, MO, USA.
Between 2008 and 2010, he was a Research
Assistant at the Hong Kong Polytechnic University.
His research interests include wireless system
research, e.g., wireless process control, wireless structural health
monitoring and control, and optimization.
Mo Sha (Member, IEEE) received the B.Eng.
degree from Beihang University, Beijing, China,
the M.Phil. degree from the City University of Hong
Kong, Kowloon Tong, Hong Kong, and the Ph.D.
degree in computer science from Washington
University, St. Louis, MO, USA, in 2007, 2009,
and 2014, respectively .
He is an Assistant Professor in the Department
of Computer Science at Binghamton University-
The State University of New York, Binghamton,
NY, SA. His research interests include cyber-physical systems, wireless
sensor networks, embedded systems, as well as their applications to
smart energy, smart home, wireless health, and industrial process
automation.
Humberto Gonzalez (Member, IEEE) received the
B.S. and M.S. degrees in electrical engineering
from Universidad de Chile, Santiago, Chile, in
2005, and the Ph.D. degree in electrical engineer-
ing and computer sciences from the University of
California at Berkeley, Berkeley, CA, USA, in 2012.
He is currently an Assistant Professor in the
Department of Electrical and Systems Engineering
at Washington University, St. Louis, MO, USA. His
research interests include the theory and imple-
mentation of control algorithms for cyber-physical systems, with an
emphasis on optimal control and numerical algorithms.
Dolvara Gunatilaka received the B.S. degree in
information and communication technology, and
the M.S. degree in computer science from Mahidol
University, Thailand, in 2010 and 2013, respectively.
She is currently working toward the Ph.D. degree in
the Department of Computer Science and Engineer-
ing at Washington University, St. Louis, MO, USA.
Her research interests include industrial wireless
sensor-actuator networks.
Chengjie Wu received the B.S. and M.S. degrees
from Tsinghua University, Beijing, China, in 2006
and 2008, respectively, and the Ph.D. degree in
computer science from Washington University,
St. Louis, MO, USA, in 2014.
He is a Software Engineer at Yahoo!. His
research interests include real-time systems,
cyber-physical systems, wireless sensor networks,
and Internet of Things.
Dr. Wu was a recipient of the McDonnell
International Scholarship and the Emerson Corporate Fellowship
awarded by Washington University in St. Louis. He received the Best
Paper Nomination at the RTAS’12.
Lanshun Nie received the B.S., M.S., and Ph.D.
degrees from the Harbin Institute of Technology,
Harbin, China.
He was a Visiting Scholar of the CPS Laboratory
at Washington University, St. Louis, MO, USA. He is
currently an Associate Professor of Computer
Science at the School of Computer Science and
Technology, Harbin Institute of Technology, China.
His research interests include cyber-physical
systems and wireless sensor networks.
Yixin Chen (Senior Member, IEEE) received B.S.
degree from University of Science and Technology of
China, Hefei, China, in 1999, and the M.S. and Ph.D.
degrees fromUniversity of IllinoisUrbana-Champaign,
Urbana, IL, USA, in 2001 and 2005, respectively.
He is an Associate Professor of Computer
Science at the Washington University in St. Louis.
His research interests include data mining, ma-
chine learning, artificial intelligence, and optimi-
zation. He received the Best Student Paper
Runner-up Award at the KDD’14, and Best Paper Awards at AAAI’10
and ICTAI’05. He has won First Prizes in the International Planning
Competitions (2004 & 2006). He received an Early Career Principal
Investigator Award from the Department of Energy (2006) and a
Microsoft Research New Faculty Fellowship (2007).
Dr. Chen is an Associate Editor for ACM Transactions of Intelligent
Systems and Technology and serves on the Editorial Board of Journal of
Artificial Intelligence Research.
Lu et al. : Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems
1024 Proceedings of the IEEE | Vol. 104, No. 5, May 2016
<<
/ASCII85EncodePages false
/AllowTransparency false
/AutoPositionEPSFiles false
/AutoRotatePages /None
/Binding /Left
/CalGrayProfile (Gray Gamma 2.2)
/CalRGBProfile (sRGB IEC61966-2.1)
/CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2)
/sRGBProfile (sRGB IEC61966-2.1)
/CannotEmbedFontPolicy /Warning
/CompatibilityLevel 1.4
/CompressObjects /Off
/CompressPages true
/ConvertImagesToIndexed true
/PassThroughJPEGImages true
/CreateJDFFile false
/CreateJobTicket false
/DefaultRenderingIntent /Default
/DetectBlends true
/DetectCurves 0.0000
/ColorConversionStrategy /LeaveColorUnchanged
/DoThumbnails true
/EmbedAllFonts true
/EmbedOpenType false
/ParseICCProfilesInComments true
/EmbedJobOptions true
/DSCReportingLevel 0
/EmitDSCWarnings false
/EndPage -1
/ImageMemory 1048576
/LockDistillerParams true
/MaxSubsetPct 100
/Optimize true
/OPM 1
/ParseDSCComments false
/ParseDSCCommentsForDocInfo true
/PreserveCopyPage true
/PreserveDICMYKValues false
/PreserveEPSInfo true
/PreserveFlatness true
/PreserveHalftoneInfo true
/PreserveOPIComments false
/PreserveOverprintSettings true
/StartPage 1
/SubsetFonts true
/TransferFunctionInfo /Remove
/UCRandBGInfo /Preserve
/UsePrologue false
/ColorSettingsFile ()
/AlwaysEmbed [ true
]
/NeverEmbed [ true
]
/AntiAliasColorImages false
/CropColorImages true
/ColorImageMinResolution 300
/ColorImageMinResolutionPolicy /OK
/DownsampleColorImages true
/ColorImageDownsampleType /Bicubic
/ColorImageResolution 300
/ColorImageDepth -1
/ColorImageMinDownsampleDepth 1
/ColorImageDownsampleThreshold 1.50000
/EncodeColorImages true
/ColorImageFilter /DCTEncode
/AutoFilterColorImages false
/ColorImageAutoFilterStrategy /JPEG
/ColorACSImageDict <<
/QFactor 0.15
/HSamples [1 1 1 1] /VSamples [1 1 1 1]
>>
/ColorImageDict <<
/QFactor 0.76
/HSamples [2 1 1 2] /VSamples [2 1 1 2]
>>
/JPEG2000ColorACSImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/JPEG2000ColorImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/AntiAliasGrayImages false
/CropGrayImages true
/GrayImageMinResolution 300
/GrayImageMinResolutionPolicy /OK
/DownsampleGrayImages true
/GrayImageDownsampleType /Bicubic
/GrayImageResolution 300
/GrayImageDepth -1
/GrayImageMinDownsampleDepth 2
/GrayImageDownsampleThreshold 1.50000
/EncodeGrayImages true
/GrayImageFilter /DCTEncode
/AutoFilterGrayImages false
/GrayImageAutoFilterStrategy /JPEG
/GrayACSImageDict <<
/QFactor 0.15
/HSamples [1 1 1 1] /VSamples [1 1 1 1]
>>
/GrayImageDict <<
/QFactor 0.76
/HSamples [2 1 1 2] /VSamples [2 1 1 2]
>>
/JPEG2000GrayACSImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/JPEG2000GrayImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/AntiAliasMonoImages false
/CropMonoImages true
/MonoImageMinResolution 1200
/MonoImageMinResolutionPolicy /OK
/DownsampleMonoImages true
/MonoImageDownsampleType /Bicubic
/MonoImageResolution 600
/MonoImageDepth -1
/MonoImageDownsampleThreshold 1.50000
/EncodeMonoImages true
/MonoImageFilter /CCITTFaxEncode
/MonoImageDict <<
/K -1
>>
/AllowPSXObjects false
/CheckCompliance [
/None
]
/PDFX1aCheck false
/PDFX3Check false
/PDFXCompliantPDFOnly false
/PDFXNoTrimBoxError true
/PDFXTrimBoxToMediaBoxOffset [
0.00000
0.00000
0.00000
0.00000
]
/PDFXSetBleedBoxToMediaBox true
/PDFXBleedBoxToTrimBoxOffset [
0.00000
0.00000
0.00000
0.00000
]
/PDFXOutputIntentProfile (None)
/PDFXOutputConditionIdentifier ()
/PDFXOutputCondition ()
/PDFXRegistryName ()
/PDFXTrapped /False
/Description <<
/CHS
/CHT
/DAN
/DEU
/ESP
/FRA
/ITA
/JPN
/KOR
/NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.)
/NOR
/PTB
/SUO
/SVE
/ENU (Use these settings to create Adobe PDF documents best suited for high-quality prepress printing. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.)
>>
/Namespace [
(Adobe)
(Common)
(1.0)
]
/OtherNamespaces [
<<
/AsReaderSpreads false
/CropImagesToFrames true
/ErrorControl /WarnAndContinue
/FlattenerIgnoreSpreadOverrides false
/IncludeGuidesGrids false
/IncludeNonPrinting false
/IncludeSlug false
/Namespace [
(Adobe)
(InDesign)
(4.0)
]
/OmitPlacedBitmaps false
/OmitPlacedEPS false
/OmitPlacedPDF false
/SimulateOverprint /Legacy
>>
<<
/AddBleedMarks false
/AddColorBars false
/AddCropMarks false
/AddPageInfo false
/AddRegMarks false
/ConvertColors /ConvertToCMYK
/DestinationProfileName ()
/DestinationProfileSelector /DocumentCMYK
/Downsample16BitImages true
/FlattenerPreset <<
/PresetSelector /MediumResolution
>>
/FormElements false
/GenerateStructure false
/IncludeBookmarks false
/IncludeHyperlinks false
/IncludeInteractive false
/IncludeLayers false
/IncludeProfiles false
/MultimediaHandling /UseObjectSettings
/Namespace [
(Adobe)
(CreativeSuite)
(2.0)
]
/PDFXOutputIntentProfileSelector /DocumentCMYK
/PreserveEditing true
/UntaggedCMYKHandling /LeaveUntagged
/UntaggedRGBHandling /UseDocumentProfile
/UseDocumentBleed false
>>
]
>> setdistillerparams
<<
/HWResolution [600 600]
/PageSize [612.000 792.000]
>> setpagedevice