ACS6124 Multisensor and Decision Systems Part I: Multisensor Systems Lecture1: Introduction
George Konstantopoulos g.konstantopoulos@sheffield.ac.uk
Automatic Control and Systems Engineering The University of Sheffield
(lecture notes produced by Dr. Inaki Esnaola and Prof. Visakan Kadirkamanathan)
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Module plan and Assessment
Time Table
Lectures: 16 hours (pre-recorded lectures + synchronous lectures)
Laboratory: 6 hrs
Tutorials: 2 hrs
Assessment
Assignments (100%) – 2 Reports, one for Multisensor Systems Lab and one for Decision Systems Lab (each 50%)
Resources
Lecture Slides
Laboratory Sheets
Tutorial Questions and Answers
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Module contents
1
2 3 4 5 6 7
Introduction to Multisensor Systems Statistical Decision Theory
Sensor Estimation
Multisensor Decision Systems Multisensor Fusion Estimation Dynamic Decision Systems
Case Study
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Recommended books
No core textbooks. Suggested reading from the following sources:
Yaakov Bar-Shalom, X. Rong Li and Thiagalingam Kirubarajan. Estimation with Applications to Tracking and Navigation, John Wiley, 2001.
Rolf Iserman. Fault Diagnosis Systems: An Introduction from Fault Detection to Fault Tolerance, Springer, 2006.
Fredrik Gustaffson. Adaptive filtering and change detection, Wiley, 2000.
Hugh Durrant-Whyte. Multi Sensor Data Fusion, Online Notes, 2001.
Pierre Moulin and Venugopal Veeravalli. Statistical Inference for Engineers and Data Scientists, Cambridge University Press, 2019.
1
2
3
4
5
ACS6124 Multisensor and Decision Systems
G. Konstantopoulos
The Importance of Sensors
Sensors provide information about the state of a system of interest and of the environment.
Operating the system with desirable behaviour requires control of the system.
Open loop control suffers from uncertainty and
changes in the environment.
Detecting the system degradations avoids cost of catastrophic failure.
Symptoms of failure are not easily detectable and impacts multiple system states differently.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Monitoring Aircraft Engines
Aircraft engines are complex systems composed of highly advanced mechanical engineering systems and tightly coupled electronic control systems that operate in harsh environments.
Need to design aircraft engines for high reliability and performance.
Engine performance degradations due to wear, fatigue. Multisensor and decision systems are required in support of:
Estimating unmeasured variables by sensor fusion to control and optimise performance.
Monitoring detection of changes in performance. Diagnosis of faults function for health monitoring system. Modelling, analysis, testing and optimisation for design.
The need for multisensor fusion and decision systems are needed with Increasing number of sensors
Increasing complexity of decisions
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Autonomous Driving Vehicles
Modern cars are increasingly being made autonomous with additional sensors being mounted on the vehicles and the information processed to control the vehicle navigation.
www.millionstartups.com
Driver assistance functionalities include,
Autonomous parking.
Cruise control and vehicle platooning.
Collision avoidance and emergency braking. Cross traffic alert, lane control and manoeuvering.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Industry 4.0
Industry 4.0 is considered to be the next generation industrial revolution that is driven by the internet and data analytics. Micro-electro-mechanical sensors (MEMS) report on environmental conditions such as temperature, light intensity, chemical concentrations and humidity.
Bosch: RoMulus Project
The vision for Industry 4.0 include,
Each product has its blueprint reports its manufacturing status. Each machine reports its health status and machine parameters.
Self-organised manufacturing process adapting to environmental state to reduce product variations.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
HMI for Assisted Living
Human machine interfaces (HMI) are critical for patients with disabilities requiring assisted living. An example of such an HMI is activity recognition based on hand gestures. Sensors used for detection and classification are inertial and electromyography (EMG) signals from body sensors, optical sensing from cameras.
Other applications that also require multisensor fusion and decision are: Patient bedside healthcare monitoring.
Fitness and activity monitoring.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Smart Home
Smart homes initiatives are made possible with multisensors and wireless connectivity (Internet of Things). Smart meters, heating and cooling, motion sensors, temperature and lighting and integration with smart appliances make the required infrastructure.
Some functionalities beyond home comfort are: Status estimation for security.
Control and optimisation for energy efficiency.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Environmental Monitoring
Monitoring of environmental hazards and pollution is being carried out with combining information from multiple sensors: Satelite imagery, camera based optical sensing, stationary and drone based chemical sensing etc.
The spatial extent poses additional challenges in environmental monitoring, in addition to:
Information received at different rates.
Combining the information and stationary and mobile sensors.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Example Multisensor Systems
E-compass
Magnetometer: x-, y- and z-axis magnetic field sensing – sensitive to
magnetic interference
Accelerometer: x-, y- and z-axis linear motion sensing – sensitive to vibration
Inertial measurement unit (IMU)
Accelerometer: x-, y- and z-axis linear motion sensing – sensitive to
vibration
Gyroscope: pitch, roll and yaw rotational sensing – zero bias drift Magnetometer: x-, y- and z-axis magnetic field sensing – sensitive to magnetic interference
Navigation System
Inertial measurement unit (IMU): 3-D pose and position
Global Positioning System (GPS): x-, y-, z- position coordinates – time delay, resolution accuracy
Odometer: Wheel speed estimation along path – sensitive to wheel slip
Environment sensor Temperature
Pressure Humidity
Sensor fusion takes the simultaneous input from the multiple sensors, processes the input and creates an output that is greater than the sum of its parts
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Inertial Navigation System (INS)
Self-contained, nonradiating, nonjammable, dead-reckoning navigation systems. Provide dynamic information through direct measurements.
Inertial Measurement Unit (IMU) is typically used in INS, which consists of:
Accelerometers provide velocity rate (linear acceleration) information.
Gyroscopes provide angular rate information.
Sensors used for providing information about orientation, position and velocity of a vehicle, robot, drone or aircraft.
Challenge: angular rate and velocity rates have to be integrated which can lead to unbounded growth of estimation errors even when with small sensor bias and drifts.
Periodic reset of IMUs using absolute sensing mechanisms.
Use of beacons and landmarks with known positions for indoor robotics applications.
Global positioning systems (GPS) data for absolute position.
Need a system capable of providing low-cost, high-precision, short time-duration position information.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Sources of Sensor Errors
Rate Gyro sensor measures angular velocity ωg . Bias drift bg – modelled as constant Sensor noise wg – modelled as white noise
ωg = ω∗ + bg + wg
Magnetometer measures heading angle θc relative to Earth’s local magnetic field.
Local magnetic disturbance δc – modelled as impulses Tilt induced errors are avoided by ‘gimballing’ Reduced precision wc – modelled as white noise
θc = θ∗ + δc + wc
Odometer measures curvilinear distance so travelled by integrating rotational velocity vo.
Scale error as measurement depends on wheel diameter leads to drift bo – modelled as constant
Quantisation error wo due to pulse counting – modelled as white noise
vo = v∗ + bo + wo
Global positioning system (GPS) measures position coordinates sp.
Position uncertainty wp – modelled as white noise
Measurement outages at random – modelled as missing measurements
sp =s∗+wp
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Tilt Estimation
Example: Tilt of platform for sedgway, quadcopter, smartphone
Gyro sensor data: ωg (angular velocity) Tilt Angle estimate only from rate gyro:
θg=θ0+ ωgdt Accelerometer sensor data: a (linear
acceleration)
Tilt Angle estimate only from accelerometer:
a −1a
θ = sin g g – gravitational acceleration
Estimation from gyro alone will lead to drift in the long term. Estimation from accelerometer will be very noisy in the short term.
ˆ ˆ g a θk = α θk−1 +ωk ∆T +(1−α)θk
α – weight, ∆T – sampling interval, t – sampling time instant.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Tilt Estimation – Complimentary Filter
Rewrite estimator as:
ˆˆg aˆg θt = θt−1+ωt ∆T +(1−α) θt − θt−1+ωt ∆T
Term 1 in RHS consists of low frequency noise from gyro estimation.
Term 2 in RHS consists of both low frequency noise from gyro and high frequency noise from accelerometer estimation.
Low pass filtering Term 2 leaves low frequency noise that can be used to cancel noise in Term 1.
ˆˆ g −1aˆ g θt = θt−1+ωt ∆T +HLP(z ) θt − θt−1+ωt ∆T
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Location Estimation
Example: Location estimation of vehicles
Target object position, velocity and acceleration are:
st, vt, at.
The sensor measurements are the acceleration from accelerometer and velocity from odometer.
yo 0 1 0 st vo
ta = vt+ ta yt 001 a vt
t
Global positioning system (GPS) uses four satelites to estimate the location.
[sx , sy , sz ] – vehicle location
[sxi ,syi ,szi ] – Known satelite position d – offset due to receiver clock error vi – random noise and uncertainties.
Measured pseudo range from satelite i is given by,
yi =(sx−sxi)2+(sy−syi)2+(sz−szi)2+d +vi =hi(sx,sy,sz,d)+vi tttttttttttttt
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Location Estimation – Complimentary Filter
Example: Location estimation of vehicles
Inertial navigation system (INS) – Position, velocity, pose [Fast update 1 kHz] Global positioning system (GPS) – Position [Slow update 1-50 Hz] Integration of INS and GPS
Complementary filter solution:
GPS measurement outages means missing measurements.
INS estimate used in absence of GPS measurements.
When GPS measurement available, corrects for drift of INS measurement.
ˆs =ˆsINS+ˆε ttt
ˆεt is the INS error estimate obtained from Kalman filter using integrated INS+GPS error model.
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Why Probability Theory?
Information Source:
Alphabet: Set of states that the system is in.
Probability distribution over the states: Governs the output of states and models the underlying physical process
Uncertainty sources:
1 Thermal noise
2 Sensor errors
3 Communication errors
4 Interference
5 Missing data
6 Mismatched models
Probability theory deals with averages of phenomena Electron emission
Radar signals Biological data Statistical mechanics
Based on observation that averages of certain phenomena approach a constant value as the number of observations increases
Aim: To describe and predict such events and phenomena in terms of probability of events.
ACS6124 Multisensor and Decision Systems
G. Konstantopoulos
Definitions of Probability
Axiomatic Definition: Based on the following three postulates:
1 The probability P[A] of an event A is a non-negative number assigned to the event P[A] ≥ 0
2 The probability of the certain event, Ω, is P[Ω] = 1
3 If the events A and B are mutually exclusive
P[A∪B] = P[A]+P[B]
Relative Frequency Definition: The probability of an
event A is the limit P[A] = lim nA n→∞ n
Classical Definition: Consider a number N of possible
outcomes and a number NA of events that are favourable
to event A, then P[A] = NA N
ACS6124 Multisensor and Decision Systems
G. Konstantopoulos
Conditional Probability
Conditional probability forms basis for data fusion.
Definition of Conditional Probability
The conditional probability of an event A conditioned on another event M is
P[A|M] = P[A∩M] P[M ]
where P[M] > 0.
Properties:
1 If M ⊂ A then P[A|M] = 1
2 If A⊂M then P[A|M]≥P[A]
3 Conditional probabilities are probabilities (Check the
axiomatic definition).
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Bayes’ Theorem
A key tool for updating probabilities by including new observations in data fusion
Bayes’ Theorem
Given sets A and B, then P[A|B] = P[B|A] P[A] P[B]
Interpretation:
P[A|B] is the posterior probability: The resulting probability after all the evidence has been incorporated in the calculation of the probability
P[A] is the prior probability: The probability of the event before evidence has been incorporated. It can be interpreted as our belief about the event.
P[B|A] is the likelihood of the evidence: The probability of the evidence given that event A is true.
P[B] is the normalizing probability: The prior probability of the evidence guarantees that the resulting expression is a probability (Check the axiomatic definition).
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Total Probability Theorem
Total Probability Theorem
Let U = [A1,…,An] be a partition of Ω and B an arbitrary event, then
n P[B]=P[B|A1]P[A1]+…+P[B|An]P[An]= ∑P[B|Ai]P[Ai]
i=1
Revisiting Bayes’ Theorem
Let U = [A1,…,An] be a partition of Ω and B an arbitrary event, then
P[Ai |B] = P[B|Ai ] P[Ai ] P[B|A1]P[A1]+…+P[B|An]P[An]
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Independence
Definition of Independence
Events A and B are independent if P[A∩B]=P[A]P[B]
If A and B are independent, then events Ac and B, A and Bc, and Ac and Bc are also independent.
Generalization of Independence
Events A1,…,An are independent
P[A1 ∩A2 ∩…An] = P[A1]…P[An]
Exercise: What does Bayes’ Theorem reduce to for independent events?
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Random Variables
Probabilities are only assigned for events
Random variables allow the description of event in a consistent numerical framework
Definition of a Random Variable
A real random variable is a real function of the elements ofasamplespaceΩ,thatis,X :Ω→R
Notation:
The random variable is represented by a capital letter, e.g., X
Particular values of the random variable are represented by a lowercase letter, e.g., x
Conditions for a function to be a random variable:
The set {X ≤x} is an event for every x
The probabilities of the events {X = ∞} and {X = −∞} is equal to 0.
A discrete random variable takes values on a finite set
A continuous random variable has a continuous range of values
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Distribution Function
Definition of Distribution Function
The distribution function of the random variable X is the function
FX(x)=P[X ≤x] defined for x ∈ (−∞, ∞)
Properties:
FX(−∞)=0 and FX(∞)=1
FX (x ) is monotonically increasing in x
If FX (x0) = 0 then FX (x) = 0 for all x ≤ x0 P[X >x]=1−FX(x)
P[x1
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Note: In later sections, we will use P to refer to both pdfs (f ) and
Conditional Distribution
Recall the definition of conditional probability:
P[A|M] = P[A∩M] P[M ]
Definition of Conditional Distribution
The conditional distribution of a random variable X, assuming event M is given by
FX|M(x|M) = P[X ≤ x|M] = P[X ≤ x,M] P[M ]
Note that FX|M(x|M) satisfies the same properties as FX(x).
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Total Probability Theorem with Random Variables
Recall that the Total Probability Theorem states
P[B] = P[B|A1]P[A1]+…+P[B|An]P[An]
How can this be extended to random variables? Setting B = {X ≤ x} yields
P[X ≤ x] = P[X ≤ x|A1]P[A1]+…+P[X ≤ x|An]P[An] which by definition of conditional distribution results in
FX =FX|A1(x|A1)P[A1]+…+FX|An(x|A1)P[An] and taking derivatives
fX =fX|A1(x|A1)P[A1]+…+fX|An(x|A1)P[An] Total Probability Theorem
Given a random variable X and event A, the probability of
the event is given by P[A] = ∞ P[A|X = x]f (x)dx −∞ X
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Bayes’ Theorem with Random Variables
Bayes’ Theorem
Given a random variable X and event A, the following holds
fX|A =
P[A|X = x] P[A]
fX(x)=
P[A|X = x]fX (x) ∞−∞P[A|X = x]fX (x)dx
How is the a priori p.d.f., fX (x ), chosen? Uniform
Gaussian Maximum entropy
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Expected value
Definition of Expected Value of a Random Variable
The expected value or mean of a random variable X is
defined as
For discrete random variables it reduces to
E[X]=μX =∑PX(xi)xi i
Definition of Expected Value of a Function of a Random Variable
The expected value or mean of a random variable g(X) is
E[X]=μX =
∞ −∞
xfX(x)dx
defined as
∞
E[g(X)]= g(x)fX(x)dx −∞
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Variance and Moments
Definition of Variance
Let X be a random variable with mean μX , then its variance is defined as
22∞2
σX =E[(X−μX) ]=
This definition can be extended to
Moment:
E[(X) ]= Central Moment
E[(X−μX) ]=
(x−μX) fX(x)dx
p∞p
−∞
(x−μX) fX(x)dx
x fX(x)dx p∞p
−∞
−∞
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Examples of Discrete Random Variables
Definition of Bernoulli Distribution
Random variable X is Bernoulli distributed with parameter p, denoted X ∼ B(p), if
P[X =1]=p and P[X =0]=1−p Example: Sensor status as normal or failed
Definition of Binomial Distribution
Random variable X taking values in 0,1,…n is Binomial distributed with parameters n and p, if
P[X = k] = npk(1−p)n−k k = 1,2,…,n k
It represents the the probability of the total number of success in an independent trial of n Bernoulli realizations.
Example:ProbabG.iKliotnystaontfopgouelotstingAC3S6f1a24iMleuldtisesnseornansdoDrecsisiaonmSyostenmgsst
Examples of Continuous Random Variables
Definition of Uniform Distribution
Random variable X is Uniformly distributed in the interval (a,b), denoted X ∼ U (a,b), if
1,a≤x≤b fX(x)= b−a
Example: The angle of motion direction Definition of Gaussian Distribution
Random variable X is Gaussian distributed with mean μ and variance σ2, denoted X ∼ N (μ,σ2), if
fX (x) = √ 1 e−(x−μ)2/2σ2 2πσ2
The special case X ∼ N (0, 1) is often referred to as the standard Gaussian random variable.
Example: The thermal noise in electronic components
0, otherwise
G. Konstantopoulos
ACS6124 Multisensor and Decision Systems
Joint Distribution of a Bivariate Distribution
Given two random variable, X and Y , their joint statistics need to be determined.
Definition of Joint Distribution of Two Random Variables
The joint distribution or random variables X and Y is FXY(x,y)=P[X ≤x,Y ≤y]
Properties:
FXY (−∞,y) = 0, FXY (x,−∞) = 0, and FXY (∞,∞) = 1 P[x1