Current and Future IoT Trends
Syllabus
This module will cover the following
• Current state of IoT landscape
• Machine learning
• Edge computing
• Privacy: issues and PIAs
• Cloud security: Platform Security Architecture
• Research topic example: Federated Learning
2 © 2020 Arm Limited
Current state of IoT landscape
Faster growth than previously anticipated
• Growth driven by expansion of connected consumer market, fast penetration of voice-controlled personal assistants (e.g., Alexa), and increase in mobile machine-to-machine connections
• Most businesses start with systems that serve a single application → rapid return of investment, but not maximizing full potential of IoT
• More investment anticipated in medical IoT, industrial IoT, and intelligent transportation (likely amplified by the pandemic)
• Evidence of security vulnerabilities continue to affect trust and limit adoption, motivating regulation and certification
12bn
Internet-connected devices at the end of 2020
3 © 2020 Arm Limited
Machine Learning
More value created from data
• Data still largely used for process monitoring and optimization
• Additional value created from forecasts based on IoT data
collected, e.g., in predictive maintenance (automotive IoT)
• Feeding data into the operation of other systems, e.g., interconnecting distribution and manufacturing (value chain)
• Increasing appetite to use machine learning (ML) and artificial intelligence (AI) to extract hidden knowledge from massive amounts of data collected by sensors
• Hardware is mature → new edge computing methods needed
5 © 2020 Arm Limited
Essential to build AI capabilities to increase the value of the data gathered
AI and ML
Teaching computers to solve problems and learn like humans
• AI paradigm aims to develop computing capabilities that mimic human cognitive functions
• Machine learning (ML) is a subset of AI by which artificial
processes learn from data and make decisions without having
been explicitly programmed
• Supervised learning: relies on training examples
• Unsupervised learning: discovers new patterns without human supervision • Reinforcement learning: agents learn actions in an environment to
maximize some reward
• Deep learning is a family of ML that seeks to mimic the biological nervous systems
6 © 2020 Arm Limited
Deep learning
Neural network architecture resembles the brain perception process in a brain, with specific neurons activated depending on the input and leading to some inference
Goal: Approximate complex functions through simple operations performed by layers of “neurons” (or units)
Examples: Classification (assigning labels to different input), regression (computing future time series values based on historical data), and control (board game moves)
Operations: Weighted combinations of groups of hidden units with a non-linear activation function Model weights learned by minimizing a loss function, through back-propagation of its gradient
7 © 2020 Arm Limited
Multi-layer Perceptron (MLP)
Densely connected layers
• Large number of weights
• Given an input x, a layer computes the following output:
y = 𝑎𝑐𝑡 𝑊 ⋅ x + 𝑏 , where 𝑊are a set of weights and 𝑏 are biases;
𝑎𝑐𝑡(⋅) is an activation function, e.g., • Sigmoid: 𝜎 x = ! x
!”#
• Rectified linear unit: ReLU x = max(x, 0)
8 © 2020 Arm Limited
Convolutional Neural Network (CNN)
Powerful in (image) classification tasks
•
Replace dense connections with filters (kernels) that share weights across small receptive fields
Pooling layers reduce the feature dimensions (avg/max of all values)
•
9
For each location 𝑝$, convolution computes
𝑦𝑝$ =0𝑤𝑝) ⋅𝑥(𝑝$+𝑝)), &!∈(
where 𝑝)are positions in the receptive field 𝐺 and 𝑤 the filter weights © 2020 Arm Limited
Edge Computing
The rise of edge computing
Deploying advanced computing, storage, and applications on devices
• Goals: Reduce application latency, conserve bandwidth, offload cloud computation, and improve privacy
• Advances in hardware:
• High-performance and power-efficient processors, e.g., Arm Cortex-A55
• Embedded Graphics Processing Units (GPUs), e.g. Arm Mali-G77 • Micro Neural Processing Units (NPUs), e.g., Arm Ethos-U55
• Nano solid-state drives (gigabytes of storage per mm2)
• Software libraries:
• Neural network kernels optimized for constrained CPUs, e.g., Cortex
Microcontroller Software Standard Neural Network (CMSIS NN)
• Lightweight ML inference frameworks, e.g., TensorFlow Light Micro
11 © 2020 Arm Limited
Micro Neural Processing Units
• Arm’s Ethos-U55 microNPU integrates with the Cortex-M toolchain
• Allows acceleration of neural networks in an low- area, with 90% lower power consumption
• Useful for AI applications in cost-sensitive devices
• MAC engine handles 16-bit Multiply-and- Accumulate instructions relevant to neural networks (e.g., convolutions).
• Supported by software library
12 © 2020 Arm Limited
CMSIS NN software library
Neural network kernels optimized for Cortex-M cores
• Utility functions can be used to construct more complex neural structures, e.g., Long Short-Term Memory (LSTM)
• Fixed-point quantization used
to reduce memory footprint
13 © 2020 Arm Limited
Tiny Machine Learning (TinyML)
An approach to machine learning at the edge
Goal: ML on low-power (< 1mW range) controllers
• Useful in battery powered IoT applications
• Only a few kilobytes of memory to run the ML model
• Focus on inferencing rather than training
• Processing raw sensor data at edge
• Applications: Recognizing speech commands, detecting vibration patterns, gesture recognition
Can use TensorFlow Lite, Google’s toolkit for TinyML
See https://tinymlbook.com/
TinyML
Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers
Pete Warden & Daniel Situnayake
15 © 2020 Arm Limited
TensorFlow Lite Micro
Machine learning at the edge
• Merger of TensorFlow Lite and Arm microTensor
• TensorFlow Lite – Open-source deep learning framework for on-device inference
• uTensor – Open-source framework for converting ML models to self-contained C++ source files, to enable deployment on embedded devices
16 © 2020 Arm Limited
Privacy in IoT
The “Internet of Stool Pigeons” and other concerns
As IoT moved outside pure-industry settings to monitor people and their devices, personal privacy issues arose quickly.
• Location data, device monitoring
• Biometrics of all kinds
• Smart CCTV: images, conversations
• Consumer appliance, vehicle behaviours
All rich sources of Personally Identifiable
Information being combined in a “world-
spanning information fabric.”
https://www.theguardian.com/technology/2015/mar/11/internet-of- 19 things-hacked-online-perils-future
Data linkage: the power of data sources in combination
20
Data in isolation can be safe (or even meaningless). Combined with other data, background knowledge and algorithms, rich new information may be learned.
2007: Netflix releases viewing data recommender challenge.
Researchers de-anonymize entries,correlating with Internet Movie Database.
Revealed private watching habits.
2014: New York City Taxi dataset 2018: Strava heatmaps exposed
released to study traffic patterns, congestion.
military training locations and patterns.
Flawed anonymization, revealed Correlations with GPS-tagged driver identities. Incomes calculated. photos, social media.
Paparazzi pics: celebrity journeys, finding homes, offices.
Six Principles for Privacy of Personal Data
The EU GDPR (2018) and UK Data Protection Act (2018 with UK-GDPR 2021 update) are designed to enshrine 6 core principles which organisations must follow:
1. Lawfulness, Fairness, and Transparency
2. Limitations on Purposes of Collection, Processing, and Storage
3. Data Minimization
4. Accuracy of Data
5. Data Storage Limits
6. Integrity and Confidentiality
Moreover, there is a duty of accountability to demonstrate adherence to these. 21
Privacy Impact Assessments
Attachment to smartphone application
Parameter Description/Sensitivity Origin Consumer(s)
Account Account number that was created by number the account server upon doll owner
Account server via doll owner
Smartphone application
A PIA considers:
account creation
Application server
(moderate sensitivity)
Doll serial Unique identifier for the doll Doll's Doll owner
• range of personal data collected, formats and sensitivity levels number (not sensitive) packaging from Application
Doll settings Day-to-day settings and Doll owner Doll
• risks of data breach, information misuse
a
nd
co
nfi
gs
c
the smartphone application or web client
onfi
gur
a
tion
sm
a
de
on
t
he
d
oll
vi
a
Application server
• redress: procedures in case of violations of privacy not sensitive, or moderate sensitivity
(depending on attributes)
• auditing and accountability
The following example information is identified as being created or consumed
during the normal daily use of the talking doll:
manufacturer server Smartphone
• intended and acceptable uses, including (cloud) storage, processing, sharing app
Chapter 7
Daily usage
Parameter
Description/Sensitivity
Origin
Consumer
Doll speech profiles
Downloadable speech patterns and behaviors
(not sensitive)
Application server
Doll user
Doll microphone data (voice recordings)
Recorded voice communication with doll
(high sensitivity)
Doll and environment
Application server and doll owner via smartphone
Transcribed Microphone Data
Derived voice-to-text transcriptions of voice communication with doll
(high sensitivity)
Application server (transcription engine)
Application server and doll owner via smartphone
22
Example PIA for a speaking doll: part of data identification process lifecycle.
See Practical Internet of Things Security, Russell and Van Duren, Packt 2016.
Outlook: PETs and PbD
Privacy Enhancing Technologies (PETs) are a range of emerging methods to help solve privacy problems by allowing privacy-limited sharing of data. Well-known ones are:
• Differential Privacy
• Secure Multi-Party Computation
• Fully-Homomorphic Encryption
Privacy By Design (PbD) encourages building privacy in from start
• Take view of whole system from start
• Ensure that policies and law can be satisfied, end-to-end
• Demonstrate embodiment of DP principles in implementation
Hope is that PETs and good design will enable privacy without loss of utility.
23
Cloud and Platform Security
endpoint data transactions. It shows typical, virtualized services available for general
IT as well as IoT-enabled deployments. Not all IoT deployers will need to make use of all the cloud capabilities available, but most will require a minimal cross-section of the above services, and require them to be well protected:
IoT Cloud Security Architecture
Data Warehousing
Secure Storage & Access
Operations Mgmt
Identity & Access Mgmt
Authentication Services
API Logging
Monitoring
{Indexing Svcs}
Search
Media
Event Data Services & Storage
Large Data Set Processing (e.g MapReduce)
NoSQL Large Table Data
Storage Services
Upload
Download
Logs
App Provisioning
Scaling & Dynamic Response
Real-Time Streaming Analytics Service
Data
Data Streaming Services
Data
DNS Services
Content Delivery Network & Services
Cloud API Gateway
Notification Services
Key Management
Internet
General architecture, virtualised services.
1. Threat model to categorize IoT
devices, data, endpoints.
2. Design security architecture,
service provider + add-ons.
Web
Sensors
Controllers
Mobile
Virtual Private Cloud Trust Boundary
25
Picture credit: Practical Internet of Things Security, Russell and Van Duren, Packt 2016.
End User IoT Vendors
Data
Motivating a Platform Security Architecture
Security challenges
Success of the IoT depends on the trust, security and privacy built into connected devices Security can be expensive to implement and there is a shortage of experts
Difficult to manage secure devices at scale
Lack of confidence in data to/from sensors/actuators
New vulnerabilities appear all the time
26 © 2020 Arm Limited
Arm’s Platform Security Architecture (PSA)
Framework for securing connected devices
1. Analyze: Threat Models and Security Analyses, derived from IoT use cases.
2. Architect: Specifications for FW and HW.
3. Implement: Open source reference implementation of FW architecture.
4. Certify: PSA Certified scheme – independent evaluation
27 © 2020 Arm Limited
Platform Security Architecture (PSA)
Phase 1: Analyze with threat models and security analyses
Example: smart meter
• Threat models created using an English Language Protection Profile-style approach, to establish a set of Security Functional Requirements (SFR) for Target of Evaluation (TOE)
• Each profile considers the functional description, the TOE, and the security requirements
• Documentation makes threat modeling more useable by engineers, regardless of prior security expertise.
28 © 2020 Arm Limited
Platform Security Architecture (PSA)
Phase 2: Architect with architecture specifications
PSA Security Model (PSA-SM) – Foundational trust models and patterns
Factory Initialization (PSA-FI) – Requirements for initial secure device programming and configuration
Trusted Base System Architecture (TBSA-M) – Hardware platform requirements
Trusted Boot and Firmware Update (TBFU) – System and FW technical requirements for ensuring MCU boot integrity
Firmware Framework (PSA-FF) – Firmware interface definition of a Secure Processing Environment (SPE) for constrained IoT platforms, including PSA Root of Trust APIs
Developer APIs – Interfaces to security services for application developers
29 © 2020 Arm Limited
Platform Security Architecture (PSA)
Phase 3: Implement with Trusted Firmware-M (TF-M)
TF-M is an open source, open governance project, providing:
• Bootloader for authenticated boot
• Implementation of PSA Firmware Framework
• Secure Services for Storage, Crypto, Attestation, etc.
• Multiple OS support
30 © 2020 Arm Limited
Platform Security Architecture (PSA)
Phase 4: Certify with PSA Certified and PSA Functional API Certification
• Enables testing IoT chipsets and devices to be tested in laboratory conditions, to evaluate their level of security
• Multi-level assurance for devices, depending on the security requirements established through analysis of threats for a specific use case.
• Three progressive levels of security certification: foundation, lab-based evaluation, and extensive attacks testing
31 © 2020 Arm Limited
Federated Learning
Research topic: Federated learning
Training neural models across decentralized edge devices
• Training global model on local data samples
• Edge nodes exchanging parameter updates with central server
• Server aggregates updates and refines model
33 © 2020 Arm Limited
Research topic: Federated learning
Challenges
How to reduce communication overhead between central server and edge devices? How to handle conflicting parameter updates?
How to ensure model updating is robust to link failures?
How to ensure global model updates are not influenced by malicious updates? How to personalize model for location-specific inference?
34 © 2020 Arm Limited
Neural model compression and acceleration
• Inference performance tends to grow with neural model “depth” (number of layers)
• This also increases memory requirements and poses challenges to deployment on IoT
devices → model compression becomes necessary
• Parameter pruning: Remove model parameters that are not critical to performance
• Knowledge distillation: Train a compact model to behave like a large network (teacher)
• Convolution operations are computationally expensive → certain applications (e.g., autonomous vehicles, robotics) require real-time recognition
• Factorization of convolutional kernels: Decomposition of matrices into products of smaller ones to speed up inference
35 © 2020 Arm Limited
Model pruning
Supported in hardware to improve efficiency
• Many weights have small values after training (low importance), hence can be removed (synapse pruning)
• Hidden units with no input connections can be removed (neuron pruning)
• Retraining necessary to preserve accuracy
• Pruning usually an iterative process
36 © 2020 Arm Limited
Summary
A few of the current trends:
• Current state of IoT landscape
• Machine learning
• Edge computing
• Platform Security Architecture
• Privacy
• Research in Federated Learning
37 © 2020 Arm Limited