INTRODUCTION
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 1
DDIIGGIITTAALL DDAATTAA AACCQQUUIISSTTIIOONN
&& MMEEDDIIAA BBAASSIICCSS
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 2
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 3
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 4
LLEECCTTUURREE SSUUBBTTOOPPIICCSS
Issues in Digitizing a Signal
• Signal Sampling
• Quantization
• Bit Rate
What do you lose in the digitization process?
Why do you lose it?
What can you do to avoid (minimize) the loss.
Filtering and Subsampling
Acquisition of media and formats
Video Progressive and Interlaced
Digital Component Video Formats
Aspect ratios
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 5
EEXXAAMMPPLLEE SSIIGGNNAALLSS
Analog Signal Digital Signal
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 6
SSAAMMPPLLIINNGG
For a signal x(t),
xs (n)=x(nT) where T is the sampling period
F=1/T is the sampling frequency.
The inverse transformation is called Interpolation
x(t) from xs (n)
Issues
• If the sampled signal is interpolated, how do you
ensure that you get back the original signal
• How fast should we sample
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 7
QQUUAANNTTIIZZAATTIIOONN
The value at every sampled location is digitized.
The digital domain has a finite bit representation
The sampled value is approximated to the nearest digital
value.
OR Formally –
x q (n)=Q(x s (n)), where Q is a rounding function
which maps the values of x s (n) into N levels with
a quantization step
Typically, N=2b so that we need b bits to represent
one quantized sample.
Issues
What is the correct quantization step?
Quantization errors may result!
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 8
QQUUAANNTTIIZZAATTIIOONN EEXXAAMMPPLLEE IINN 11DD
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 9
QQUUAANNTTIIZZAATTIIOONN EEXXAAMMPPLLEE IINN 22DD ((11))
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 10
QQUUAANNTTIIZZAATTIIOONN EEXXAAMMPPLLEE IINN 22DD ((22))
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 11
QQUUAANNTTIIZZAATTIIOONN EEXXAAMMPPLLEE IINN 22DD ((33))
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 12
CCOOLLOORR QQUUAANNTTIIZZAATTIIOONN IINN IIMMAAGGEESS
24 bit RGB Color
(8 bits per channel)
16 Colors
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 13
BBIITT RRAATTEE
How many bits do you get per second?
Bit rate = (number of samples per second) x
(bits per sample)
Bit rate relates to the network through put
Examples of bitrate
• Audio – CD Bitrate
Sampling frequency: F= 44.1 KHz
Quantization with 16 bits
Bit-rate = 705.6 Kb/s (per channel)
As sampling rate increases, bit rate increases
As quantization bits used increase, bit rate increases
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 14
BBIITT RRAATTEE
Signal Sampling
Rate
Quantization Bit Rate
Speech 8 KHz 8 bits per sample 64Kbps
Audio CD 44.1 KHz 16 bits per sample 706 Kbps (mono)
1.4 Mbps (stereo)
Teleconferencing 16 KHz 16 bits per sample 256 Kbps
AM Radio 11 KHz 8 bits per sample 88 Kbps
FM Radio 22 KHz 16 bits per sample 352 Kbps (mono)
704 Kbps (stereo)
NTSC TV image
frame
Width – 486
Height – 720
16 bits per sample 5.6 Mbits per frame
HDTV (1080i) Width – 1920
Height – 1080
12 bits per pixel
on average
24.88 Mbits per
frame
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 15
SSOOMMEE TTHHEEOORRYY
Linear Time Invariant Systems
• Can be completely characterized by impulse response
• Impulse Response Vs Transfer Function
• Time Domain View: The output of the system is the
convolution of the input with the system’s impulse
response
• Frequency Domain View: The frequency transform
output of the system is the product of the transfer
function and the frequency transform of the input
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 16
TTIIMMEE DDOOMMAAIINN VVSS FFRREEQQUUEENNCCYY DDOOMMAAIINN
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 17
WWHHAATT’’SS TTHHEE CCOORRRREECCTT SSAAMMPPLLIINNGG RRAATTEE??
If F is too large (T is too small), we obtain too high a bit-
rate
If F is too small (T is too large), too much information is
lost in the sampling process
We want to capture as much information as necessary
to represent the signal correctly
the minimum sampling rate for “correct” sampling
depends on the frequency characteristics of the signal
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 18
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 19
NNYYQQUUIISSTT’’SS SSAAMMPPLLIINNGG TTHHEEOORREEMM
Let x(t) have a maximum frequency F. Then we can
“perfectly” interpolate the signal x(t) from its sampled
version x s (n)=x(nT) only if the sampling period T is less
than 1/(2F)
In other words, the sampling frequency should be at
least 2F for a signal whose maximum frequency is F –
Otherwise – aliasing
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 20
AALLIIAASSIINNGG EEXXAAMMPPLLEESS
Spatial Aliasing in one dimension
Example of a sinusoidal function in 1D
Audio aliasing (single frequency)
Audio without aliasing –
and with aliasing
Spatial Aliasing in two dimensions
http://195.134.76.37/applets/AppletNyquist/Appl_Nyquist2.html
http://www.cs.brown.edu/exploratories/freeSoftware/catalogs/signal_processing.html
Media/aliastpt.au
Media/aliastpt2.au
Media/aliastpt2.au
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 21
Spatial Aliasing
Spatial Aliasing – moiré lines
Temporal Aliasing
Revolving Light A real example
https://www.scientificpsychic.com/graphics/moire-applet.html
http://psych.hanover.edu/JavaTest/Media/Chapter4/MedFig.FlickerMotion.html
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 22
OOTTHHEERR EEXXAAMMPPLLEESS
Convolution –
http://www.jhu.edu/signals/lecture1/frames.html
Fourier Transform –
http://www.jhu.edu/signals/sampling/index.html
http://www.jhu.edu/signals/sampling/index.html
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 23
BBAANNDDLLIIMMIITTEEDD SSIIGGNNAALLSS AANNDD FFIILLTTEERRSS
Fourier Transform X(f) of a signal x(t): describes how the
“energy” of x(t) distributes among frequencies f
If the highest frequency in X(f) is B, we say x(t) is Band
Limited to B
A “filter” is an operator characterized by its frequency
response H(f):
• The Fourier transform of y(t) is Y(f)=H(f)X(f)
• Therefore, the band By of y(t) is the band Bh of
the filter
• Filters can be low-pass, band-pass or high-pass
x(t)
X(f)
h(t)
H(f)
y(t)
Y(f)
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 24
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 25
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 26
EEXXAAMMPPLLEESS UUSSIINNGG FFIILLTTEERRSS IINN CCOOMMPPRREESSSSIIOONN
Audio Filtering Example
Cut-off frequency of microphone is 100KHz. We
should sample at 200KHz. If Quantized at 16 bits
per sample -> 3 Mbs!
Our hearing system can only detect frequencies up to
~20KHz.
Prefilter the signal
Use a low-pass filter with cut-off frequency
B=20KHz. Then, we sample the signal at 40KHz
producing only 640 Kb/s
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 27
SSUUBBSSAAMMPPLLIINNGG ((DDEECCIIMMAATTIIOONN))
Given x (n), subsampling by M means generating a
signal y (n) = x (Mn).
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 28
SSUUBBSSAAMMPPLLIINNGG EEXXAAMMPPLLEE
Example: A continuous signal x(t), band-limited to
B=4KHz is sampled without aliasing with F=10 KHz.
Suppose now we subsample the resulting signal by 2.
This is equivalent to sampling the original signal with
rate F=5KHz (which gives aliasing)
Solution: digital low-pass filter before subsampling.
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 29
SSTTAATTIISSTTIICCAALL DDEEFFIINNIITTIIOONNSS
Mean or Expectation of the signal x (n), for a large
sample space M is defined as
( ) MnxMn
nx
=
=
=
1
)(
The Variance of the signal x(n) is defined as
The power of the quantization error, e
2 is the variance
of the signal e(n) = x q (n)-x(n)
The signal-to-quantization noise ratio (measured in dB)
SNR = 10 log 10 (x
2 e
2)
( )( ) Mxnxx
Mn
n
=
=
−=
1
2
2 )(
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 30
OORRIIGGIINNAALL IIMMAAGGEE
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 31
SSUUBBSSAAMMPPLLEEDD BBYY 22
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 32
SSUUBBSSAAMMPPLLEEDD BBYY 44
Recent research in signal processing
https://www.inf.ufrgs.br/~eslgastal/SpectralRemapping/Supplementary/
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 33
MMEEDDIIAA RREEPPRREESSEENNTTAATTIIOONNSS
Audio Signals – Time Varying Signals (amp @ t)
Images – 2D Signal (color @ x, y)
Video Signals – 3D Signals (color @ x, y, t)
Graphics –
– Inherently Digital
– 2D graphics objects
– 3D graphical objects
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 34
VVIIDDEEOO SSIIGGNNAALLSS
Video is obtained via raster scanning, which transforms
a 3-D color signal (function of x, y and t) into a one-
dimensional signal for transmission
Scanning is a sampling operation:
Samples in time: Frames
Samples along y (vertical direction): Lines
Samples along x (horizontal direction): Pixels
Scanning is done using two formats
• Progressive Scanning
• Interlaced Scanning
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 35
AANNAALLOOGG VVIIDDEEOO
History of Television and Analog Video
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 36
PPRROOGGRREESSSSIIVVEE SSCCAANNNNIINNGG
Rows are scanned left to right and top to bottom
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 37
IINNTTEERRLLAACCEEDD SSCCAANNNNIINNGG
Each frame is scanned twice (two fields)
First, scan all even lines
then, scan all odd lines
• Slow-moving objects can be perceived with high
spatial resolution
• fast-moving objects can be perceived at high
frame rate
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 38
IINNTTEERRLLAACCEEDD SSCCAANNNNIINNGG EEXXAAMMPPLLEE
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 39
LLUUMMIINNAANNCCEE AANNDD CCHHRROOMMIINNAANNCCEE
In color video, we have 3 signals:
1 signal of luminance
2 signals of chrominance
The three signals are composed together to form a color
image.
If only the luminance signal is received: grayscale image
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 40
CCHHRROOMMIINNAANNCCEE SSUUBBSSAAMMPPLLIINNGG SSCCHHEEMMEESS
Human visual system is less sensitive to the
Chrominance channels than to Luminance channel
We can subsample the chrominance channels without
noticeable loss of detail
Color subsampling schemes:
• 4:2:0 (a.k.a. 4:1:1): 1 sample of each chrominance
channel every 4 samples of luminance
• 4:2:2: 1 sample of each chrominance channel
every 2 samples of luminance
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 41
44::22::00 SSUUBBSSAAMMPPLLIINNGG SSCCHHEEMMEE
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 42
44::22::22 SSUUBBSSAAMMPPLLIINNGG SSCCHHEEMMEE
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 43
EEQQUUIIVVAALLEENNTT BBIITTSS//PPIIXXEELL
Assume luminance and the two channels of
chrominance are quantized with 8 bits/sample
4:2:0 – For every 4 pixels, we have 4 samples of
luminance and 1 sample each of chrominance.
• Overall, 4·8+8+8=48 bits per 4 pixels.
• On average, 48/4=12 bits per pixel.
4:2:2 – For every 2 pixels, we have 2 samples of
luminance and 1 sample each of chrominance.
• Overall, 2·8+8+8=32 bits per 2 pixels.
• On average, 32/2=16 bits per pixel.
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 44
IIMMAAGGEE AASSPPEECCTT RRAATTIIOOSS
Image Aspect Ratio: ratio of width to height in the image
• Typically 4:3 for standard TV
• HDTV has 16:9
• Cinemascope has 47:20!
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 45
PPIIXXEELL AASSPPEECCTT RRAATTIIOOSS
Pixel Aspect Ratio: ratio width to height of a pixel,
assuming it is a rectangle
• Computers have square pixels, ratio = 1
• NTSC Wide Screen 16:9, ratio = 1.2
Example:
Image Aspect Ratio = 4:3; N l =486; N p =720;
Then Pixel Aspect Ratio = (4/3)(486/720)=0.9
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 46
SSEEAAMM CCAARRVVIINNGG
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 47
SSTTIILLLL IIMMAAGGEE FFOORRMMAATTSS
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 48
VVIIDDEEOO FFOORRMMAATTSS
Format Name
Lines
per
Frame
Pixels
per
Line
Frames
per
Second
Interlaced?
Sub
sampling
scheme
Image
Aspect
Ratio
CIF 288 352 N 4:2:0 4:3
QCIF 144 176 N 4:2:0 4:3
SQCIF 96 128 N 4:2:0 4:3
4CIF 576 704 N 4:2:0 4:3
SIF-525 240 352 30 N 4:2:0 4:3
SIF-625 288 352 25 N 4:2:0 4:3
CCIR 601 NTSC
(DV, DVB, DTV)
480 720 29.97 Y 4:2:2 4:3
CCIR 601
PAL/SECAM
576 720 25 Y 4:2:0 4:3
EDTV (576p)
480 /
576
720 29.97 N 4:2:0
4:3 /
16:9
HDTV (720p) 720 1280 29.97 N 4:2:0 16:9
HDTV (1080i) 1080 1920
59.94
(field
rate)
Y 4:2:0 16:9
HDTV (1080p) 1080 1920 29.97 N 4:2:0 16:9
Digital Cinema (2K) 1080 2048 24 N 4:4:4 47:20
Digital Cinema (4K) 2160 4096 24 N 4:4:4 47:20
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 49
VVIIDDEEOO FFOORRMMAATTSS –– BBIITT RRAATTEE CCOOMMPPUUTTAATTIIOONN
Bit-rate for interlaced HDTV format is calculated as
N l =1080 lines per frame,
N p =1920 pixels per line,
N FPS =29.97 frames/second
P = 12 bits per pixels (luminance + chrominance)
N l N p N FPS ·12 = 745,749,504 bits/s.
Dr. Parag Havaldar – CS576 Lecture 2 8/30/2021 Page 50
GGRRAAPPHHIICCSS
Representation – vector and raster
Graphical object in 2D/3D