DIGITAL MEDIA COMPUTING
Digital Media Basics
Digital Media Acquisition
Digital Image Digital Video Digital Audio
Copyright By PowCoder代写 加微信 powcoder
School of Computer Science
Visual Perception
Structure of the Human Eyes
Concentric layers of fibrous cells Absorb ~8% of the visible light spectrum
Innermost membrane of the eye Light from object imaged on retina
Contracts & expands to control the amount of light entering the eye
Central opening of the Iris: Pupil (diameter: ~2 to 8 mm)
School of Computer Science
Structure of the Human Eyes
http://cdna.allaboutvision.com/i/resources-2017/eye-anatomy-700×635.jpg
School of Computer Science
Visual Perception
Structure of the Human Eyes
Distribution of discrete light receptors over the surface of the retina
2 classes of receptors: cones and rods
Cones: 6 – 7 million in each eye, mainly located in the fovea. Highly sensitive
to color, fine details. (Photopic or bright-light vision)
Rods: 75 – 150 million, distributed. Sensitive to low level illumination, not involved in color vision. (Scotopic or dim-light vision)
School of Computer Science
Visual Perception
Image Formation in the Eye
Photo camera: lens has fixed local length. Focusing at various distances by varying distance between the lens and an imaging plane (location of film or sensor chip)
Human eye: converse. Distance lens-imaging region (retina) is fixed. Focal length for proper focus obtained by varying the shape of the lens.
School of Computer Science
Visual Perception
Structure of the Human Eyes
Approximation: fovea ≈square sensor array of size
1.5mm x 1.5mm
Density of cones in this area: 150,000 elements/mm2
Number of cones in the region of highest acuity in the eye: ~337,000 elements
Just in terms of raw resolving power, a CCD can have this number of elements in a receptor array no larger than 5mm x 5mm.
Basic ability of the eye to resolve details is comparable to current electronic imaging sensors
School of Computer Science
Visual Perception
Perceived intensity is not a simple function of actual intensity
Although the intensity of the stripes is constant, we actually perceive a brightness pattern that is strongly scalloped, especially near the boundaries.
These seemingly scalloped bands are called Mach bands after , who first described the phenomenon in 1865.
This phenomenon clearly demonstrate that perceived brightness is not a simple function of intensity. The human visual system tends to undershoot or overshoot around the boundary of regions of different intensities.
Illustration of the Mach band effect.
School of Computer Science
Visual Perception
Perceived intensity is not a simple function of actual intensity
All the center squares have exactly the same intensity. However, they appear to the eye to become progressively darker as the background becomes lighter.
A more familiar example is a piece of paper that seems white when lying on a desk, but can appear totally black when used to shield the eyes while looking directly at a bright sky.
School of Computer Science
School of Computer Science
School of Computer Science
Light void of color = monochromatic (or achromatic) light Only attribute: intensity or gray level
Range of measure values = gray scale Monochromatic images = gray-scale images
Chromatic light source: frequency + radiance, luminance, brightness
Radiance = total amount of energy that flows from the light source (W)
Luminance (in lumens, lm) = measure the amount of energy an observer perceives from a light source
Brightness = subjective descriptor of light perception practically impossible to measure
School of Computer Science
Digital Image Acquisition
There are numerous ways to acquire images, but our objective in all is the same: to generate digital images from sensed data. The output of most sensors is a continuous voltage phenomenon being sensed.
To create a digital image, we need to convert the continuous sensed data into digital form. This involves two processes: sampling and quantization.
School of Computer Science
Digital Image Acquisition
Illuminationsourcereflectedfromasceneelement
Imagingsystemcollectstheincomingenergyandfocusitontoanimageplane
(sensory array)
Responseofeachsensorproportionaltotheintegralofthelightenergy projected
Sensoroutput:analoguesignal→digitized
School of Computer Science
Digital Image Acquisition
A general-purpose image processing system
Example: In a digital video camera, the sensors produce an electrical output proportional to light intensity. The digitizer converts these outputs to digital data.
A physical device that is sensitive to the energy radiated by the object we wish to image
A digitizer is a device for converting the output of the physical sensing device into digital form.
School of Computer Science
Digital Image Acquisition
(a) Single imaging sensor
The three principal sensor arrangements used to transform illumination energy into digital images.
Incoming energy is transformed into a voltage by the combination of input electrical power and sensor material that is responsive to the particular type of energy being detected. The output voltage waveform is the response of the sensor(s), and a digital quantity is obtained from each sensor by digitizing its response.
(c) Array sensor
(b) Line sensor
School of Computer Science
Digital Image Acquisition
Single imaging sensor
In order to generate a 2D image using a single sensor, there has to be relative displacements in both the x- and y-directions between the sensor and the area to be imaged.
Example: an arrangement used in high- precision scanning, where a film negative is mounted onto a drum whose mechanical rotation provides displacement in one dimension. The single sensor is mounted on a lead screw that provides motion in the perpendicular direction. Since mechanical motion can be controlled with high precision, this method is an inexpensive (but slow) way to obtain high-resolution images.
Combining a single sensor with motion to generate a 2D image
School of Computer Science
Digital Image Acquisition
Line sensor
A geometry that is used much more frequently than single sensors consists of an in-line arrangement of sensors in the form of a sensor strip.
The strip provides imaging elements in one direction. Motion perpendicular to the strip provides imaging in the other direction.
The imaging strip gives one line of an image at a time, and the motion of the strip completes the other dimension of a 2D image.
This is the type of arrangement used in most flat bed scanners.
Image acquisition using a linear sensor strip
School of Computer Science
Digital Image Acquisition
Image acquisition using a circular sensor strip
Sensor strips mounted in a ring configuration are used in medical and industrial imaging to obtain cross- sectional (“slice”) images of 3D objects.
A rotating X-ray source provides illumination and the portion of the sensors opposite the source collect the X-ray energy that pass through the object (the sensors obviously have to be sensitive to X-ray energy).
This is the basis for medical and industrial computerized tomography (CT) imaging.
School of Computer Science
Digital Image Acquisition
Image acquisition using a circular sensor strip
The output of the sensors must be processed by image reconstruction algorithms which can transform the sensed data into meaningful cross- sectional images, i.e., images are not obtained directly from the sensors but require extensive processing.
A 3D digital volume consisting of stacked images is generated since the object is moved in a direction perpendicular to the sensor ring.
School of Computer Science
Digital Image Acquisition
Image sampling and quantization
(a) Continuous image projected (b) Result of image sampling onto a sensor array and quantization
School of Computer Science
Digital Image Representation
Image sampling and quantization
Representing digital images
(a) Continuous image projected (b) Result of image sampling onto a sensor array and quantization
The result of sampling and quantization is a matrix of real numbers.
School of Computer Science
Digital Image Representation
f (x, y)
f (0,0) f (1,0)
f (0,1) f (1,1)
f (0,N 1) f (1,N 1)
f(M1,0) f(M1,1) f(M1,N1) a0,0 a0,1 a0,N1
a1,0 a1,1 a1,N1 A
aM1,0 aM1,1 aM1,N1
Each element of the matrix array is called an image element, picture element, pixel, or pel.
School of Computer Science
Digital Image Representation
An image is referred to as a 2D light intensity function f(x,y) where
(x,y) denotes the spatial coordinate, and
f is a function of (x,y) and is proportional to the brightness or grey
level of the image at that point Geometrically (0,0)
School of Computer Science
Digital Image Representation
TheimagedigitizationprocessrequiresdecisionsaboutvaluesforM (rows), N (columns), and for the number , L, of discrete gray levels allowed for each pixel.
Due to processing, storage, and sampling hardware considerations, the number of gray levels typically is an integer power of 2.
The number, b, of bits required to store a digitized image is b=M×N×k
When an image can have 2k gray levels, it is common practice to refer to the image as a “k-bit image”.
School of Computer Science
Digital Image Representation
Samplingistheprincipalfactordeterminingthespatialresolutionofan image.
Basically, spatial resolution is the smallest discernible detail in an image.
In terms of acquisition, spatial resolution can be obtained by multiplying the physical size of sensors and sampling resolution (e.g. dpi or ppi)
Printsizecanbedeterminedviceversa
School of Computer Science
1024 × 1024
512 × 512 256 × 256
Spatial Resolution of Image
128 × 128 64 × 64 32 × 32
Gray-Level Resolution of Image
Gray-level resolution refers to the smallest discernible change in gray level.
The most common number is 8 bits, with 16 bits being used in some applications where enhancement of specific gray-level ranges is necessary.
School of Computer Science
Gray-Level Resolution of Image
256 gray level
128 gray level
64 gray level
32 gray level
16 gray level
8 gray level
4 gray level
2 gray level
Color Imaging
Digital Camera
Sensors: CCD (Charge Coupled Device) vs CMOS (Complementary Metal Oxide Semiconductor)
http://www.dalsa.com/markets/ccd_vs_cmos.asp Digital Camera
http://www.fas.org/irp/imint/docs/rst/Intro/Part2_5a.html
School of Computer Science
Color Imaging
Nikon patents full-color RGB sensor http://dslr-cameras.blogspot.com/2007/08/nikon-patents-full-color-rgb-sensor.html
School of Computer Science
Color Imaging
CCD Sensor
Rectangular grid of electro-collection sites laid over a thin silicon wafer
Image readout of the CCD one row at a time, each row transferred in parallel to a serial output register
School of Computer Science
Color Imaging
CMOS Sensor
CMOS can potentially be implemented with fewer components, use less power and provide
data faster than CCDs
Image readout of the CCD one row at a time, each row transferred in parallel to a serial output register
School of Computer Science
Color Imaging
CCD vs CMOS
CCD: when exposure complete, transfers
each pixel’s charge packet sequentially to
a common output structure, which converts
the charge to voltage, buffer it and send it off-chip.
CMOS imager: the charge-to-voltage conversion takes place in each pixel
D. Litwiller, CCD vs CMOS: Facts and Fiction, Photonics Spectra, Jan 2001, Co. Inc.
School of Computer Science
Color Imaging
CCDvsCMOS
Responsivity (amount of signal the sensor delivers per unit of input optical
energy): CMOS imagers marginally superior to CCDs
Dynamic range (ratio of a pixel’s saturation level to its signal threshold): CCDs have advantage by factor of 2 in comparable circumstances
Uniformity (consistency of response for different pixels under identical illumination conditions): CMOS imagers “Traditionally worse”
Shuttering (ability to start and stop exposure arbitrarily): standard feature of virtually all consumer and industrial CCDs
Speed: CMOS argubly has the advantage over CCDs (all camera functions can be placed on the image sensor)
Windowing: CMOS has ability to read out a portion of the image sensor
Anti-blooming (ability to gracefully drain localized overexposure without compromising the rest of the image in the sensor): CMOS generally has natural blooming immunity, CCDs require specific engineering
Reliability: CMOS have advantage (all circuit functions can be placed on a single integrated circuit chip)
D. Litwiller, CCD vs CMOS: Facts and Fiction, Photonics Spectra, Jan 2001, Co. Inc.
School of Computer Science
Sensor Types
In general, the bigger the sensor, the better quality, the more expensive.
http://en.wikipedia.org/wiki/Image_sensor_format
School of Computer Science
Photography – Angle of View
In photography, angle of view describes the angular extent of a given scene that is imaged by a camera. It is used interchangeably with the more general term field of view (FOV).
http://en.wikipedia.org/wiki/Angle_of_view
School of Computer Science
Focal Length
The focal length of an optical system is a measure of how strongly the system converges (focuses) or diverges (defocuses) light.
The focal length (f), the distance from the front nodal point to the object to photograph (S1), and the distance from the rear nodal point to the image plane (S2)
School of Computer Science
Focal Length and Angle of View
http://en.wikipedia.org/wiki/Angle_of_view
School of Computer Science
Focal Length and Angle of View
Common lens angles of view
36 mm × 24 mm format (that is, 135 film or full-frame 35mm digital using width 36 mm, height 24 mm, and diagonal 43.3 mm).
http://en.wikipedia.org/wiki/Angle_of_view
School of Computer Science
Focal Length and Angle of View
http://en.wikipedia.org/wiki/Angle_of_view
School of Computer Science
Focal Length
http://www.europe-nikon.com/en_GB/product/nikkor-lenses/simulator
School of Computer Science
Focal Length vs View Angle
Fish Eye Lens
up to 180°
Ultra-Wide Angle (UWA) Lens
less than 24mm
greater than 84°
Wide Angle (WA) Lens
Normal/Standard Lens
36mm-60mm (50mm)
Telephoto Lens
greater than 60mm
less than 40°
35mm equivalent
School of Computer Science
Focal Length vs Perspective
Barrel distortion
School of Computer Science
In optics, an aperture is a hole or an opening through which light travels. aperture determines how collimated the admitted rays are, which is of great importance for the appearance at the image plane
If an aperture is narrow, then highly collimated rays are admitted, resulting in a sharp focus at the image plane.
If an aperture is wide, then uncollimated rays are admitted, resulting in a sharp focus only for rays with a certain focal length.
The lens aperture is usually specified as an f-number, the ratio of focal length to effective aperture diameter. A lens typically has a set of marked “f-stops” that the f- number can be set to.
A lower f-number denotes a greater aperture opening which allows more light to reach the film or image sensor.
http://en.wikipedia.org/wiki/Aperture_stop
School of Computer Science
F-stop or F-number
http://en.wikipedia.org/wiki/F-stop
School of Computer Science
Depth of Field (DoF)
Depthoffieldincreaseswithf-number,whichmeansthatphotographs taken with a low f-number will tend to have subjects at one distance in focus, with the rest of the image (nearer and farther elements) out of focus.
This is frequently useful for nature photography, portraiture, and certain special effects.
Thedepthoffieldofanimageproducedatagivenf-numberisdependent on other parameters as well, including the focal length, the subject distance, and the format of the film or sensor used to capture the image.
depth of field (DOF) is the distance between the nearest and farthest objects in a scene that appear acceptably sharp in an image.
School of Computer Science
Depth of Field (DoF)
School of Computer Science
f/2.8 f/5.6 f/8
images taken with a 200 mm lens (320 mm field of view on a 35 mm camera)
Varying exposure
Ways to change exposure Shutter speed
Aperture
Neutral density filters
School of Computer Science
Aperture & Shutter Speed
School of Computer Science
Aperture and Shutter Speed
School of Computer Science
http://www.nikonusa.com/en/Learn-And-Explore/Article/fue0dnl6/a-basic-look-at-the-basics-of-exposure.html
http://www.nikonusa.com/en/Learn-And-Explore/Article/fue0dnl6/a-basic-look-at-the-basics-of-exposure.html
http://www.nikonusa.com/en/Learn-And-Explore/Article/fue0dnl6/a-basic-look-at-the-basics-of-exposure.html
http://www.nikonusa.com/en/Learn-And-Explore/Article/fue0dnl6/a-basic-look-at-the-basics-of-exposure.html
http://www.nikonusa.com/en/Learn-And-Explore/Article/fue0dnl6/a-basic-look-at-the-basics-of-exposure.html
http://www.nikonusa.com/en/Learn-And-Explore/Article/fue0dnl6/a-basic-look-at-the-basics-of-exposure.html
Lens Quality
Spherical aberration
Solution: Aspherical lens
School of Computer Science
Lens Quality
Chromatic Aberration
Solution:Apochromatic Lens and Achromats
School of Computer Science
Color Image Representation
Formation of a vector from corresponding pixel values in three RGB component images.
School of Computer Science
Color Image Representation
Green Plane
Blue Plane
School of Computer Science
There are two different ways of generating moving pictures in a digital form for inclusion in a multimedia production.
Video – we can use a video camera to capture a sequence of frames recording actual motion as it is occurring in the real world;
Animation – we can create each frame individually, either within the computer or by capturing single images one at a time.
School of Computer Science
The Video Image
Avideoimageisaprojectionofa3Dsceneontoa2Dplane.
A 3D scene consisting of a number of objects each with depth, texture and
illumination is projected onto a plane to form a 2D representation of the scene.
The 2D representation contains varying texture and illumination but no depth information.
A still image is a “snapshot” of the 2D representation at a particular instant in time whereas a video sequence represents the scene over a period of time.
School of Computer Science
Digital Video Sequence
A “real” visual scene is continuous both spatially and temporally.
In order to represent and process a visual scene digitally, it is necessary to sample the real scene spatially (typically on a rectangular grid in the video image plane) and temporally (typically as a series of “still” images or frames sampled at regular intervals in time).
Digital video is the representation of a spatio-temporally sampled video scene in
digital form.
Spatial Sampling
Moving Scene
Temporal Sampling t
School of Computer Science
Digitization
real world scene
digitized video
The typical processing steps involved in the digitization of video.
After signal acquisition and amplification, the key processing steps are spatial sampling, temporal sampling, and quantization.
raster scanner
temporal sampler
RGB filter
School of Computer Science
Spatial Sampling
Spatial sampling consists of taking measurements of the underlying analog signal at a fini
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com