CS194-26/294-26: Intro to Computer Vision and Computational
Photography
Computer Science Division University of California Berkeley
INSTRUCTOR: Alexei (Alyosha) Efros (Office hours: after lecture)
GSI: Zhe Cao (Office hours: 9 – 10 AM Fri)
Tutors: Chenyue Cai (Office hours: TBD), Kenny Chen (Office hours: 11 – 12 PM Wed), Yibin Li (Office hours: 11 – 12 PM Thurs), and Violet Yao (Office hours: 10 – 11 AM Thurs).
UNIVERSITY UNITS: 4
SEMESTER: Fall 2020
WEB PAGE: http://inst.eecs.berkeley.edu/~cs194-26/fa20/
Q&A: Piazza Course Website
LOCATION: Remote (Zoom links are posted on Piazza)
TIME : MW 5:00 PM-6:30 PM
PREREQUISITES:
This is a heavily project-oriented class, therefore good programming proficiency (at least CS61B) is absolutely essential. Moreover, familiarity with linear algebra (MATH 54 or EE16A/B or Gilbert Strang’s online class) and calculus are vital. Experience with neural networks (e.g. CS189) is a plus. For these taking CS294-26, consent of instructor is required to register (please sign up on the waitlist first).
COURSE DESCRIPTION:
The aim of this advanced undergraduate course is to introduce students to computing with visual data (images and video). We will cover acquisition, representation, and manipulation of visual information from digital photographs (image processing), image analysis and visual understanding (computer vision), and image synthesis (computational photography). Key algorithms will be presented, ranging from classical (e.g. Gaussian and Laplacian Pyramids) to contemporary (e.g. ConvNets, GANs), with an emphasis on using these techniques to build practical systems. This hands-on emphasis will be reflected in the programming assignments, in which students will have the opportunity to acquire their own images and develop, largely from scratch, the image analysis and synthesis tools for solving applications.
PROGRAMMING ASSIGNMENTS:
Project 1: Images of the Russian Empire — colorizing the Prokudin-Gorskii photo collection
See student submissions here Class Choice Awards:
Project 2: Fun with Filters and Frequencies
See student submissions here Class Choice Awards:
Project 3: Face Morphing and Modelling a Photo Collection
See student submissions here
Project 4: Face Keypoint Detection with Neural Networks
Project 5: (Auto)stitching and photo mosaics
Final Project
Btrap Atrap snoissimbus tneduts eeS
:sdrawA eciohC ssalC ereh snoissimbus tneduts eeS
:sdrawA eciohC ssalC
See student submissions pre-canned own proposed
TEXT:
There is a textbook that covers most (if not all) of the topics related to Computational Photography. This will be the primary reference for the course:
Computer Vision: Algorithms and Applications, Richard Szeliski, 2010
There is a number of other fine texts that you can use for general reference:
Computer Vision: A Modern Approach (2nd edition), Forsyth and Ponce (classic computer vision text) Vision Science: Photons to Phenomenology, Stephen Palmer (great book on human visual perception) Digital Image Processing, 2nd edition, Gonzalez and Woods (a good general image processing text) Linear Algebra and its Applications, Gilbert Strang (a truly wonderful book on linear algebra)
CLASS NOTES
The instructor is extremely grateful to a large number of researchers for making their slides available for use in this course. Steve Seitz and Rick Szeliski have been particularly kind in letting me use their wonderful lecture notes. In addition, I would like to thank Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner and others, as noted in the slides. The instructor gladly gives permission to use and modify any of the slides for academic and research purposes. However, please do also acknowledge the original sources where appropriate.
TENTATIVE CLASS SCHEDULE:
CLASS DATE
Introduction
TOPICS
Material
Aug 26
Slides: pdf, pptx How Photography Became an Art Form by Aaron Hertzmann
Capturing Light… in man and machine
The Camera
Szeliski Ch. 2
Sampling and Reconstruction
Start Szeliski Ch 3 Reinhard et al., Color Transfer Between Images, IEEE Computer Graphics and Applications, 2001
Sampling and Reconstruction
Continue Szeliski Ch 3
Derivative and Template Filters
Image Blending and Compositing
Continue Szeliski Ch 3 + Sec 9.3.4 Additional Reading:
Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG, 1983 McCann & Pollard, Real- Time Gradient- Domain Painting, SIGGRAPH 2008
SIGGRAPH 2003
Bhat et al., GradientShop: A Gradient- Domain Optimization Framework for Image and Video Filtering, SIGGRAPH 2010
Avidan and Shamir, Seam Carving for Content-Aware Image Resizing, SIGGRAPH 2007
Kwatra et al.,
Graphcut Textures: Image and Video Synthesis Using Graph Cuts, SIGGRAPH 2003 Agarwala et al., Interactive Digital
,gnitidE egamI nossioP ,.la te zereP
Photomontage, SIGGRAPH 2004
Compression and Gradient Domain
Edge Detection and Image Warping
Image Morphing
Continue Szeliski Ch 3
Data-driven Methods: Faces
Galton, “Composite portraits made by combining those of many different persons into a single figure.” Nature,1878 Rowland and Ferrett, “Manipulating Facial Appearance through Shape and Color”, CG&A, 1995
Blanz and Vetter, “A Morphable Model for the Synthesis of 3D Faces”, SIGGRAPH 1999
Cootes, Edwards, and Taylor, “Active Appearance Models”, ECCV 1998
Data-driven Methods: Video Textures
Schodl et al., Video Textures SIGGRAPH’ 00
Efros and Leung, Texture Synthesis by Non-parametric Sampling ICCV’99
,
Efros and Freeman,
Image Quilting for Texture Synthesis and Transfer, SIGGRAPH’01 Hertzmann et al. Image Analogies, SIGGRAPH 2001.
Feature Learning with Neural Networks
Convolutional Neural Networks
Modeling Light
Light of Other Days
by Bob Shaw Lightfield
Homographies and Mosaics
Szeliski Ch 9
Automatic Alignment
Brown et al., ¡°Multi- Image Matching using Multi-Scale Oriented Patches¡±, CVPR 2005 M. Brown and D. G. Lowe, ¡°Recognising Panoramas¡±, ICCV 2003
RANSAC
Scene Modeling for a Single View
Multiview Geometry: Stereo & Structure from Motion
Class chosen special topics
CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital camera for use in the course.
METHOD OF EVALUATION:
Grading will be based on a set of programming and written assignments (60%), 6-7 (with 1 drop) in-class pop quizzes (20%)
and a final project (20%). For the programming assignments, students will be allowed a total of 5 (five) late days per semester; each additional late day will incur a 10% penalty.
Students taking CS294-26 will also be required to submit a conference-style paper describing their final project.
PROGRAMMING RESOURCES:
Students will be encouraged to use either MATLAB (with the Image Processing Toolkit) or Python (with either scikit-image or opencv) as their primary computing platform. Specific libraries in both languages offer tons of build-in image processing functions. Here is a link to some useful MATLAB and Python resources compiled for this class.
PREVIOUS OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here. SIMILAR COURSES IN OTHER UNIVERSITIES:
Computational Photography (Hoiem, UIUC) Computational Photography (Hays, Brown)
Digital and Computational Photography (Durand, MIT) Computer Vision (Seitz & Szeliski, UWashington)
Page design courtesy of Doug James