Overview

This homework has two (or three) parts covering basic geometric techniques in visual computing: warping, morphing, blending, and geometric model estimation. Generalization of these methods are routinely used in more advanced applications of computer vision. Thus, hands-on experience that you obtain while working on this assignment should aid in better understanding some later material in this course.

As in assignemnt 1, the students are strongly encouraged to work in python notebook environment (Jupiter/Anaconda/Python 2.7). Python’s domain transformation functions that will be used in assignment 2 are available in standard skimage.transform module (e.g. warp, AffineTransform, PiecewiseAffineTransform, ProjectiveTransform).

The students should use several starter notebooks provided specifically for this homework inside hw2.zip. These starter notebooks includes the necessary GUI interfaces for entering corresponding points and image display. In particular, CrossDissolver and LineBuilder classes are fully implemented.

It is expected that in each notebook (for parts I, II, or III below) you will use or create multiple “cells” corresponding to different specific tasks/experiemnts. In particular, use “markdown” cells to comment or discuss your results. Presentatation clarity and good organization of your notebooks will be evaluated/marked. Note: all results should be generated in your code cells – do not show results loaded from external images.

IMPORTANT: When preparing your notebooks for submission, you should test each notebook after restarting the python kernel in Jupiter (Kernel->Restart). Then, run the cells consequtively (top to bottom) and make sure that they work correctly in this order. When saving your notebook before submission, the cell numbers on the left margin of the notebook should be 1,2,3,4,… from top to bottom. When testing your code I can re-run the cells of your notebook only in this specific order. Also, save your notebook after each (interactive) figure cell is in the state showing the exact result that you want to demonstrate (e.g. for specific seeds) and that you might be discussing in a markdown cell right below the figure. Besides these tested notebook files, you should also save and submit their “html” versions (File->Download as->HTML), which also contain the “current” state of your figures.

Note that the assignment mentions some “optional” tasks. As all other required tasks, the optional tasks should be implemented in separate cells. Conclusive figures/experiments with specific points discussed in “markdown” cells (below) can yield bonus points.

Specific Tasks

• Part I (Image Morphing). 
  

  

  Implement image morphing and demonstrate your results over any two images. The starter notebook for this part is “MyMorpher_student.ipynb”. It implements mouse clicks for entering sequences (polylines) of corresponding points in two images. The starter notebook implements all the needed interface. Most of your work reduces to implementing function “pwAff_T” and completing class Morpher (method “showMorph”).

  • Implement function “pwAff” with input and return parameters as detailed in the starter notebook. You can use standard methods/classes PiecewiseAffineTransform() and warp() from skimage.transform module. Your goal is to find an intermediate warp from image A to B defined by step parameter T in [0,1] (see also slides for topic 8). The function should also return similarly warped user-defined points ptsA.
  • Use cells below function “pwAff_T” to validate correctness of your intermediate warps returned by your implementation “pwAff_T”. In particular, compare registration between imA and imB after warping them to the same intermediate frame using cross-dissolve between the corresponding two warps. Also, make sure that user points ptsA and ptsB warped to the same intermediate frame match each other, as well as the selected image features in warped imA and imB.
  • Write code for function “showMorph” inside Morpher class and validate that interactive morphing in the cell below works correctly (left-side and right-side clicks increment or deincrement T). As discussed in topic 8, morphing (showMorph function) should combine (intermediate) warps of two images to the same frame T and cross-dissolve of intensities for the two corresponding warped images.
• Part II (Panoramas). 
  

  

  Implement (planar) panoramic mosaicing for any two images obtained by rotating a camera around the same view point (optical center). You can use two provided images or any two that you take with your own camera. The starter notebook for this part is “MyPanorama_student.ipynb”. It implements mouse clicks for entering sequences (polylines) of corresponding points in two images. The starter notebook implements all the needed interface (PolylineBuilder and CrossDissolve that you can use for quick testing). The starter also inlcudes a small demo showing capabilities of homographies (as demoed in class) to generate a synthetic camera rotation (using “hall” image). Your assignment is detailed in the last four cells in the starter notebook. In particular, you should

  • warp the right image (imR) into the “reference frame” (image plane of imL) using affine and homographic transformations (AffLtoR and PrjLtoR) estimated from the corresponding points selected by clicks (implemented).
  • implement function “bondaryDT()”
  • in the last two cells you should generate two mosaics using two different blending schemes (averaging and DT-weighted averaging) discussed in topic 9.
• Part III (RANSAC). To be posted soon.
  

  What to submit, where to submit, and when to submit

  Submit one file hw2_your_name.zip containing three notebooks (MyMorphing.ipynb, MyPanorama.ipynb, MyRansac.ipynb) and their html copies generated as explained above (MyMorphing.html, MyPanorama.html, MyRansac.html). Your notebooks should include cells with your python code, figures, and explanations/discussions. Your code cells should include “code comments” (e.g. description of function interfaces/parameters, etc). All files with images used in your notebook should be placed in “images” subdirectory next to your notebook files (such images should be loaded to your notebooks using paths like “images/file_name.bmp”). When ready to submit your work, zip the notebook files, their html versions, and the subdirectory “images” into your submission file hw2_your_name.zip.The corresponding file (.zip) should be submitted electronically via OWL. The notebook (code and comments) should be your independent effort.

  All files should be submitted to OWL before 11:55pm on the due date (Nov.17). Do not wait until the last moment. A rush just before 11:55pm will not be accepted as a valid excuse.