CS558 Optional Projects
http://cvlabwww.epfl.ch/data/multiview/denseMVS.html
What should you turn in: A single zip file containing: 1) a pdf file with all results and a discussion of your solution 2) all your resulting images files and code. In addition to the above you should prepare a short (~5min) presentation describing your work, which you may be required to present at some point.
1) Image Data Association (2.5 Points Max)
The goal of the assignment is to apply an unsupervised clustering algorithm to segregate a set of images observing different scenes. This is similar to the image classification problem in Assignment 4, in the sense that you need to build a global image representation and use that representation to associate images. However, instead of assigning a single label from a predetermined number of possible labels, you will perform unsupervised clustering to partition the set of all images into disjoint subsets.
a) Download the datasets fountain-P11, Herz-Jesu-P8, entry-P10 from the Strecha MVS evaluation website and form a single aggregated dataset comprised of all the images in the three datasets (i.e. a superset). For this problem you will only use the images and discard other info.
b) Extract a global image descriptor for each image. You may choose any available descriptor and are NOT REQUIRED to develop descriptor code. Example descriptors include color histograms (e.g. Homework 4), GIST, bag of visual words, etc. Alternatively, you may choose to engineer your own image descriptor if desired.
c) Implement a mean-shift unsupervised clustering algorithm to partition the aggregated dataset into disjoint image sets. You will need to execute the mean-shift algorithm multiple times using each of the images in the superset as a starting point and record the convergence “mean” obtained by the algorithm. Images converging to the same “mean” are considered to belong to the same cluster.
d) Download the Castle-P19 image dataset and aggregate with the entry-P10 image dataset into a new superset. Perform the same clustering procedure using the same parameters. Please note, there is scene overlap among these datasets, making correct segregation more ambiguous.
Include in your report the attained clusters; discuss your choice of image descriptor as well as the mechanisms used for parameter tuning of your clustering algorithm.
2) Segmentation (5 Points Max)
The goal of this assignment is to develop a framework to perform segmentation of a specific foreground object. The object of interest is a foreground tractor found in the images of the Castle-19 dataset.
The desired output is a pixel-wise segmentation for each of the images in the dataset.
This is an open-ended problem for which we seek a solution that balances segmentation quality with the level of supervision required to carry out the task. Moreover, in designing your solution you may leverage the shared information among datasets (e.g. Castle-P19 has common scene regions with entry-P10, while the available Castle-P30 dataset also depicts the
foreground object and can be seen as a superset of Castle-P19), the available 3D ground truth data for each dataset (provided as a dense 3D mesh), or even elicit user input as part of a semi- automated segmentation framework.
Include in your report the attained segmentation and provide a detailed discussion of your solution.
3) Advanced Image Filtering (7.5 Points Max)
The goal of this assignment is to implement a multi-dimensional Gaussian filter using the Permutohedral Lattice. This advanced technique was NOT covered in the course slides, hence you will need to research and implement it. Develop a MATLAB implementation (no other languages will be considered, no exceptions) based on the description in
http://graphics.stanford.edu/papers/permutohedral/
a) Compare results of using a permutohedral-based 2D filter with with spatial std dev of 10 pixels on an RGB image vs running the typical 2D filter in each channel independently in order to aggregate results.
b) Deploy 5D filtering using pixel color and position (RGB+XY), define spatial std dev of 5, 10, and 10 pixels and RGB std dev of 10, 20, 30 intensity value. Experiment on any of the training outdoor images of the fourth assignment and test the nine possible combinations.
Notes: There are existing implementations available online, you are encouraged to use them for benchmarking/testing your own implementation (if you do so, include the comparative results in your report). The goal of this assignment is NOT for you to simply translate the code of an existing implementation into MATLAB, but to understand the material and implement it. There is NO REQUIREMENT to implement data access through a hash table as this is only an efficiency optimization.
Reviews
There are no reviews yet.