1 Basic Requirements

1. Programming language: Matlab (recommended and template codes provided), or other languages(without template codes)
2. Three people in a team. List team members in your report, with name and asu ID.
3. Implement your code under Code folder, and read data from Data Under Data folder, Database contains all the images to retrieve from.
4. Final project report under name pdf at the project root folder.

2 Part I: Image Retrieval (50pt)

2.1 Image Feature Extraction (20pt)

Implement functions that takes in one image, and several hyper-parameters (based on the specific feature you are using), return a fixed length feature vector. Pick four from the following image feature extraction methods to implement.

1. Average pixel color;
2. Spatial grid of average pixel colors;
3. Color histograms;
4. Edge extraction and compute the edge energy in spatial grid;5. Shape context: assume the center of image to be the center of the shape;
5. Histograms of Oriented Gradient (HoG).

2.2 Database feature extraction (20pt)

Apply the four feature extraction methods you implemented onto each image from the Database folder, and create your feature library.

2.3 An interface for image retrieval (10pt)

Implement an interface (graphic interface preferred, command-line is also okay) that takes in one query image and return the top 10 images from your feature library. The user should be able to indicate which image feature extraction method is used and which distance measure (SSD or Angle between vectors) is used.

1

P02: Image Retrieval and Embedding 2

2.4 Test your implementation

Test your system with the images (4.png, 25.png, 39.png, 63.png, 129.png, 147.png, 252.png, 374.png, 540.png, 572.png) from Database folder (assume they are the query images), and screen capture your top 10 closest images retrieved for each image query. Qualitatively evaluate your systems performance and write them up in your report. Especially, are their evolution family members got successfully retrieved? Why or why not?

Also, test your system with other images from online resources, could be natural images.

3 Part II: Image Embedding (30pt)

Another way to show how well a certain image feature work with the image data is to embed the images onto a lower dimensional space (2-d or 3-d) based on their feature distances. In such a manner, developer could have a straight perception of how well the feature works. In this task, we are going to use the deep neural net based image features, which is extracted from a pre-trained neural net of millions annotated image data.

The idea is to mimic human vision neural system for image classification and retrieval.

3.1 PCA feature embedding

Read in the DeepNetFeature/feature vgg f.mat, which contains a matlab structure that has two elements: 1) feat: a 4096 elements Deep Neural Net feature vector, and 2) name: corresponding image name from the Database folder.

Use Principle Component Analysis method (pca() in matlab) to compute the top two principle components of the data, and plot the data based on their principle components.

3.2 Test your implementation

Test your implementation and capture the plot, put it in your report. Qualitatively evaluate your plot and write them up in your report. A sample embedding looks like Fig. 1.

4 Report Requirements (20pt)

Please include the following analysis in your report.

1. Make sure to expalin where the algorithms worked and where they didnt and why. You are encouragedto use both text and plots to explain your observations.
2. Include retrieval tests output and embedding output.
3. Include qualitative analysis. You could also propose evaluation metrics and do quantitative analysis.

5 Submission Instruction

Please place your answers under one .zip file with a formatted file name: P02 ASUID.zip (for example, if your asu ID is 101010101010, then the file name should be P02 101010101010.zip), and send it to P02: Image Retrieval and Embedding 3

Figure 1: A sample embedding.

[email protected] The .zip file shall include one folder with Matlab source code under name code and one report in .pdf format. Each group only needs one submission.

This assignment is due on midnight Oct 28th. Submission will be accepted after deadline with a 20% reduction in value per day late.