1 Motivation

The goal of this project is to implement template matching for the detection of objects in images. One of the most common tasks in computer vision is object detection; be it cars, faces, motion capture markers, or in this case airplanes.

Specifically, the goal is to detect airplanes from a satellite view. In this project you will be provided Google Earth images of airplane depots, with the goal of implementing a template matching based algorithm to detect and count the airplanes in each image.

While this appears to be a simple task at first, you will discover that practical applications have hurdles to overcome. Specifically, in each picture there are multiple different models of airplanes, and these planes are in different orientations in the image.

The specific project goals include:

• Gain experience in Matlab programming to implement computer vision algorithms
• Successfully import, manipulate, and convert images/matrices in Matlab
• Establish a method to select the templates used for matching
• Use correlation to score the similarity of the template to the image
• Use non-maximal suppression to locate and enumerate template matches
• Overlay graphics to show location/identification of each detection
• Successfully explain and visualize each step of the final algorithm

2 The Basic Operations

The following steps will be essential to the successful completion of the project:

1. Image input and conversion for manipulation: Each image needs to be read into Matlab and converted from a standard color image (MxNx3) to a grayscale image (MxNx1).
2. Selection of a template to use in the correlation: to detect airplanes using template matching the image must be correlated with an airplane template. To generate a template, there needs to be some automatic / manual template identification / selection / computation.
3. Localizing peaks from the template match: the output of template correlation will be a matrix with multiple peaks where the peaks are higher when the content in the image

has increased similarity to the template. Use non-maximum suppression to provide single pixel locations of each peak.

4. Use graphic overlays to enumerate each detection: the final locations need to be counted with a marker showing the detected airplane and a count added as a graphical overlay to the original image.

3 Evaluation

Quantitative

For this project, quantifying the accuracy of your algorithms implementation is easy for you as a human. There are 3 values you should use:

• True positives: airplanes that the algorithm detects that do exist in the picture based on human perception.
• False positives: airplanes that the algorithm detects that do not exist in the picture based on human perception.
• False negatives: airplanes that the algorithm does not detect that do exist in the picture based on human perception.

It is essential to show specific examples of the successful detections and failure cases. It is important to identify, enumerate, and evaluate failure cases as part of a process of cyclical improvement to the algorithms implementation.

Qualitative

As part of the report, each intermediate step of the process should be visualized to demonstrate the proper functionality of the algorithms implemented for the project. This will include visualizing the input image, the template(s) utilized, the correlation score matrix, the results of non-maximum suppression, and the final graphic overlay results. Figure 1 shows an example input and output image (this is a small sample from the provided test images.

The final report should provide results for each of the input files provided in the assignment. There are two (2) files (dma.jpg and vcv.jpg) provided for testing and evaluation which are both to be included in the report. Additionally, the example1.jpg input image shown in Figure 1 is provided for initial testing. It is recommended that during the initial development of the project code that only a portion of the full resolution test images be used as processing and testing the full images can be time consuming depending on the efficiency of the algorithms implementation. Each of the evaluation images is approximately 14MP with a resolution of 48002935.

4 What Libraries Can I Use?

The intent is that you will implement the various computational modules described using general processing functions (https://www.mathworks.com/help/matlab/functionlist.html) in Matlab. The single notable exception is that you MAY use the image processing toolbox (https://www.mathworks.com/help/images/index.html). Specifically, you MAY NOT use anything from the computer vision toolbox, or any third-party libraries/packages. Dont plagiarize any code from anywhere or anyone else. It saddens us to even have to say that, but youd be surprised what we discover each and every semester.

5 Project Reporting and Evaluation

Half of your grade will be based on submitting a fully operation program, and the other half will be based on a written report discussing your program, design decisions, and experimental observations. The following components will be submitted:

• Submit a written report in which you discuss at least the following:
  1. Summarize in your own words what you think the project was about. What were the tasks you performed; what did you expect to achieve?
  2. Present an outline of the procedural approach along with a flowchart showing the flow of control and subroutine structure of your Matlab code. Explain any design decisions you had to make. For example, how did you choose or extract airplane templates? If you used raw cross-correlation, did you subtract off the mean of the template first? Did you use normalized cross-correlation instead? How did you implement non-maximum suppression? Basically, explain at each step why you chose to do whatever you did. Be sure to document any deviations you made from the above project descriptions (and why), or any additional functionality you added to increase robustness or generality of the approach. This project is a little open-ended, so detailed explanation of everything you did is necessary for us to fully appreciate your approach to solving this project.
  3. Experimental observations. What do you observe about the behavior of your program when you run it? Does it seem to work the way you think it should? Run the different portions of your code and show pictures of intermediate and final results that convince us that the program does what you think it does.
  4. Extensions and Explorations. If you are in an exploratory mood, find some images of your own on the web and input them to your code. Try an airport like Pittsburgh International (PIT), or Los Angeles International (LAX). Can your implementation find some or all of the airplanes?
  5. Document what each team member did to contribute to the project. It is OK if you divide up the labor into different tasks (it is expected), and it is OK if not everyone contributes precisely equal amounts of time/effort on each project. However, if two people did all the work because the third team member could not be contacted until the day before the project was due, this is where you get to tell us about it, so the grading can reflect the inequity.
• Turn in a running version of your main program and all subroutines in a single zip or tar archive file (e.g. put all code, subroutines, etc in a single directory, then make a zip file of that directory for submission via Canvas). We can then unzip/untar everything and go from there.
  1. Include one or more demo routines that can be invoked with no arguments and that load any input needed from the local directory and display intermediate and final results that show the different portions of your program are working as intended.
  2. We might be running the code ourselves on other input, so include a routine that takes as input a single image and that outputs the matching location results.
  3. Include enough comments in your functions so that we have a good idea what each section of code does. The more thought and effort you put in to demonstrating / illustrating in your written report that your code works correctly, the less likely we feel the need to poke around in your code, but in case we do, make your code and comments readable.

FAQ: How long should your report be? We dont have a strict number of pages in mind, but as a general rule-of-thumb, if the length of your report (excluding figures) is less than the length of this project description document, you probably havent included enough detail about what you did/observed.

FAQ: Will the submitted code be evaluated for functionality and coding quality? Yes. Your code and all subsequent functions you write and submit can be reviewed and tested. Please make sure the code is your own unique work and no one elses, and that each function and what it does is easy for graders to identify.