[SOLVED] EBU7240 Computer Vision

EBU7240 Computer Vision
Detection1: Pedestrian detection
Semester 1, 2021
Changjae Oh

Copyright By Assignmentchef assignmentchef

Outline Overview
Dalal-Triggs (pedestrian detection) Histogram of Oriented Gradients
Learning with SVM

Object Detection
Focus on object search: Where is it?
Build templates that differentiate object patch from background patch
Object or Non-Object?

Challenges in modeling the object class
Illumination
Object pose
Occlusions
Intra-class appearance
[K. Grauman, B. Leibe]

Challenges in modeling the non
object class
True Detections
Bad Localization
Confused with Similar Object
Misc. Background
Confused with Dissimilar Objects

Object Detection Design challenges
How to efficiently search for likely objects
Even simple models require searching hundreds of thousands of positions and scales.
Feature design and scoring
How should appearance be modeled?
What features correspond to the object?
How to deal with different viewpoints?
Often train different models for a few different viewpoints

General Process of Object Detection
Specify Object Model Generate Hypotheses Score Hypotheses
Resolve Detections
What are the object parameters?

Specifying an object model
Statistical Template in Bounding Box
Object is some (x,y,w,h) in image
Features defined with respect to bounding box coordinates
Image Template Visualization
Images from Felzenszwalb 8

Specifying an object model
Articulated parts model
Object is configuration of parts Each part is detectable
Images from Felzenszwalb 9

Specifying an object model 3. Hybrid template/parts model
Detections
Template Visualization

Specifying an object model
4. 3D-ish model
Object is collection of 3D planar patches under affine transformation

Specifying an object model
4. Deformable 3D model
Object is a parameterized space of shape/pose/deformation of class of 3D object

Why not just pick the most complex model? Inference is harder
More parameters
Harder to fit (infer / optimize fit) Longer computation
, Leveraging MoCap Data for Human Mesh Recovery, Arxiv 2021

General Process of Object Detection
Specify Object Model Generate Hypotheses Score Hypotheses
Resolve Detections
Propose an alignment of the model to the image

Generating hypotheses
1. 2D template model / sliding window
Test patch at each location and scale

Generating hypotheses
1. 2D template model / sliding window
Test patch at each location and scale
Note Template did not change size

Each window is separately classified

Generating hypotheses
2. Region-based proposal
Arbitrary bounding box + image cut segmentation

General Process of Object Detection
Specify Object Model Generate Hypotheses Score Hypotheses
Resolve Detections
Mainly gradient-based features, usually based on summary representation, many classifiers.

General Process of Object Detection
Specify Object Model Generate Hypotheses Score Hypotheses
Resolve each proposed object based on the whole set

Resolving detection scores 1. Non-max suppression
Score = 0.8
Score = 0.8
Score = 0.1

Resolving detection scores 2. Context/reasoning
Via geometry
Via known information or prior distributions Non-max suppression
Hoiem et al. 2006

: Person detection with HOG & linear SVM
Histograms of Oriented Gradients for Human Detection,,, International Conference on Computer Vision & Pattern Recognition June 2005

Statistical Template
Object model = sum of scores of features at fixed positions!
? +3+2 -2-1 -2.5 = -0.5> 7.5
Non-object
? +4+1+0.5+3+0.5= 10.5 > 7.5

1. Extract fixed-sized (64128 pixel) window at each position and scale
2. Compute HOG (histogram of gradient) features within each window
3. Score the window with a linear SVM classifier
4. Perform non-maxima suppression to remove overlapping detections w ith lower scores
pedestrian detector
and, Histograms of Oriented Gradients for Human Detection, CVPR05

and, Histograms of Oriented Gradients for Human Detection, CVPR05
Slides by 27

Tested with
Grayscale
Gamma Normalization and Compression
Square root Log
Very slightly better performance vs. no adjustment
Slightly better performance vs. grayscale

Outperforms
uncentered
cubic-corrected

Histogram of Oriented Gradients
Orientation: 9 bins (for un Histograms over signed angles 0 -180) k x k pixel cells
Votes weighted by magnitude
Bilinear interpolation between cells
(4 bins shown)

Normalize with respect to surrounding cells
Rectangular HOG (R-HOG)
How to normalize?
Concatenate all cell responses from neighboring blocks into vector.
Normalize vector.
Extract responses from cell of interest.
Do this 4x for each neighbor set in 22.
e is a small constant
(to remove div. by zero on empty bins)

# orientations # features = 15 x 7 x 9 x 4 = 3780
# normalizations by neighboring cells

neg w Training data classes

pedestrian

Strengths/Weaknesses of Statistical Template Approach Strengths
Works very well for non-deformable objects with canonical orientations: faces, cars, p edestrians
Fast detection Weaknesses
Not so well for highly deformable objects or stuff Not robust to occlusion
Requires lots of training data

EBU7240 Computer Vision
Detection2: Face detection
Semester 1, 2021
Changjae Oh

Outline Overview
Viola-Jones (face detection) Boosting for learning
Decision trees

Consumer application: Apple iPhoto
Things iPhoto thinks are faces

Challenges of face detection
Sliding window = tens of thousands of location/scale evaluations
One megapixel image has ~106 pixels, and a comparable number of candidate face lo cations
Faces are rare: 010 per image
For computational efficiency, spend as little time as possible on the non-face windows
For 1 Mpix, to avoid having a false positive in every image, our false positive rate has to be less than 106

The Viola/Jones Face Detector
A seminal approach to real-time object detection.
Training is slow, but detection is very fast
Key ideas:
1. Integral images for fast feature evaluation
2. Boosting for feature selection
3. Attentional cascade for fast non-face window rejection
P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR 2001. P. Viola and M. Jones. Robust real-time face detection. IJCV 57(2), 2004.

1. Integral images for fast feature evaluation
The integral image computes a value at each pixel (x,y) that is the sum of all pixel values above and to the left of (x,y), inclusive.
This can quickly be computed in one pass through the image.
Summed area table

Computing the integral image
Region already computed
Current pixel

Computing the integral image
ii(x, y-1)
Cumulative row sum: s(x, y) = s(x1, y) + i(x, y)
Integral image: ii(x, y) = ii(x, y1) + s(x, y)
Python: ii = np.cumsum(i)

Computing sum within a rectangle
Let A,B,C,D be the values of the integra l image at the corners of a rectangle
The sum of original image values within
the rectangle can be computed as: C
sum = A B C + D
Only 3 additions are required for any size of rectangle!

Integral Images
ii = cumsum(cumsum(im, 1), 2)
ii(x,y) = Sum of the values in the grey region
SUM within Rectangle D is ii(4) ii(2) ii(3) + ii(1)

Integral Images
Find the integral image of the figure below and computer the sum of pix els in the grey region based on the integral image.
ii(4) ii(2) ii(3) + ii(1) = 42 10 4 + 1

Features that are fast to compute Haar-like features
Haar wavelet
Differences of sums of intensity
Computed at different positions and scales within sliding window
Two-rectangle features Three-rectangle features Etc.
CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=801361

Image Features
Rectangle filters
Value = (pixels in white area) (pixels in black area)

Computing a rectangle feature
Integral Image

But these features are rubbish!
Yes, individually they are weak classifiers
Jargon: feature and classifier are used interchangeably here. Also with learner.
But, what if we combine thousands of them
Two-rectangle features Three-rectangle features Etc.

How many features are there?
For a 2424 detection region, the number of possible rectangle features is ~160,000!

How many features are there?
For a 2424 detection region, the number of possible rectangle features is ~160,000!
At test time, it is impractical to evaluate the entire feature set.
Can we learn a strong classifier using just a small subset of all possible f eatures?

for feature selection
Initially, weight each training example equally. Weight = size of point
Slide credit:

In each boosting round:
Find the weak classifier, trained for each feature, that achieves the lowest weighted training error.
Raise the weights of training examples misclassified by current weak classifier.
Weak Classifier 1
for feature selection
Slide credit:

Boosting illustration
In each boosting round:
Find the weak classifier, trained for each feature, that achieves the lowest weighted training error.
Raise the weights of training examples misclassified by current weak classifier.
Weights Increased

Boosting illustration
In each boosting round:
Find the weak classifier, trained for each feature, that achieves the lowest weighted training error.
Raise the weights of training examples misclassified by current weak classifier.
Weak Classifier 2

Boosting illustration
In each boosting round:
Find the weak classifier, trained for each feature, that achieves the lowest weighted training error.
Raise the weights of training examples misclassified by current weak classifier.
Weights Increased

Boosting illustration
In each boosting round:
Find the weak classifier, trained for each feature, that achieves the lowest weighted training error.
Raise the weights of training examples misclassified by current weak classifier.
Weak Classifier 3

Boosting illustration
Compute final classifier as linear combination of all weak classifier.
Weight of each classifier is directly proportional to its accuracy.
Exact formulas for re-weighting and combining weak learners depend on the particular boosting scheme (e.g., AdaBoost).
Y. Freund and R. Schapire, A short introduction to boosting, Journal of Japanese Society for Artificial Intelligence, 14(5):771-780, September, 1999.

Boosting for face detection
First two features selected by boosting:
The first feature measures the difference in intensity between the region of the eyes a nd a region across the upper cheeks. The feature capitalizes on the observation that t he eye region is often darker than the cheeks.
The second feature compares the intensities in the eye regions to the intensity across the bridge of the nose.
The two features are shown in the t op row and then overlayed on a typi cal training face in the bottom row.

Feature selection with boosting
Create a large pool of features (180K)
Select discriminative features that work well together
Final strong learner window
Weak learner Learner weight
Weak learner = feature + threshold + polarity
polarity = black or white region flip
value of rectangle feature threshold
Choose weak learner that minimizes error on the weighted training set, then reweight

Boosting: Pros and Cons Advantages of boosting
Integrates classifier training with feature selection
Complexity of training is linear instead of quadratic in the number of training examples Flexibility in the choice of weak learners, boosting scheme
Testing is fast
Disadvantages
Needs many training examples Training is slow

Cascade for Fast Detection
Stage 1 h1(x) > t1?
Stage 2 h2(x) > t2?
Stage N hn(x) > tn?
Fast classifiers early in cascade which reject many negative examples but detect almost all positive examples.
Slow classifiers later, but most examples dont get there.

Attentional cascade
Chain classifiers that are progressively more complex and have lower fal
se positive rates:
Receiver operating characteristic
T T T Classifier 2 Classifier 3
IMAGE SUB-WINDOW
Classifier 1
NON-FACE NON-FACE NON-FACE

Attentional cascade
The detection rate and the false positive rate of the cascade are found by m ultiplying the respective rates of the individual stages
A detection rate of 0.9 and a false positive rate on the order of 106 can be achieved by a 10-stage cascade if each stage has a detection rate of 0.99 (0 .9910 0.9) and a false positive rate of about 0.30 (0.310 6106)
IMAGE SUB-WINDOW
Classifier 1
T T T Classifier 2 Classifier 3
NON-FACE NON-FACE NON-FACE

Training the cascade
Set target detection and false positive rates for each stage
Keep adding features to the current stage until its target rates have been met
Need to lower boosting threshold to maximize detection (as opposed to minimizing total classification error)
Test on a validation set
If the overall false positive rate is not low enough, then add another stage
Use false positives from current stage as the negative training examples f or the next stage

The implemented system Training Data
5000 faces
9500non-faceimages
All frontal, rescaled to 2424 pixels
300 million non-faces Faces are normalized
Scale,translation
Many variations Across individuals
Illumination Pose

38 stages with 1, 10, 25, 50 features 6061 total used out of 180K candidates
10 features evaluated on average
Training Examples
4916 positive examples
10000 negative examples collected after each stage
Scanning
Scale detector rather than image
Scale steps = 1.25 (factor between two consecutive scales) Translation 1*scale (# pixels between two consecutive windows)
Jones details
Non-max suppression: average coordinates of overlapping boxes
Train 3 classifiers and take vote

Jones Results
Speed = 15 FPS (in 2001)
MIT + CMU face dataset

Boosting for face detection
A 200-feature classifier can yield 95% detection rat e and a false positive rate of 1 in 14084
positive Not good enough!
Receiver operating characteristic (ROC) curve

Output of Face Detector on Test Images

Summary: Viola/Jones detector
Rectangle features
Integral images for fast computation
Boosting for feature selection
Attentional cascade for fast rejection of negative windows

CS: assignmentchef QQ: 1823890830 Email: [email protected]