Opponent Color Space Descriptors, Evaluations

The Sande, Gevers and Snoek paper proposes using color to complement the existing intensity-based corner detection and salient region description. The goal is be able to find more salient key-points, and represent the surrounding region with a discriminative descriptor for better matching and object recognition.

The idea is basically to extend the current single-channel methods to support multi-channel. Images get pre-processed for image-space transform. The author uses Opponent Color Space as an example. Described how to extend Harris-Laplace Corner Detector. The paper briefly go over a few color-SIFT detector such as: OpponentSIFT, WSIFT, rgSIFT.

The authors compared the degree of color-invariance among those color-SIFT methods. Color Invariance - invariant to illumination highlights, shadow, noise.

Opponent Color Transformation Steps:
1. RGB -> Opponent Color Space (O1, O2, O3)
2. Salient Color Boosting - Normalize the Opponent values with weights 0.850, 0.524, 0.065.

W-SIFT - I guess that 'W' is the W invariant property, which is a ratio of some spatial-differential transformed pixel value. The transformation is Gaussian Color Model. I suppose this property would be part of the descriptor, useful in matching.

Code
OpenCV implements a Opponent Color Conversion. But I cannot find where it does the Saliency Boosting. It  does not seem to implement the corner detection using multi-channel images. And it supports descriptor using separate channels implicitly.

The OpponentDescriptorExtractor expands an existing Descriptor Type with Opponent Color. It does so by repeating the extraction on all 3 channels and concatenate them together as one big descriptor.

Demo (descriptor_extractor_matcher.cpp)

Cannot find dedicated demo for Opponent Color Space, borrowing this generic one as a try-out.

User picks a trio of Detector, Descriptor and Matcher to perform keypoint-matching between 2 images. The second (query) image could come from 2 different sources: 1) user provided image, 2) a 'warped' version synthesized from the first one.

• Since Opponent Color Descriptor builds on an intensity-based one, specify the latter by peggy-backing, such as OpponentSURF = OpponentColor + SURF.
• Cross-Check-Matching: Pick out strong matches by including only those appearing on both forward and backward matching.
• Optionally draw in-lier matches only: Use the original homography or (RANSAC) approximate one from the strongly matched keypoints. Transform the reference key-points using this H. Query key-points must be within a threshold distance from the corresponding warped key-point in order to be considered an inlier match.

Secondary function of this application demonstrates how to use Evaluation API from OpenCV. Implementation in evaluation.cpp under features2d module.

• Feature-Detector Evaluation - Given 2 sets of key-points from different viewpoints of the same scene and its homography matrix. The evaluator returns the number of correspondences and repeatability value. Repeatability is basically a ratio of the correspondences to key-points count. It does so by analyzing the overlapping elliptical key-point region between the query key-point and the projected reference key-point.
• Generic Descriptor Matcher Evaluation - Given a DescriptorExtractor-DescriptorMatcher pair in the form of GenericDescriptorMatcher type, and two-sets of key-points and its homography. The evaluator returns the Recall-Precision Curve. Recall values are the ratio of the Current Correct Matches to Total Correspondences. Precision values are the ratio of Current Correct Matches to Current Total Correspondences. Using the term 'Current' in a sense that each ratio value is associated with a match. They are calculated in the order of descending strength (matching distance). A match is Correct if the overlapping region is big enough, just like how the detector is evaluated.

Results using Graf set from Oxford UGG

Results (img1 vs img2, 3203 vs 3536 keypoints detected)
[OpponentSURF]
Strong Matches 988 - Inliers 499
[SURF]
Strong Matches 1323 - Inliers 834

Results (img1 vs warped-img1 3203 vs 2758 keypoints detected)
FD Evaluation (took 5 minutes) - repeatability 0.673, correspondences 1719
[OpponentSURF]
Strong Matches 993 - Inlier 565
GDM Evaluation (took 20 minutes)

1-precision = 0.0; recall = 0.000966184
1-precision = 0.1; recall = 0.000966184
1-precision = 0.2; recall = 0.000966184
1-precision = 0.3; recall = 0.000966184
1-precision = 0.4; recall = 0.000966184
1-precision = 0.5; recall = 0.000966184
1-precision = 0.6; recall = 0.00966184
1-precision = 0.7; recall = 0.0995169
1-precision = 0.8; recall = 0.19372
1-precision = 0.9; recall = 0.319324
[SURF]
Strong Matches 1175 - Inlier 761
1-precision = 0.0; recall = 0.00193237
1-precision = 0.1; recall = 0.00193237
1-precision = 0.2; recall = 0.00193237
1-precision = 0.3; recall = 0.00193237
1-precision = 0.4; recall = 0.00289855
1-precision = 0.5; recall = 0.00338164
1-precision = 0.6; recall = 0.00772947
1-precision = 0.7; recall = 0.0144928
1-precision = 0.8; recall = 0.0550725

1-precision = 0.9; recall = 0.241063

Readings

• Color Descriptors for Object Category Recognition, van de Sande, Gevers and Snoek
• (Opponent Color Space) Boosting Saliency in Color Image Features, Weijer, Gevers.