Object Recognition using Basic Image Features
We introduce two novel feature alphabets and show how they can be used with simple image descriptors as a basis for several different types of visual vocabularies. The PASCAL VOC 2008 [5] challenge is used as evaluation framework.Basic Image Features (BIFs)
BIFs [1,2,3] label local image structure as one of seven types of local symmetry: flat, slope-like, minima-like, maxima-like, dark line-like, light line-like, and saddle-like. In the illustrations below the symmetry types are coloured pink, gray, black, white, blue, yellow, and green respectively. Labels are assigned as the index of the maximal of one of seven simple expressions in 0th-2nd order image derivatives. Images derivatives are calculated as inner products with a 2nd order Gaussian filter bank.
Oriented Basic Image Features (oBIFs)
Using the BIF type, local orientation can augment BIFs in a `controlled' fashion. Pink, black, and white (flat, minima, and maxima) BIFs have no meaningful intrinsic orientation and remain un-oriented. Gray or slope-like BIFs are dominated by their 1st order structure and augmented using 8-quantised gradient orientation. Blue, yellow, and green (dark/light lines and saddles) are dominated by their undirected 2nd order structure and augmented using 4-quantised principal orientations. The two image patches below show the orientation augmented oBIFs [2] of the two red highlighted squares from the motorbike image above. oBIFs can be thought of a natural 2nd order generalisation of the gradient orientation alphabet typically used in local images descriptors such as SIFT [4].
Basic Image Patterns (BIPs) and Object Recognition
Modern object recognition algorithms typically use a bag-of-visual-words style approach. A central problems is the construction of the visual vocabulary. We use patch based encodings of BIFs and/or oBIFs into Basic Image Patterns (BIPs). Several different BIP types have been investigated e.g., templates, histograms, quantised histograms, and approximate graphs. A BIP scheme induces a partition of the space of image descriptors into visual words. We have tested three different visual vocabularies; two [2,3] based on such geometrical partitionings of visual word space and one based on the more standard data-driven quantisation of an initial partitioning. Each vocabulary has been used with simple un-optimised of-the-shelf classifiers and applied to the 20-class PASCAL VOC object recognition challenge. Our score below is based on a late fusion of the three schemes.
Results
Results for the PASCAL VOC 2008 challenge. Each blue bar in the chart is a challenge entry - our result in light-green:
References
| [1] Griffin LD & Lillholm M (2008) Symmetry-sensitivities of derivative-of-Gaussian filters. IEEE PAMI, submitted. | |
| [2] Lillholm M & Griffin LD (2008) Novel Image Feature Alphabets for Object Recognition. In Proc of ICPR, to appear. | |
| [3] Crosier, M and Griffin LD (2008) Texture Classification with a Dictionary of Basic Image Features. In Proc of CVPR, pages 1-7. | |
| [4]. Lowe, DG (2004) Distinctive Image Features from Scale-Invariant Keypoints. In Int J Comput Vis, 60(2):91-110. | |
| [5] Everingham, M, Van Gool, L Williams, CKI, Winn, J and Zisserman, A (2008) The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results. http://www.pascal-network.org/challenges/VOC/voc2008/workshop/index.html. | |