[link to COREL
categorization] basic-level categories sorted according to
percentage correct
[link to COREL category labels] how we categorized the COREL image classes
[pdf manuscript] 80-citation
manuscript
[talk]
slides about the categorization system
The Challenge:
Visual image classification is the assignment of a given image to a category
such as ‘chair’, ‘animal’, ‘street scene, and so on. This assignment is
difficult because categories bear a lot of structural variability, for instance
different chairs appear with varying geometry. This geometrical variability is
underestimated and not properly expressed in any previous and present
classification approach.
Method: I therefore pursue a
classification system, in which an image is firstly decomposed into a large number of parameters
describing contours and areas. For instance, contour vectors describe aspects
such as length, orientation, curvature, smoothness, fuzziness, contrast and
degree of isolation in a structure.
Evaluation:
The decomposition was evaluated on the COREL draw and the Caltech 101
collection. For each I obtained an average of 12 percent correct categorization,
using a simple histogramming approach! Using a
vector-based image search, I obtained an average of 22 and 28 percent correct
categorization (for the first 100 images). That is extremely promising!
Long-term goal: To
obtain perfect categorization, I now need to span the appropriate multi-dimensional
space with those parameters, which allows forming abstract category representations.
Evaluation Examples:
Here are some example searches (using all 60000 COREL images), specifically
similarity-based image searches of which the similar images were basic-level
categorized [without color information!]:
Pretty
good, but obviously not perfect yet. However that was only the beginning (this
was only a histogramming search, without the explicit
matching of individual contours). The following shows how specific contours can be (individual contour
matching):
And this demonstrates how specific areas (relation between two contours)
can be:
The method is obviously highly potential
but now requires a clever learning procedure.
The long-term goal is of course to
build a complete scene-understanding system.
Image
sorting according to aspects, length, curvature,
contrast,…
More
similarity-based image search: The first
one is the sample image, the remaining ones are the detected similar ones
(without categorization though):
The
following are learned category-specific contours for the Caltech collection,
with which I obtained a search performance of 28 percent in average:
Conclusion: The parameterization is very detailed and can obviously be used for
categorization and image search to some extent. But now I need to perform more
grouping and abstraction.