Toward a Visual Thesaurus

A thesaurus is a book containing related words, such as synonyms, in a given language; it provides similarity links when trying to retrieve articles or stories about a particular topic. A “visual thesaurus” works with pictures, not words. It aids in recognizing visually similar events, “visual synonyms,” including both spatial and motion similarity. This paper describes a method for building such a tool, and recent research results in theMIT Media Lab which contribute toward this goal. The heart of the method is a learning systemwhich gathers information by interacting with a user of a database. The learning system is also capable of incorporating audio and other perceptual information, ultimately constructing a representation of common sense knowledge.