Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statistics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement recording systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The storage and reactivation of perceptual symbols operates at the level of perceptual components--not at the level of holistic perceptual experiences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a common frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspection (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and abstract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinatorially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a perceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal symbol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored.

Grasping requires coding of the object's intrinsic properties (size and shape), and the transformation of these properties into a pattern of distal (finger and wrist) movements. Computational models address this behavior through the interaction of perceptual and motor schemas. In monkeys, the transformation of an object's intrinsic properties into specific grips takes place in a circuit that is formed by the inferior parietal lobule and the inferior premotor area (area F5). Neurons in both these areas code size, shape and orientation of objects, and specific types of grip that are necessary to grasp them. Grasping movements are coded more globally in the inferior parietal lobule, whereas they are more segmented in area F5. In humans, neuropsychological studies of patients with lesions to the parietal lobule confirm that primitive shape characteristics of an object for grasping are analyzed in the parietal lobe, and also demonstrate that this 'pragmatic' analysis of objects is separated from the 'semantic' analysis performed in the temporal lobe.

We used fMRI to examine the neural response in frontal and parietal cortices associated with viewing and naming pictures of different categories of objects. Because tools are commonly associated with specific hand movements, we predicted that pictures of tools, but not other categories of objects, would elicit activity in regions of the brain that store information about motor-based properties. We found that viewing and naming pictures of tools selectively activated the left ventral premotor cortex (BA 6). Single-unit recording studies in monkeys have shown that neurons in the rostral part of the ventral premotor cortex (canonical F5 neurons) respond to the visual presentation of graspable objects, even in the absence of any subsequent motor activity. Thus, the left ventral premotor region that responded selectively to tools in the current study may be the human homolog of the monkey canonical F5 area. Viewing and naming tools also selectively activated the left posterior parietal cortex (BA 40). This response is similar to the firing of monkey anterior intraparietal neurons to the visual presentation of graspable objects. In humans and monkeys, there appears to be a close link between manipulable objects and information about the actions associated with their use. The selective activation of the left posterior parietal and left ventral premotor cortices by pictures of tools suggests that the ability to recognize and identify at least one category of objects (tools) may depend on activity in specific sites of the ventral and dorsal visual processing streams.