When Are Graphs Worth Ten Thousand Words? An Expert-Expert Study

To describe phenomena that occur at different time scales, computational models of the brain must incorporate different levels of abstraction. At time scales of approximately 1/3 of a second, orienting movements of the body play a crucial role in cognition and form a useful computational level--more abstract than that used to capture natural phenomena but less abstract than what is traditionally used to study high-level cognitive processes such as reasoning. At this "embodiment level," the constraints of the physical system determine the nature of cognitive operations. The key synergy is that at time scales of about 1/3 of a second, the natural sequentiality of body movements can be matched to the natural computational economies of sequential decision systems through a system of implicit reference called deictic in which pointing movements are used to bind objects in the world to cognitive programs. This target article focuses on how deictic binding make it possible to perform natural tasks. Deictic computation provides a mechanism for representing the essential features that link external sensory data with internal cognitive programs and motor actions. One of the central features of cognition, working memory, can be related to moment-by-moment dispositions of body features such as eye movements and hand movements.