Responses are faster when the task-irrelevant orientation of a graspable object's handle corresponds to the location of the response hand. Over the past decade, research has focused on dissociating between two competing accounts of this effect: One rooted in motoric object affordances and the other resting on attentional mechanisms (i.e., Simon effect). Following this avenue of inquiry, we conducted three experiments, in which subjects had to respond bimanually to grayscale photographs of frying pans and saucepans. In addition to horizontal orientation (control/leftward/rightward handles), Experiments 1 and 2 also manipulated the direction of exogenous attentional shifts (left/right) using laterally placed, colored markers within the objects. Both experiments yielded regular Simon effects based on the location of the colored markers. However, in stark contrast to previous research, a negative stimulus-response compatibility effect was obtained with regard to the orientation of the graspable handles. This reversed affordance effect was also observed using the original, unedited grayscale photographs (Experiment 3), which suggested that its occurrence cannot be attributed to the use of colored markers. These unexpected findings appear to support the idea that Simon effects result from automatic and exogenous attentional orienting mechanisms, whereas affordances arise from controlled and endogenous attentional processes. Such a top-down attentional account of affordance can accommodate the observed reversal of the effect in the context of task characteristics.