This paper considers the coupled orbit and attitude dynamics of a dumbbell spacecraft around an asteroid. Geometric methods are used to derive the coupled equations of motion, defined on the configuration space of the special Euclidean group, and then a nonlinear controller is designed to enable trajectory tracking of desired landing trajectories. Rather than relying on sliding mode control or optimization based methods, the proposed approach avoids the increased control utilization and computational complexity inherent in other techniques. The nonlinear controller is used to track a desired landing trajectory to the asteroid surface. A monocular imaging sensor is used to provide position and attitude estimates using visual odometry to enable relative state estimates. We demonstrate this control scheme with a landing simulation about asteroid Itokawa.