Augmented surgical manipulation tasks can be viewed as a sequence of smaller, simpler steps driven primarily by the surgeon’s input. These steps can be abstracted as controlled interaction of the tool/end-effector with the environment. The basic research problem here is performing a sequence of control primitives. In computing terms, each of the primitives is a predefined computational routine (e.g. compliant motion or some other “macro”) with initiation and termination predicates. The sequencing of these primitives depends upon user control and effects of the environmental interaction. We explore a sensor driven system to perform simple manipulation tasks. The system is composed of a core set of “safe” system states and task specific states and transitions. Using the “steady hand” robot as the experimental platform we investigate using such a system.