A hierarchical intelligent control strategy based on the physical components of a system was applied to the control of a robot arm. The controller was designed to make local decisions and carry out local actions to manipulate various different objects on a work table. The information about the environment was provided by a simulated vision system. To design the hierarchical controller the robot arm was first decomposed into its components. Next, functionalities were assigned to the nodes in terms of goals, tasks, procedures, constraints, measurements, and resources. The goals were then propagated down the hierarchy, with constraints applied to enforce the necessary sequential operation in real-time. To provide external information to nodes and to help them evaluate their constraints, temporary connections which are called smart links were used.