This paper describes a method to learn task primitives from observation. A framework has been developed that allows an agent to use observed data to initially learn a predefined set of task primitives and the conditions under which they are used. A method is also included for the agent to increase its performance while operating in the environment. Data that is collected while a human performs a task is parsed into small parts of the task called primitives. Modules are created for each primitive that encode the movements required during the performance of the primitive, and when and where the primitives are performed.