Systematic longitudinal observations were made as typically developing toddlers and young children with autism and with Down syndrome interacted with their caregivers in order to document how joint engagement developed over a year-long period and how variations in joint engagement experiences predicted language outcome. Children with autism displayed a persistent deficit in coordinated joint attention; children with Down syndrome were significantly less able to infuse symbols into joint engagement. For all groups, variations in amount of symbol-infused supported joint engagement, a state in which the child attended to a shared object and to language but not actively to the partner, contributed to differences in expressive and receptive language outcome, over and above initial language capacity.