Males with fragile X syndrome (FXS) have difficulties with social interaction and many show autistic features. This study examined whether the social deficits characteristic of FXS are associated with theory of mind difficulties. Two groups of boys with FXS participated: a group with few autistic features and a group with many autistic features. An intellectual disability control group also participated. In addition to using standard theory of mind tasks, new techniques were used that were able to separate out the various processing demands of the task (e.g., memory, inhibitory control). Overall, the findings indicate that both groups of boys with FXS have difficulty with theory of mind tasks compared to an intellectual disability control group. However, both groups with FXS also performed worse on comparison trials that required working memory but not theory of mind. Theory of mind difficulties are likely to be an important aspect of the FXS clinical profile, but are most likely the result from a more basic difficulty with working memory.