Sex is a significant social category, and adults derive information about it from both faces and bodies. Research indicates that young infants process sex category information in faces. However, no prior study has examined whether infants derive sex categories from bodies and match faces and bodies in terms of sex. In the current study, 5-month-olds exhibited a preference between sex congruent (face and body of the same sex) versus sex-incongruent (face and body belonging to different genders) images. In contrast, 3.5-month-olds failed to exhibit a preference. Thus, 5-month-olds process sex information from bodies and match it to facial information. However, younger infants’ failure to match suggests that there is a developmental change between 3.5 and 5 months of age in the processing of sex categories. These results indicate that rapid developmental changes lead to fairly sophisticated social information processing quite early in life.