Increases brain size has been hypothesized to be inversely associated with the expression of behavioral and brain asymmetries within and between species. We tested this hypothesis by analyzing the relation between asymmetries in the planum temporale (PT) and different measures of the corpus callosum (CC) including surface area, streamline count as measured from diffusion tensor imaging, fractional anisotropy values and the ratio in the number of fibers to surface area in a sample of chimpanzees. We found that chimpanzees with larger PT asymmetries in absolute terms had smaller CC surface areas, fewer streamlines and a smaller ratio of fibers to surface area. These results were largely specific to male but not female chimpanzees. Our results partially support the hypothesis that brain asymmetries are linked to variation in corpus callosum morphology, although these associations may be sex-dependent.