We compared cortical anatomy, using 3D brain MRI scans, between three groups of university students: proficient readers (skilled at phonological decoding and text comprehension), poor readers (impaired at phonological decoding and text comprehension), and resilient readers (impaired at phonological decoding but skilled in text comprehension). This latter group provides a unique opportunity to investigate associations between cortical morphology and phonological decoding deficits in individuals without attendant reading comprehension deficits. We predicted widespread reductions in gray matter thickness and brain size in temporal and frontal regions in poor readers, and more focal differences in brain morphology in resilient readers. Typical asymmetry of gray matter thickness in the temporo-parietal region was reduced in both poor and resilient readers. Poor readers also exhibited smaller brain sizes in the right inferior frontal region than both proficient and resilient readers. Altered asymmetry in the temporo-parietal region may therefore be associated with poor phonological decoding and impaired text comprehension may be associated with altered frontal morphology. Resilient readers show relatively focal behavioral differences from typical readers, so it is interesting that they show reliable differences in brain morphology.