Purpose: This study compared pragmatic language in youths (9–17 years) with autism spectrum disorder (ASD) and those with typical development (TD) on the Yale in vivo Pragmatic Protocol (YiPP), a semistructured, dynamic conversational assessment. Method: Participants (n = 118) were divided into groups based on age and diagnosis. Each completed the YiPP, which included 4 pragmatic domains (discourse management, communicative functions, conversational repair, presupposition). The participant’s response to each probe was scored correct or incorrect; incorrect scores elicited cues from the examiner, and level of cue required for a correction was also scored. Results: The YiPP showed high reliability and internal consistency, with moderate concurrent validity, sensitivity, and specificity. The group with ASD performed worse overall on YiPP probes compared to their TD counterparts on both error (d = 0.96) and cue (d = 0.91) scores. Item analyses revealed greater gaps between older students with ASD and their TD peers than between the 2 younger groups. Conclusions: These data suggest that a probe measure designed to assess pragmatic abilities in children with ASD within a conversational context has some validity for contributing to diagnostic classification and can identify specific areas of pragmatic vulnerabilities as part of a clinical assessment.