Human mathematical abilities comprise both learned, symbolic representations of number and unlearned, non-symbolic evolutionarily primitive cognitive systems for representing quantities. However, the mechanisms by which our symbolic (verbal) number system becomes integrated with the non-symbolic (non-verbal) representations of approximate magnitude (supported by the Approximate Number System, or ANS) are not well understood. To explore this connection, forty-six children participated in a 6-month longitudinal study assessing verbal number knowledge and non-verbal numerical acuity. Cross-sectional analyses revealed a strong relationship between verbal number knowledge and ANS acuity. Longitudinal analyses suggested that increases in ANS acuity were most strongly related to the acquisition of the cardinal principle, but not to other milestones of verbal number acquisition. These findings suggest that experience with culture and language is intimately linked to changes in the properties of a core cognitive system.