In previous work, 11-month-old infants were able to learn rules about the relation of the consonants in CVCV words from just four examples. The rules involved phonetic feature relations (same voicing or same place of articulation), and infants' learning was impeded when pairs of words allowed alternative possible generalizations (e.g. two words both contained the specific consonants p and t). Experiment 1 asked whether a small number of such spurious generalizations found in a randomly ordered list of 24 different words would also impede learning. It did – infants showed no sign of learning the rule. To ask whether it was the overall set of words or their order that prevented learning, Experiment 2 reordered the words to avoid local spurious generalizations. Infants showed robust learning. Infants thus appear to entertain spurious generalizations based on small, local subsets of stimuli. The results support a characterization of infants as incremental rather than batch learners.