Recent studies on retuning of phonetic categories by lexically guided perceptual learning suggest an inverse relationship between inherent acoustic variability and malleability of phonetic categories—the greater the variability, the smaller the retuning effect. However, it remains to be investigated what perceptual processing mechanisms are responsible for the observed relationship. Extending our previous study (Kataoka and Koo, 2017) that compared the degree of malleability between [u] (more variable) and [i] (less variable), we not only compared the size of retuning effect between the two vowels but also examined how listeners judge category goodness of synthesized stimuli from a [i]-[u] continuum. Our subjects (1) showed signs of category retuning for [i] but not for [u] and (2) judged goodness of stimuli in a more gradient manner and took longer to do so when asked to judge with reference to [u] than [i]. The results suggest that the two vowels differ in terms of their acoustic variability as well as their internal structure and that relative difficulty in resolving input speech signal in reference to a category such as [u] might be one reason the category is less malleable than a less variable category such as [i].