In this paper, we present preliminary work on identifying equivalent structures required by LDA, and how they affect inference of document-specific topic distributions. This work is based on the observation that when occurrences of a particular word type in a document could be explained by multiple topics, inference will almost always force the model to choose between these topics. Not only is this required by the structure of optimal solutions to the LDA inference problem (see lemma 4 of Sontag and Roy \cite{david11:compl}) but, intuitively, this is also why learning the topic-specific distributions over words is possible: by explaining all document-specific occurrences of a word type with one topic and searching for sparse document--topic distributions, the latent topics should ``move apart'' and eventually represent sets words that exhibit within-topic, but not across-topic, co-occurrences.