The opsin gene family encodes key proteins animals use to sense light and has expanded dramatically as it originated early in animal evolution. Understanding the origins of opsin diversity can offer clues to how separate lineages of animals have repurposed different opsin paralogs for different light-detecting functions. However, the more we look for opsins outside of eyes and from additional animal phyla, the more opsins we uncover, suggesting we still do not know the true extent of opsin diversity, nor the ancestry of opsin diversity in animals. To estimate the number of opsin paralogs present in both the last common ancestor of the Nephrozoa (bilaterians excluding Xenoacoelomorpha), and the ancestor of Cnidaria + Bilateria, we reconstructed a reconciled opsin phylogeny using sequences from 14 animal phyla, especially the traditionally poorly-sampled echinoderms and molluscs. Our analysis strongly supports a repertoire of at least nine opsin paralogs in the bilaterian ancestor and at least four opsin paralogs in the last common ancestor of Cnidaria + Bilateria. Thus, the kernels of extant opsin diversity arose much earlier in animal history than previously known. Further, opsins likely duplicated and were lost many times, with different lineages of animals maintaining different repertoires of opsin paralogs. This phylogenetic information can inform hypotheses about the functions of different opsin paralogs and can be used to understand how and when opsins were incorporated into complex traits like eyes and extraocular sensors.