A series of eight polymers based on the parent structure, poly[2,6-(4,4-bis-(2-ethylhexyl)-4H-cyclopenta[2,1-b;3,4-b′]-dithiophene)-alt-4,7-(2,1,3-benzothiadiazole)] (PCPDTBT), were synthesized for a systematic group-14 and group-16 single atom substitution study. The eight polymers were constructed with C/Si/Ge and S/Se/Te varied in the donor and acceptor, respectively. By examining experimental spectroscopic data and DFT calculated geometry and electronic structure, we gain new physical insights into the effects of heavy atom substitution at different positions in a donor–acceptor polymer. Absorption and emission experiments demonstrate that group-14 substitution in the donor unit only slightly blue shifts the long wavelength absorption (HOMO to LUMO transition) and that group-16 substitution in the acceptor affects this absorption to a much greater extent. Solvatochromism experiments show that the charge transfer excited state is most polarized when the acceptor contains a lighter atom and is influenced very little by the atom in the donor. Changing the atom in the acceptor has less effect on the absorption of the Si-donor and Ge-donor polymers than the C-donor polymers. Polymers that contain C-donors are stronger light absorbers than their Si-donor and Ge-donor analogues regardless of which atom is in the acceptor position. These results clarify the effects of single atom substitution on donor–acceptor polymers and aid in the future design of polymers containing heavy atoms.