We consider the phenomenology of a dimension-four operator that violates electromagnetic gauge invariance. Its magnitude is severely constrained by the lack of scattering of very low energy electromagnetic radiation off of the Cosmic Microwave Background (CMB) and by the lack of an induced mass when photons propagate in the CMB. We also discuss possible Lorentz-violating extensions of the operator basis. If a bare Proca mass exists and dominates over the induced mass, there is also a tight constraint from high energy scattering.