We consider the interaction of a two-level atom inside an optical parametric oscillator. In the weak-driving-field limit, we essentially have an atom-cavity system driven by the occasional pair of correlated photons, or weakly squeezed light. We find that we may have holes, or dips, in the spectrum of the fluorescent and transmitted light. This occurs even in the strong-coupling limit when we find holes in the vacuum-Rabi doublet. Also, spectra with a subnatural linewidth may occur. These effects disappear for larger driving fields, unlike the spectral narrowing obtained in resonance fluorescence in a squeezed vacuum; here it is important that the squeezing parameter N tends to zero so that the system interacts with only one correlated pair of photons at a time. We show that a previous explanation for spectral narrowing and spectral holes for incoherent scattering is not applicable in the present case, and propose an alternative explanation. We attribute these anomalous effects to quantum interference in the two-photon scattering of the system.