The near-resonant beat wave excitation of an electron plasma wave (EPW) can be employed for generating the trains of few-femtosecond electromagnetic (EM) pulses in rarefied plasmas. The EPW produces a comoving index grating that induces a laser phase modulation at the difference frequency. As a result, the cascade of sidebands red and blue shifted by integer multiples of the beat frequency is generated in the laser spectrum. The bandwidth of the phase-modulated laser is proportional to the product of the plasma length, laser wavelength, and amplitude of the electron density perturbation. When the beat frequency is lower than the electron plasma frequency, the red-shifted spectral components are advanced in time with respect to the blueshifted ones near the center of each laser beat note. The group velocity dispersion of plasma compresses so chirped beat notes to a few-laser-cycle duration thus creating a train of sharp EM spikes with the beat periodicity. Depending on the plasma and laser parameters, chirping and compression can be implemented either concurrently in the same, or sequentially in different plasmas. Evolution of the laser beat wave and electron density perturbations is described in time and one spatial dimension in a weakly relativistic approximation. Using the compression effect, we demonstrate that the relativistic bistability regime of the EPW excitation [G. Shvets, Phys. Rev. Lett. 93, 195004 (2004)] can be achieved with the initially sub-threshold beat wave pulse.