A method for measuring composite distortion levels using a coherent multicarrier RF signal generator having incrementally related frequencies is disclosed. The invention provides for the use of a coherent multicarrier signal generator that permits arbitrary RF carrier phase control on an individual carrier basis in order to enable sequential distortion measurements under varying carrier phase conditions. In order to obtain measurement results that match those obtained by the use of prior art non-coherent signal sources, the present invention provides for averaging of distortion measurement results over a `phase configuration ensemble` that is obtained by multiple sequential measurements, in which the individual carrier phases for each measurement are preselected at random. The advantages afforded by the invention in comparison with prior art methods are improved repeatability and accuracy, reduction of the required measurement resolution bandwidths (from 30 kHz to as low as 30 Hz) thereby the reduction of noise levels by 30 dB. This feature greatly improves the measurement system's dynamic range and permits the elimination of overload protection preselector filters that are commonly used in prior art non-coherent measurement systems.