We investigate the thermally activated escape of a Brownian particle over a potential barrier whose height fluctuates with a rate α between the values E+ and E−. We are mainly interested in the low-temperature behavior where E+/T≫E−/T. We calculate the mean exit time as a function of the rate of the barrier fluctuations for the piecewise linear and the piecewise constant barrier, τ=τ(α). For the piecewise constant potential we find three different regimes: τ∼τ+ for α<τ−1+=exp(-E+/T), τ∼2τ− for α>τ−1−=exp(-E−/T), and τ∼α−1 for τ−1+<α<τ−1−. The mean exit time for the piecewise linear potential has a different behavior for fast barrier fluctuations, α>τ−1−; τ(α) is a monotonously increasing function that approaches the asymptotic value τ∼ √τ+τ− for α→∞. We show that the behavior of the mean exit time for the piecewise constant potential is characterized by the absence of correlations between barrier crossings and barrier fluctuations. We discuss these correlations in some detail for the piecewise linear potential barrier.