The Kalman filter (KF) is optimal with respect to minimum mean square error (MMSE) if the process noise and measurement noise are Gaussian. However, the KF is suboptimal in the presence of non-Gaussian noise. The maximum correntropy criterion Kalman filter (MCC-KF) is a Kalman-type filter that uses the correntropy measure as its optimality criterion instead of MMSE. In this paper, we modify the correntropy gain in the MCC-KF to obtain a new filter that we call the measurement-specific correntropy filter (MSCF). The MSCF uses a matrix gain rather than a scalar gain to provide better selectivity in the way that it handles the innovation vector. We analytically compare the performance of the KF with that of the MSCF when either the measurement or process noise covariance is unknown. For each of these situations, we analyze two mean square errors (MSEs): the filter-calculated MSE (FMSE) and the true MSE (TMSE). We show that the FMSE of the KF is less than that of the MSCF. However, the TMSE of the KF is greater than that of the MSCF under certain conditions. Illustrative examples are provided to verify the analytical results.