Severely ill-conditioned Hermitian matrices are commonly seen in digital signal processing as well as in other applications where the Galerkin method and the least squares method are employed to seek optimal solutions of linear or nonlinear models. A new treatment for such matrices is proposed in this paper, which is fundamentally different from the well-known conjugate gradient preconditioners, SVD-based methods, and other popular algorithms for solving ill-conditioned systems in the literature. We propose an exact, alternative formula for the inverse of a Hermitian matrix via parametric diagonal perturbation. It reduces the condition number of the original ill-conditioned matrix exponentially. The performance of the proposed algorithm is established via condition analysis and demonstrated over severely ill-conditioned matrix systems from digital filter designs.