We provide theoretical insights on a mathematical approach used to calculate the Coulomb electrostatic potential of a uniformly charged square plate at any given point on its plane that we developed in the past. It is shown here that this method can be generalized in an elegant way to calculate this quantity at any arbitrary point in space. We explain the formalism and illustrate the implementation of the technique by providing all the key results that lead to an exact analytic expression for the value of the electrostatic potential on a point in the line perpendicular to the square plate that passes through its center.