The mathematical representation of large data sets of electronic energies has seen substantial progress in the past 10 years. The so-called Permutationally Invariant Polynomial (PIP) representation is one established approach. This approach dates from 2003, when a global potential energy surface (PES) for CH5+ was reported using a basis of polynomials that are invariant with respect to the 120 permutations of the five equivalent H atoms. More recently, several approaches from “machine learning” have been applied to fit these large data sets. Gaussian Process (GP) regression is such an approach. Here, we consider the implementation of the (full) GP due to Krems and co-workers, with a modification that renders it permutationally invariant, which we denote by PIP-GP. This modification uses the approach of Guo and co-workers and later extended by Zhang and co-workers, to achieve permutational invariance for neural-network fits. The PIP, GP, and PIP-GP approaches are applied to four case studies for fitting data sets of electronic energies: H3O+, OCHCO+, and H2CO/cis-HCOH/trans-HCOH with the goal of assessing precision, accuracy in normal-mode analysis and barrier heights, and timings. We also report an application to (HCOOH)2, where the full PIP approach is possible but where the PIP-GP one is not feasible. However, by replicating data, which is feasible in this case, the GP approach is able to represent the data with precision comparable to that of the PIP approach. We examine these assessments for varying sizes of data sets in each case to determine the dependence of properties of the fits on the training data size. We conclude with some comments on the different aspects of computational effort of the PIP, GP, and PIP-GP approaches and also challenges these methods face for more “rugged” PESs, exemplified here by H2CO/cis-HCOH/trans-HCOH.