We study two extension problems, and their interconnections: (i) extension of positive definite continuous functions defined on subsets in locally compact groups G; and (ii) (in case of Lie groups G) representations of the associated Lie algebras La (G), i.e., representations of La (G) by unbounded skew-Hermitian operators acting in a reproducing kernel Hilbert space H-F (RKHS). Our analysis is non-trivial even if G = R-n, and even if n = 1. If G = R-n, (ii), we are concerned with finding systems of strongly commuting selfadjoint operators {T-i} extending a system of commuting Hermitian operators with common dense domain in H-F.

Our general results include non-compact and non-Abelian Lie groups, where the study of unitary representations in H-F is subtle.