The acoustic vibration of single gold nanoprism pairs on a glass substrate has been investigated in the time-domain combining a spatial modulation spectroscopy microscope with a high-sensitivity femtosecond pump−probe setup. Three modes were observed and ascribed to two in-plane and one out-of-plane vibration of the nanoprisms forming the pair, in agreement with a theoretical analysis. The periods of the two former modes with similar nature show weak (about 10%) and well correlated pair to pair fluctuations that can be unambiguously ascribed to variation of the prism geometry. In contrast, strong fluctuations, by almost a factor of 6, of the mode damping are evidenced with no correlation with their period. This indicates large variations of the prism-substrate coupling, providing a unique way for its local investigation.