An atomistic model of metallic contacts using realistic interatomic potentials is used to study the connection between friction, slip and the structure of the buried interface. Incommensurability induced by misalignment and lattice mismatch is modeled with contact sizes that are large enough to observe superstructures formed by the relative orientations of the surfaces. The periodicity of the superstructures is quantitatively related to inhomogeneous shear stress distributions in the contact area, and a reduced order model is used to clarify the connection between friction and structural inhomogeneity. Finally, the movement of atoms is evaluated before, during and after slip in both aligned and misaligned contacts to understand how the interfacial structure affects the mechanisms of slip and the corresponding frictional behavior.