This paper studied the influence of carbon nanotube (CNT) and graphite nanoplatelet (GNP) used at 0 to 0.3%, by mass of binder, on composition, structure, and nano-mechanical characteristics of C-S-H of ultra-high performance concrete (UHPC). Hydration kinetics, pore structure, and mechanical properties of UHPC were also investigated. The non-proprietary UHPC was proportioned with 17% lightweight sand for internal curing. Test results indicated that the addition of 0.3% CNT or GNP led to approximately 20 MPa increase in compressive strengths of non-fibrous UHPC mortar and fibrous UHPC (2% steel fibers), compared to those prepared without any nanomaterial. Such enhancement is attributed to the nucleation and filling effect of the nanomaterials that can refine the pore structure. The increase in CNT and GNP contents from 0 to 0.3% increased the proportion of high density and ultra-high density C-S-H from 65% to 90%. This led to a 20% greater elastic modulus of the C-S-H. The use of CNT and GNP also favored the transformation from Q0 and Q1 to Q2 and Q3 (Qn represents the connectivity of silicate tetrahedron, and larger n indicates greater connectivity). This led to 140% and 110% increase in the mean chain length of C-S-H with the use of 0.3% CNT and GNP, respectively. Moreover, the Ca/Si of C-S-H decreased from 2.25 to 1.75, and the Al/Si increased from 0.1 to 0.15. The increased degree of Al-Si substitution reflects an enhanced pozzolanic reaction of the fly ash with the use of CNT and GNP.