This paper evaluated the cement-based materials incorporated with novel solid-gel phase change materials (PCMs) for improving the energy efficiency of building envelopes. This novel PCM is form-stable, which will not leak as solid-liquid PCMs do and not need encapsulation, and it features high energy-storage capacity. Experimental results showed that the thermal properties of cement-based materials were improved as the increase of PCM content. A 30% replacement of sand by volume with PCM can increase the latent heat of the mixture from around 0 to 7 J/g and decrease the thermal conductivity of PCM mortar based on the generalized self-consistent (GSC) model by about 20%. However, the workability and mechanical properties were compromised. The simulation results indicated that 30% PCM-incorporated walls can contribute to 5% energy saving for cooling in a whole year and 12% reduction in peak cooling load compared with the reference without PCM. The proposed PCM composite offers a promising avenue to achieve energy-efficient building envelopes.