This paper presents the results of an experimental study about the pore structure of lightweight self-compacting concrete with different amounts of silica fume and water-to-cement ratios. Mercury intrusion porosimetry was used to test the porosity and pore sizes. The fractal characteristics of the pore structures were studied, and the pore mass and solid mass fractal dimensions were calculated based on well-established methods. The optimum amount of silica fume was determined based on fractal dimension. The results have showed that compressive strength and water penetration depth increase with the increase of fractal dimension and have good correlation with fractal dimensions.