Aluminum alloy (AA)7075 is widely used to fabricate parts and components on aircrafts, which are subjected to contact loading that may induce fretting fatigue and catastrophic failure. In this work, a crystal plasticity finite element (CPFE) model accounting for the microstructural features is developed for simulating the fretting fatigue of AA7075-T651. A submodel technology is adopted to refine the contact region to obtain more accurate simulation data. An energy-based criterion is developed for prediction of crack initiation life. The hotspots for the fretting fatigue crack nucleation are identified by the maximum of plastic strain energy density. The proposed CPFE model achieves high accuracy on predicting the fretting fatigue crack initiation and validated by fretting fatigue experimental results.