A common defect of chip seals is chip loss or raveling. The previous studies showed uniform grading of aggregate will enhance the retention ability of the chip seal. Also, it was shown that using crumb rubber as an aggregate will enhance the chip seal behavior including aggregate retention. However, no specific study has been done focusing on the effect of aggregate size for rubber nor natural aggregate. This paper is evaluating the effect of chip size on aggregate retention of both natural and rubber aggregate. Standard and modified Vialit tests, and standard and modified Pennsylvania tests which apply different forms of mechanical energy in different temperature was used to assess the aggregate-binder bond interaction and study the chip seal retention. Test results showed different trends for the effect of size on chip retention under impact load versus dynamic load because of different modes of failure. However, rubber particles showed a superior performance rather than natural aggregate in all cases.