Habitat fragmentation and invasive species are two of the most prominent threats to terrestrial ecosystems. Few studies have examined how these factors interact to influence the diversity of natural communities, particularly primary consumers. Here, we examined the effects of forest fragmentation and invasion of exotic honeysuckle (Lonicera maackii, Caprifoliaceae) on the abundance and diversity of the dominant forest herbivores: woody plant-feeding Lepidoptera. We systematically surveyed understory caterpillars along transects in 19 forest fragments over multiple years in southwestern Ohio and evaluated how fragment area, isolation, tree diversity, invasion by honeysuckle and interactions among these factors influence species richness, diversity and abundance. We found strong seasonal variation in caterpillar communities, which responded differently to fragmentation and invasion. Abundance and richness increased with fragment area, but these effects were mitigated by high levels of honeysuckle, tree diversity, landscape forest cover, and large recent changes in area. Honeysuckle infestation was generally associated with decreased caterpillar abundance and diversity, but these effects were strongly dependent on other fragment traits. Effects of honeysuckle on abundance were moderated when fragment area, landscape forest cover and tree diversity were high. In contrast, negative effects of honeysuckle invasion on caterpillar diversity were most pronounced in fragments with high tree diversity and large recent increases in area. Our results illustrate the complex interdependencies of habitat fragmentation, plant diversity and plant invasion in their effects on primary consumers and emphasize the need to consider these processes in concert to understand the consequences of anthropogenic habitat change for biodiversity.