Soil extracellular enzymes play a significant role in the N mineralization process. However, few studies have documented the linkage between enzyme activity and the microbial community that performs the function. This study examined the effects of inorganic and organic N fertilization on soil microbial communities and their N mineralization functions over 4 years. Soils were collected from silage corn field plots with four contrasting N treatments: control (no additional N), ammonium sulfate (AS; 100 and 200 kg of N ha−1), and compost (200 kg of N ha−1). Illumina amplicon sequencing was used to comprehensively assess the overall bacterial community (16S rRNA genes), bacterial ureolytic community (ureC), and bacterial chitinolytic community (chiA). Selected genes involved in N mineralization were also examined using quantitative real-time PCR and metagenomics. Enzymes (and marker genes) included protease (npr and sub), chitinase (chiA), urease (ureC), and arginase (rocF). Compost significantly increased diversity of overall bacterial communities even after one application, while ammonium fertilizers had no influence on the overall bacterial communities over four seasons. Bacterial ureolytic and chitinolytic communities were significantly changed by N fertilization. Compost treatment strongly elevated soil enzyme activities after 4 years of repeated application. Functional gene abundances were not significantly affected by N treatments, and they were not correlated with corresponding enzyme activities. N mineralization genes were recovered from soil metagenomes based on a gene-targeted assembly. Understanding how the structure and function of soil microbial communities involved with N mineralization change in response to fertilization practices may indicate suitable agricultural management practices that improve ecosystem services while reducing negative environmental consequences.