Large herbivores and topo-edaphic gradients are well-documented, major determinants of grassland plant production and species composition. In contrast, there is limited information about how these factors together may influence the composition of the arbuscular mycorrhizal fungus (AMF) communities associated with plants. AMF are a common component of grassland ecosystems where they can influence plant productivity, diversity, and soil stability. In this study, AMF community composition was analyzed in paired plots located inside and outside 40-44-year-old ungulate exclosures at six grassland sites in Yellowstone National Park (YNP), USA, that varied in soil moisture and the availability of soil nitrogen (N) and phosphorus (P). AMF spore abundance, species richness, and the relative abundance of AMF species were determined from soil samples collected (1) randomly (n = 5 samples) within each of the 12 plots and (2) from beneath the dominant grass (n = 5 samples per plot) at each site. Randomly collected soil samples explored the effects of ungulates and topographic position on AMF composition at the plant community level, subsuming potential effects of ungulates on plant species composition. Dominant plant samples examined how grazers, in particular, influenced AMF communities, while controlling for host-plant identity. Grazing decreased AMF spore abundance across the landscape (examined by random sampling) but increased the AMF species richness associated with dominant plants. Grazing influenced the AMF species composition at the plant community level and at the host-plant level by shifting the relative abundances of individual AMF species. Individual AMF species responded differently to grazing and N and P availability. Our results demonstrate how soil moisture and N and P availability across the landscape interact with grazing to influence AMF species composition.