Understanding the impact of biodiversity loss on ecosystem functioning and services is of fundamental and practical importance in ecology. Here we report on the evidence from the Inner Mongolia Grassland Removal Experiment (IMGRE), which was designed to test how changes in plant functional diversity (PFD) and composition of the community would affect underpinning ecosystem processes along a manipulated PFD gradient generated by removing combinations of plant functional groups (PFGs). Our results showed that: (1) Aboveground net primary productivity (ANPP) decreased while the interannual variability of ANPP increased linearly with the increasing number of PFGs removed; (2) Soil moisture increased linearly with number of PFGs removed, indicating that greater functional diversity lead to a full use of soil moisture; (3) Biomass change depended on both functional richness and PFG composition of the community. The loss of perennial bunchgrasses alone and in combination with other PFGs explained large amounts of variation in ANPP. Our experimental evidence demonstrated that compensation is the key process governing the recovery of ecosystem functioning lost through PFG removals. The strength of compensation was determined mainly by the differences in recruitment rates and dispersal ability of the remaining species or PFGs.