The study site (Fukuyama, Hiroshima, Japan) had been used for rice and wheat cultivation by conventional methods, and fallow management practices and abandoned had been carried out for them for 3 years. The 3-year accumulated aboveground dry weights of the mowed plots were slightly lower in the former paddy field than in the upland field. In the abandoned plot, a higher accumulated aboveground dry weight was found in the former paddy field. The accumulated root dry weight was significantly higher in the former paddy field (0.6 to 1.1 kg/m²), irrespective of the fallow management. The proportion of annual grasses was 74% to 85% in the former upland field. It was 33% to 44% in the former paddy field. Despite the change in the dominant plant species, there were no significant differences in the soil carbon content for the soil profiles among the management methods. We found the grass type to be more deterministic of soil carbon storage through the abundance of rhizomatous perennials. Perennial grasses had both deeper root systems and higher belowground allocations in the former paddy field. In contrast, very low inputs from shallow-rooted annual grasses associated with untapped soil moisture beneath the rooting zone caused a loss in soil carbon at the former upland field.