Magnolia stellata is a threatened tree, endemic to Japan, and most of its populations are small because it occurs in small swampy places. In theories, successive generations in a small isolated population results in low genetic diversity and drift load (reduction of fitness due to fixation of deleterious mutations). In M. stellata, isolated populations show low genetic diversity compared to non-isolated ones (Tamaki et al. 2008), suggesting that they suffer from the drift load. We conducted pollination experiments (intra- and inter- population crossing) in one small isolated population (Yokkaichi) and three non-isolated populations to evaluate the drift load in seed set per fruit. The drift load was measured by d value (d = 1 - [seed set for intra-population crossing]/[seed set for inter-population crossing]), and a negative d value indicates outbreeding depression. As a result, the drift load was detected (d = 0.11 - 0.45) for a small isolated population, Yokkaichi, whereas outbreeding depression was not detected for any populations. Given that selfing load is 0.39 (Hirayama et al. 2007), we estimated that 64% of embryos failed to develop because of the selfing and drift loads at Yokkaichi, suggesting that management to reduce genetic load is essential to increase reproductive success for the isolated populations of the species.