Recently, many plant populations are threatened with extinction due to decline in genetic diversity caused by habitat loss and fragmentation. Predicting future diversity loss is thus necessary to evaluate the viability of fragmented populations. Here, we used both demographic genetic analysis, which is the comparative evaluation of genetic diversity among different age/stage classes, and matrix population model to examine temporal genetic dynamics in fragmented populations of an understory perennial herb, Trillium camschatcense, in Tokachi, Hokkaido. Genetic diversity was evaluated using genome-wide 2,062 SNPs in two (large and small) populations. As a result, in the small population, genetic diversity significantly decreased in newly germinated seedling stage compared to matured stages, and effective number of breeders was lower than that of the large population. Simulation analysis confirmed that genetic drift and demographic stochasticity rapidly decreases genetic diversity in the small population. Besides, simulation using randomly perturbed transition matrices revealed that stasis at juvenile stages can maintain genetic diversity by mitigating the detrimental effects of genetic drift and demographic stochasticity. Overall, this study shows that both population size and demographic properties play crucial roles in temporal genetic dynamics and should be taken into consideration for the genetic management of fragmented populations.