Sexual size dimorphism (SSD) describes divergent body size of adult males and females in a species. Although it is already known that SSD occurs because individuals undergo sex-specific ontogenetic growth trajectories, how growth trajectories evolve and result in SSD under sex-specific selection and life history trade-offs remains unsolved. To elucidate this, we extend a quantitative genetic model through redefining fitness of individuals with accelerated growth rate or delayed maturation, two primary traits generating adult body size. Benefits of large males and females are high mating success and fecundity respectively. Costs corresponding to accelerated growth rate and delayed maturation are low survivorship and short reproduction period. Results indicate that SSD evolves because growth rate and duration of two sexes respond differently to changes of life history traits and costs of two growth strategies. Present theoretical framework provides an insight for explaining evolution of SSD based on ontogenetic perspective, as well as emphasizes the necessity of this perspective in SSD studies.