Continuous exposure to humans causes wildlife to either habituate or sensitize. Although habituation to humans play important role in minimizing human impact in urbanized-areas and outdoor recreational sites, the mechanisms and fitness consequences of habituation to humans are not fully understood. We developed a novel predictive framework to study habituation and sensitization to humans and applied it to yellow-bellied marmots (Marmota flaviventer) living in highly and less disturbed areas. We focused on the time allocation to vigilance while foraging to quantify baseline vigilance levels, and flight initiation distance (FID) to quantify subsequent responsiveness to threat. We also focused on rate of body-mass gain as fitness outcome. By using 15 years’ responses to human disturbances, we found that marmots allocated more time to vigilance in highly disturbed colonies, but this did not change over time. FID tended to decrease in highly disturbed areas and over time. Although FID decreased on average when they were experimentally approached more, we also found that there was individual variation in FIDs. Additionally, marmots gained less body mass over time in highly disturbed colonies than conspecifics in less disturbed colonies. Our results suggested that, although marmots habituated to humans, long-term human disturbance has negative fitness consequences.