Predicting how species distribution will respond to climate change is important for future conservation. In theory, species will shift their ranges to track their suitable thermal conditions, but some species will maintain their original distribution if suitable microclimates are found in situ. However, empirical evidence demonstrating the role of microclimate in shaping species distribution is limited. In this study, we aimed to understand the relative contributions of local (ambient) climatic conditions and microclimate on the distribution of a rock-dwelling lagomorph, the northern pika, Ochotona hyperborea. We surveyed their occupancy across a wide elevational gradient (350–2200 m) at 56 sites (site-year combinations) in Hokkaido and investigated its relationships with local climatic conditions (ambient air temperature) and microclimate (interstitial temperature) using occupancy models. At the whole-distribution scale, occupancy was best predicted by mean air temperature, indicating the dominant effect of local climatic conditions in shaping the overall distribution. However, mean interstitial temperature predicted occupancy well at the range edge of the distribution, suggesting its importance in relatively unsuitable regions. Our findings suggest that microclimate plays an important role in shaping species distributions, particularly at range boundaries, highlighting the importance of incorporating isolated patches with distinct microclimates near range edges as conservation targets.