Intergroup conflicts entail costs and benefits for group-living animals. While larger groups have advantages in intergroup aggression, the decline in reproductive success in larger groups suggests the minor importance of intergroup conflicts in primates. Japanese macaques in Yakushima are interesting subjects because, unlike most primates, reproductive success increases with group size. Although previous studies showed that the outcome of intergroup aggression depends on group size, macaques' behaviors during intergroup aggression remain unclear. We aimed to reveal whether behavioral responses to simulated intergroup encounters differed between two different-sized groups in Japanese macaques in Yakushima by conducting vocal playback experiments. In response to vocalizations of other groups, the smaller group left a feeding site more frequently than the larger group, and contact calling decreased in the smaller group but not in the larger group. The duration of looking toward the speaker and changes in the frequency of visual scanning and the group cohesion did not differ between the two groups. These results suggested that the smaller group reduced the risk of intergroup aggression at the expense of feeding and avoided being detected by other groups by decreasing contact calling.