Many species of rewardless flowers exhibit polymorphism in flower color. Previous empirical studies suggested that learning by pollinators might impose negative frequency dependent selection on flower color and maintain color polymorphism in rewardless flowers. However, the evolutionary mechanism underlying the color polymorphism of rewardless flowers has not been fully understood. To explore the mechanism theoretically, we developed an individual-based simulation model describing the evolution of flower color in a population of rewardless flowers with animal-driven pollen transfer. The model assumed flower color as a continuous trait, and explicitly considers the color vision and learning processes of pollinators. Simulations were started from a population consisting of only one flower color. The simulations showed that a polymorphism of discrete color evolved when pollinators could learn rewardless flowers but with inaccurate discrimination between similar flower colors. When pollinator’s color discrimination was accurate, flower color in a population evolved to show a continuous distribution across the whole gradation of possible flower colors. Our results suggests that inaccurate color discrimination by pollinators is a key component in the mechanism by which negatively frequency-dependent disruptive selection caused by pollinator’s learning drives color polymorphism in rewardless flowers.