Different influenza subtypes can evolve at very different rates, but the causes are not well understood. In this paper, we explore whether differences in transmissibility between subtypes can play a role if there are fitness constraints on antigenic evolution. We investigate the problem using a mathematical model that separates the interaction of strains through cross-immunity from the process of emergence for new antigenic variants. Evolutionary constraints are also included with antigenic mutation incurring a fitness cost. We show that the transmissibility of a strain can become disproportionately important in dictating the rate of antigenic drift: strains that spread only slightly more easily can have a much higher rate of emergence. Further, we see that the effect continues when vaccination is considered; a small increase in the rate of transmission can make it much harder to control the frequency at which new strains emerge. Our results not only highlight the importance of considering both transmission and fitness constraints when modelling influenza evolution, but may also help in understanding the differences between the emergence of H1N1 and H3N2 subtypes.