Schools play a central role in the transmission of many respiratory infections. Heterogeneous social contact patterns associated with the social structures of schools (i.e., classes/grades) are likely to influence the within-school transmission dynamics, but data-driven evidence on fine-scale transmission patterns between students has been limited. Using a mathematical model, we analyzed a large-scale dataset of seasonal influenza outbreaks in Matsumoto city, Japan, to infer social interactions within and between classes/grades from observed transmission patterns. While the relative contribution of within-class and within-grade transmissions to the reproduction number varied with the number of classes per grade, the overall within-school reproduction number, which determines the initial growth of cases and the risk of sustained transmission, was only minimally associated with class sizes and the number of classes per grade. This finding suggests that interventions that change the size and number of classes, e.g., splitting classes and staggered attendance, may have a limited effect on the control of school outbreaks. We also found that vaccination and mask-wearing of students were associated with reduced susceptibility (vaccination and mask-wearing) and infectiousness (mask-wearing), and hand washing was associated with increased susceptibility. Our results show how analysis of fine-grained transmission patterns between students can improve understanding of within-school disease dynamics and provide insights into the relative impact of different approaches to outbreak control.