Households are important settings for the transmission of seasonal influenza. Previous studies found that the per-person risk of within-household transmission decreases with household size. However, more detailed heterogeneities driven by household composition and contact patterns have not been studied. We employed a mathematical model that accounts for infections both from outside and within the household. The model was applied to citywide primary school seasonal influenza surveillance and household surveys from 10,486 students during the 2014/15 season in Matsumoto city, Japan. We compared a range of models to estimate the structure of household transmission and found that familial relationship and household composition strongly influenced the transmission patterns of seasonal influenza in households. Children had a substantially high risk of infection from outside the household (up to 20%) compared with adults (1-3%). Intense transmission was observed within-generation (between children/parents/grandparents) and also between mother and child, with transmission risks typically ranging from 5-20% depending on the transmission route and household composition. Children were identified as the largest source of secondary transmission, with family structure influencing infection risk.