We use the United Kingdom's Born in Bradford study to investigate whether women in lower quality environments are less likely to breastfeed. We use measures of physical environmental quality (water disinfectant by-products [DBPs], air pollution, passive cigarette smoke, and household condition) alongside socio-economic indicators, to explore in detail how different exposures influence breastfeeding. Drawing on evolutionary life history theory, we predict that lower environmental quality will be associated with lower odds of initiating, and higher hazards of stopping, breastfeeding. As low physical environmental quality may increase the risk of adverse birth outcomes, which may in turn affect breastfeeding chances, we also test for mediation by gestational age, birthweight, head circumference, and abdominal circumference. Our sample is composed of mothers who gave birth at the Bradford Royal Infirmary in West Yorkshire between March 2007 and December 2010 for whom breastfeeding initiation data was available. Analyses were stratified by the two largest ethnic groups: White British (n = 3,951) and Pakistani-origin (n = 4,411) mothers. After controlling for socio-economic position, Pakistani-origin mothers had lower chances of initiating and higher chances of stopping breastfeeding with increased water DBP exposure (e.g., OR for 0.03-0.61 vs. <0.02 μg/day dibromochloromethane exposure 0.70 [0.58-0.83], HR 1.16 [0.99-1.36]), greater air pollution exposure predicted lower chances of initiation for both ethnic groups (e.g., OR for 10 μg/m3 increase in nitrogen dioxide 0.81 [0.66-0.99] for White British mothers and 0.79 [0.67-0.94] for Pakistani-origin mothers) but also a reduced hazard of stopping breastfeeding for White British mothers (HR 0.65 [0.52-0.80]), and exposure to household damp/mould predicted higher chances of breastfeeding initiation amongst White British mothers (OR 1.66 [1.11-2.47]). We found no evidence that physical environmental quality effects on breastfeeding were mediated through birth outcomes amongst Pakistani-origin mothers and only weak evidence (p < 0.10) amongst White British mothers (exposure to passive cigarette smoke was associated with having lower birthweight infants who were in turn less likely to be breastfed whereas greater air pollution exposure was associated with longer gestations and in turn reduced hazards of stopping breastfeeding). Overall, our findings suggest that there is differential susceptibility to environmental exposures according to ethnicity. Although the water DBP results for Pakistani-origin mothers and air pollution-initiation results for both ethnic groups support our hypothesis that mothers exhibit reduced breastfeeding in poorer quality environments, several physical environmental quality indicators showed null or positive associations with breastfeeding outcomes. We consider physiological explanations for our findings and their implications for life history theory and public health policy.