X-ray repair cross-complementing group 1 (XRCC1) and group 3 (XRCC3) polymorphisms are relatively frequent in Caucasian populations and may have implications in skin cancer modulation. A few studies have evaluated their association with non-melanoma skin cancer (NMSC), but the results are inconsistent. In the current study, we aim to assess the impact of XRCC1 R399Q and XRCC3 T241M polymorphisms on the risk of NMSC associated with sunlight and arsenic exposure. Study participants consist of 618 new cases of NMSC and 527 hospital-based controls frequency matched on age, sex, and county of residence from Hungary, Romania, and Slovakia. Adjusted effects are estimated using multivariable logistic regression. The results indicate an increased risk of squamous cell carcinoma (SCC) for the homozygous variant genotype of XRCC1 R399Q (OR 2.53, 95% CI 1.14-5.65) and a protective effect against basal cell carcinoma (BCC) for the homozygous variant genotype of XRCC3 T241M (OR 0.61, 95% CI 0.41-0.92), compared with the respective homozygous common genotypes. Significant interactions are detected between XRCC3 T241M and sunlight exposure at work, and between XRCC3 T241M and exposure to arsenic in drinking water (p-value for interaction <0.10). In conclusion, the current study demonstrates that polymorphisms in XRCC genes may modify the associations between skin cancer risk and exposure to sunlight or arsenic. Given the high prevalence of genetic polymorphisms modifying the association between exposure to environmental carcinogens and NMSC, these results are of substantial relevance to public health.