Occupational groups at high-risk of HIV in sub-Saharan Africa (SSA) may be at increased risk of substance use because of occupation-related factors. We synthesised qualitative data on determinants and context of alcohol misuse and illicit drug use in these groups. We systematically searched five databases for qualitative studies reporting on alcohol misuse or illicit drug use in fisherfolk, uniformed personnel, miners, truckers, motorcycle taxi riders, and sex workers in SSA. Qualitative data and interpretations were extracted and synthesised using a systematic iterative process to capture themes and overarching concepts. We searched for papers published prior to January 2018. We identified 5692 papers, and included 21 papers in our review, published from 1993 to 2017. Most studies were conducted among fisherfolk (n = 4) or sex workers (n = 12). Ten papers reported on alcohol use alone, three on illicit drug use alone and eight on both. Substance use was commonly examined in the context of work and risky behaviour, key drivers identified included transactional sex, availability of disposable income, poverty, gender inequalities and work/living environments. Substance use was linked to risky behaviour and reduced perceived susceptibility to HIV. Our review underscores the importance of multilevel, integrated HIV prevention and harm reduction interventions in these settings.