BACKGROUND: The period 2006-2009 saw intensive scale-up of HIV prevention efforts and an increase in reported safer sex among brothel and street-based sex workers in Mumbai and Thane (Maharashtra, India). Yet during the same period, the prevalence of HIV increased in these groups. A better understanding of sex workers' risk environment is needed to explain this paradox. METHODS: In this qualitative study we conducted 36 individual interviews, 9 joint interviews, and 10 focus group discussions with people associated with HIV interventions between March and May 2012. RESULTS: Dramatic changes in Mumbai's urban landscape dominated participants' accounts, with dwindling sex worker numbers in traditional brothel areas attributed to urban restructuring. Gentrification and anti-trafficking efforts explained an escalation in police raids. This contributed to dispersal of sex work with the sex-trade management adapting by becoming more hidden and mobile, leading to increased vulnerability. Affordable mobile phone technology enabled independent sex workers to trade in more hidden ways and there was an increased dependence on lovers for support. The risk context has become ever more challenging, with animosity against sex work amplified since the scale up of targeted interventions. Focus on condom use with sex workers inadvertently contributed to the diversification of the sex trade as clients seek out women who are less visible. Sex workers and other marginalised women who sell sex all strictly prioritise anonymity. Power structures in the sex trade continue to pose insurmountable barriers to reaching young and new sex workers. Economic vulnerability shaped women's decisions to compromise on condom use. Surveys monitoring HIV prevalence among 'visible' street and brothel-bases sex workers are increasingly un-representative of all women selling sex and self-reported condom use is no longer a valid measure of risk reduction. CONCLUSIONS: Targeted harm reduction programmes with sex workers fail when implemented in complex urban environments that favour abolition. Increased stigmatisation and dispersal of risk can no longer be considered as unexpected. Reaching the increasing proportion of sex workers who intentionally avoid HIV prevention programmes has become the main challenge. Future evaluations need to incorporate building 'dark logic' models to predict potential harms.