This paper explores the accountability relationships that arise between researchers, the community and community representative structures known as Community Advisory Boards (CABs). It draws on ethnographic and case study research that documented the history, recruitment, composition and representativeness of two CABs and their relationships with researchers and communities, carried out in two studies in Zambia between 2010 and 2014. The findings revealed contradictions, nuances and imbalances in actual community participation and representation. In both studies, the general population was not given the opportunity to participate in the election of their CAB representatives, and the elected CAB members themselves were initially told to have little or no direct contact with research participants whom they were supposed to represent (unless researchers dictated otherwise). Owing to the researchers’ monopoly of scientific knowledge, literacy and financial resources, power relations were imbalanced. Further, researchers were quick to ask for and formalise community commitment through the CABs whilst reticent about their own accountability to the community. Yet despite these imbalances and CABs lacking formal authority over researchers, CABs did have subtle powers arising from their wider influence in the community, which they could tap into to either support or resist research. To achieve a more balanced and open accountability between research stakeholders, more genuine participatory processes need to be built and sustained.