BACKGROUND: Despite the expansion of the Integrated Community Case Management services for childhood illness, quality and utilization of services have remained low. To address the problem, the Government of Ethiopia introduced a complex intervention that included community engagement, capacity building of health workers and enhanced district-level ownership of sick child management. We examined whether this complex intervention was associated with improved management of sick children by health extension workers. METHODS: The study was conducted in four Ethiopian regions. A baseline survey was conducted in 26 intervention and 26 comparison districts from December 2016 to February 2017, followed by an end-line survey 24 months later. We observed health extension workers' consultations of sick 2-59 months old children. The analysis has evaluated if children with pneumonia, diarrhoea and malnutrition were assessed, classified and treated according to guidelines, and included difference-in-difference analyses. RESULTS: We observed 1325 consultations of sick children. At baseline, 86% of the sick children with cough in the intervention areas and 85% in comparison areas were assessed according to the guidelines, without any change at end-line associated with the intervention (difference-in-difference = -21%, p = 0.55). Sixty-two percent of children were assessed for dehydration at baseline in intervention and 47% in comparison areas, with no improvement associated with the intervention. Similarly, 87% of sick children in intervention and 91% in comparison areas were assessed for malnutrition, with no change over time associated with the intervention (difference-in-difference = 5%, p = 0.16). Appropriate pneumonia treatment with antibiotics declined and diarrhea treatment increased in both areas. Half of the malnourished children received ready-to-use therapeutic foods without any improvement associated with the intervention. CONCLUSION: The intervention was not associated with improved quality of the health extension workers' management of sick children. The lack of association may be linked to low fidelity in the implementation of the intervention. Our findings suggest that training healthcare providers without continued clinical mentoring and support does not improve the quality of care. Community-based programs can be strengthened by ensuring high coverage and continued clinical mentorships, supportive supervision, and supply of medicines and other essential commodities. TRIAL REGISTRATION NUMBER: ISRCTN12040912, retrospectively registered on 19/12/ 2017.