BACKGROUND: Due to low care utilization, a complex intervention was done for two years to optimize the Ethiopian Health Extension Program. Improved quality of the integrated community case management services was an intermediate outcome of this intervention through community education and mobilization, capacity building of health workers, and strengthening of district ownership and accountability of sick child services. We evaluated the association between the intervention and the health extension workers' ability to correctly classify common childhood illnesses in four regions of Ethiopia. METHODS: Baseline and endline assessments were done in 2016 and 2018 in intervention and comparison areas in four regions of Ethiopia. Ill children aged 2 to 59 months were mobilized to visit health posts for an assessment that was followed by re-examination. We analyzed sensitivity, specificity, and difference-in-difference of correct classification with multilevel mixed logistic regression in intervention and comparison areas at baseline and endline. RESULTS: Health extensions workers' consultations with ill children were observed in intervention (n = 710) and comparison areas (n = 615). At baseline, re-examination of the children showed that in intervention areas, health extension workers' sensitivity for fever or malaria was 54%, 68% for respiratory infections, 90% for diarrheal diseases, and 34% for malnutrition. At endline, it was 40% for fever or malaria, 49% for respiratory infections, 85% for diarrheal diseases, and 48% for malnutrition. Specificity was higher (89-100%) for all childhood illnesses. Difference-in-differences was 6% for correct classification of fever or malaria [aOR = 1.45 95% CI: 0.81-2.60], 4% for respiratory tract infection [aOR = 1.49 95% CI: 0.81-2.74], and 5% for diarrheal diseases [aOR = 1.74 95% CI: 0.77-3.92]. CONCLUSION: This study revealed that the Optimization of Health Extension Program intervention, which included training, supportive supervision, and performance reviews of health extension workers, was not associated with an improved classification of childhood illnesses by these Ethiopian primary health care workers. TRIAL REGISTRATION: ISRCTN12040912, http://www.isrctn.com/ISRCTN12040912.