AIM: Pneumonia is the leading infectious cause of death among children under five globally. Many pneumonia deaths result from inappropriate treatment due to misdiagnosis of signs and symptoms. This study aims to identify whether health extension workers (HEWs) in Ethiopia, using an automated multimodal device (Masimo Rad-G), adhere to required guidelines while assessing and classifying under five children with cough or difficulty breathing and to understand device acceptability. METHODS: A cross-sectional study was conducted in three districts of Southern Nations, Nationalities, and Peoples' Region, Ethiopia. Between September and December 2018, 133 HEWs were directly observed using Rad-G while conducting 599 sick child consultations. Usability was measured as adherence to the World Health Organization requirements to assess fast breathing and device manufacturer instructions for use. Acceptability was assessed using semi-structured interviews with HEWs, first-level health facility workers and caregivers. RESULTS: Adherence using the Rad-G routinely for 2 months was 85.3% (95% CI 80.2, 89.3). Health workers and caregivers stated a preference for Rad-G. Users highlighted a number of device design issues. CONCLUSION: While demonstrating high levels of acceptability and usability, the device modifications to consider include better probe fit, improved user interface with exclusive age categories and simplified classification outcomes.