BACKGROUND: The 2014-2016 Ebola epidemic in West Africa was the largest Ebola epidemic to date. Contact tracing was a core surveillance activity. Challenges with paper-based contact tracing systems include incomplete identification of contacts, delays in communication and response, loss of contact lists, inadequate data collection and transcription errors. The aim of this study was to design and evaluate an electronic system for tracing contacts of Ebola cases in Port Loko District, Sierra Leone, and to compare this with the existing paper-based system. The electronic system featured data capture using a smartphone application, linked to an alert system to notify the District Ebola Response Centre of symptomatic contacts. METHODS: The intervention was a customised three-tier smartphone application developed using Dimagi's CommCare platform known as the Ebola Contact Tracing application (ECT app). Eligible study participants were all 26 Contact Tracing Coordinators (CTCs) and 86 Contact Tracers (CTs) working in the 11 Chiefdoms of Port Loko District during the study period (April-August 2015). Case detection was from 13th April to 17th July 2015. The CTCs and their CTs were provided with smartphones installed with the ECT app which was used to conduct contact tracing activities. Completeness and timeliness of contact tracing using the app were compared with data from April 13th-June 7th 2015, when the standard paper-based system was used. RESULTS: For 25 laboratory-confirmed cases for whom paper-based contact tracing was conducted, data for only 39% of 408 contacts were returned to the District, and data were often incomplete. For 16 cases for whom app-based contact tracing was conducted, 63% of 556 contacts were recorded as having been visited on the app, and the median recorded duration from case confirmation to first contact visit was 70 h. CONCLUSION: There were considerable challenges to conducting high-quality contact tracing in this setting using either the paper-based or the app-based system. However, the study demonstrated that it was possible to implement mobile health (mHealth) in this emergency setting. The app had the benefits of improved data completeness, storage and accuracy, but the challenges of using an app in this setting and epidemic context were substantial.