Leveraging Machine Learning for Enhanced Population Health Monitoring in LMICs: Linking Demographic Surveillance and Clinic Records

Background: In low- and middle-income countries (LMICs), population health monitoring is important for healthcare policies and interventions. Traditional methods often face challenges in the retrospective integration of data from Health and Demographic Surveillance Systems (HDSS) and clinic records, limiting the depth of health trend analysis. Leveraging machine learning offers a solution to enhance population health monitoring by effectively linking demographic surveillance data with clinic records. This integration facilitates comprehensive insights into health trends, enabling more informed decision-making for healthcare professionals and policymakers in LMICs. Objective: This study aims to explore how machine learning (ML) techniques can retrospectively integrate demographic surveillance data and clinic records to enhance disease surveillance within specified populations in low- and middle-income countries (LMICs). Methods: Using real-world data in ML pipeline learning stages presents challenges due to quality concerns. This study aims to identify the most effective method for linking demographic surveillance data and clinic records, requiring a controlled data environment for rigorous testing and comparison. Synthetic datasets were generated to simulate real-world conditions, and categorized into error-free datasets, those with minor errors, and those with major errors. ML techniques were applied in the record linkage phase for efficient integration, employing feature engineering for data extraction and algorithmic comparison for matching. This comprehensive approach facilitated effective dataset linkage and model evaluation. The model can now be deployed on additional real-world datasets with the aim of enhancing disease surveillance within specified populations in LMICs Results: The study generated three synthetic datasets, each containing different levels of errors, to explore the effectiveness of ML techniques in retrospective record linkage. ML algorithms were evaluated for their adaptability, resilience, precision, and computational efficiency in handling these datasets. Conclusion: The integration of ML techniques for retrospective data analysis marks a significant step forward in enhancing population health surveillance capabilities within LMICs. This advancement holds promise for guiding targeted public health interventions aimed at bolstering disease control efforts in resource-limited settings.