research
Journal Publication
-
Improving Access to Essential Medicines via Decision-Aware Machine Learning
Nature 653, 1178–1183 (research article).*denote equal last authorNature Nature News SSRN Behind the Paper
Media Coverage
Abstract
A critical challenge in healthcare systems in low- and middle-income countries (LMICs) is the efficient and equitable allocation of scarce resources, particularly essential medicines. This problem is complicated by limited high-quality data, which restricts the applicability of traditional data-driven techniques. We propose a novel decision-aware machine learning framework for essential medicines allocation, which additionally leverages multi-task learning to ensure sample efficiency and catalytic priors to ensure equitable allocation. In collaboration with the Sierra Leone national government, we performed a staggered, nationwide deployment of our system as a decision support tool. Our econometric evaluation finds an estimated 19% increase in consumption of allocated products in treated districts, demonstrating its efficacy at improving access to essential medicines. Our tool was subsequently scaled nationwide, covering an estimated 2 million women and children under five. Our work demonstrates how machine learning methods can improve efficiency at very low cost in resource-constrained global health settings.
Working Papers
-
Effective Personalized AI Tutors via LLM-Guided Reinforcement Learning
Available at SSRN and arXiv.*denote equal last authorSSRN arXiv
Media Coverage
- Mack Institute Research spotlight
- The Hechinger Report Proof Points
- Knowledge at Wharton How personalized AI tutors can help students learn
- Wharton AI & Analytics Why better AI tutors aren't about better answers
- 3-2-1 on AI and learning adaptive
- EdTech Insider EdTech, AI in education, adaptive learning
Abstract
Generative AI (GenAI) is rapidly reshaping education by unlocking the potential for personalized tutoring. Yet, emerging platforms largely focus on GenAI chatbot tutors that reactively answer student questions. We hypothesize that the efficacy of GenAI chatbot tutors can be substantially improved by proactively guiding student learning. To test this, we design a novel tutoring platform that tightly integrates a carefully-designed GenAI chatbot with a reinforcement learning algorithm for sequencing practice problems. Critically, this algorithm leverages rich signals from student-chatbot interactions to adaptively select practice problems of an appropriate difficulty level. In partnership with the Taipei City Government and American Institute in Taiwan, we deployed our tutoring platform in conjunction with a five-month course to teach Python to students across ten high schools. We randomized students between a fixed practice problem sequence and our adaptive sequencing algorithm. We find that adaptive sequencing increased unassisted final exam performance by 0.15 standard deviations (equivalent to 6-9 months of schooling by some estimates); mediation analysis suggests that gains were driven by increased engagement. Our work provides large-scale field evidence that student-chatbot interactions provide valuable signals for proactively optimizing and personalizing student learning.
-
The Impact of Human-AI Collaboration in Outpatient Care: Evidence from Somaliland
Draft available on request.
Abstract
Healthcare documentation is one of the major operational burdens in resource-constrained health systems, often competing directly with patient care. This paper evaluates a human-AI collaborative documentation tool implemented at the largest public hospital in Somaliland. Physicians enter a clinical note draft, from which AI generates a polished clinical note that physicians may directly submit, edit then submit, or abandon AI note and submit their own draft. Exploiting the staggered rollout of this tool across hospital departments, we estimate intent-to-treat (ITT) effects using a Callaway and Sant’Anna (2021) difference-in-differences design. Our results show that AI significantly reduces service time by 34% and reduces service time dispersions across encounters. These findings suggest that AI has the potential to increase effective service capacity and standardize care delivery. We find no evidence that these gains come at the expense of clinical care quality. In particular, provider surveys indicate greater patient interaction and clinical note quality improves substantially. However, AI-enabled faster service do not necessarily improve patient access and downstream operational benefits, for example, patient waiting time does not decrease significantly. We show that this is because physicians strategically respond to the reduced service time by shortening total working time and by starting clinical day later. Furthermore, we find significant improvement on physicians’ independently written notes (without AI) post-treatment, suggesting that improvement does not necessarily require adoption; exposure to AI (but without adoption) can be of great value as well.
-
Grand Challenges for Operations Management: UN Sustainable Development Goal 3
Minor Revision, Manufacturing & Service Operations Management.
-
Incentive-Compatible Human-AI Collaboration via Adversarial Tasks
Draft under preparation.
Refereed Conference Papers
-
Decision-Aware Learning for Optimizing Health Supply Chains
Machine Learning for Health (ML4H).arXiv
Abstract
We study the problem of allocating limited supply of medical resources in developing countries, in particular, Sierra Leone. We address this problem by combining machine learning (to predict demand) with optimization (to optimize allocations). A key challenge is the need to align the loss function used to train the machine learning model with the decision loss associated with the downstream optimization problem. Traditional solutions have limited flexibility in the model architecture and scale poorly to large datasets. We propose a decision-aware learning algorithm that uses a novel Taylor expansion of the optimal decision loss to derive the machine learning loss. Importantly, our approach only requires a simple re-weighting of the training data, ensuring it is both flexible and scalable, e.g., we incorporate it into a random forest trained using a multitask learning framework. We apply our framework to optimize the distribution of essential medicines in collaboration with policymakers in Sierra Leone; highly uncertain demand and limited budgets currently result in excessive unmet demand. Out-of-sample results demonstrate that our end-to-end approach can significantly reduce unmet demand across 1040 health facilities throughout Sierra Leone.
Book Chapter
-
Optimizing Health Supply Chains in LMICs with Machine Learning: A Case Study in Sierra Leone
In C. S. Tang (Ed.), Responsible and Sustainable Operations: The New Frontier (pp. 187-202). Springer Nature Switzerland.Full Chapter
Abstract
This chapter overviews the challenges in pharmaceutical supply chains (PSCs) in Low- and Middle-Income Countries (LMICs), with a focus on Sierra Leone. Furthermore, it describes how traditional supply chain optimization strategies can be used to improve performance of PSCs in Sierra Leone. Finally, it describes the significant potential for using machine learning in this framework for effective demand forecasting. We highlight challenges such as limited data availability, the need to ensure equitable distribution, as well as the potential for transfer learning to address some of these challenges.
Invited Paper
-
Application of AI in Healthcare Management in Developing Countries (in Chinese)
Development Focus Quarterly, Issue 20.
Abstract
Artificial intelligence (AI) is rapidly becoming a key technology for improving healthcare and public health services, especially in developing countries with limited medical resources. This article first integrates international reports and academic literature to summarize the core applications of AI in four areas: disease prevention, telemedicine, healthcare resource allocation, and health education. It then draws on the author’s field experience in Sierra Leone and Somaliland to explain how AI technologies such as decision-aware machine learning, large language models (LLMs), and reinforcement learning (RL) can substantially improve the efficiency and equity of healthcare services.
Despite its promising prospects, challenges such as insufficient data, weak infrastructure, lack of funding and talent, and incomplete regulatory and ethical frameworks still limit the implementation of AI in resource-constrained settings. To realize the benefits of AI while avoiding risks such as data privacy violations, algorithmic bias, and the widening of healthcare inequality, future development should focus on strengthening infrastructure, cultivating local talent, and establishing appropriate regulatory policies to ensure the ethical and inclusive use of AI, and to achieve the goals of health equity and sustainable development.
Work In Progress
- Human–AI Agentic Networks for Community Health in Somaliland
- Trust in AI for Resource Allocation Using Housing Images
- AI for Poverty Targeting