Going Beyond Diagnosis and Prognosis: Machine Learning to Guide Treatment Suggestions

Title: Going Beyond Diagnosis and Prognosis: Machine Learning to Guide Treatment Suggestions

Date: 2020-10-28

Series: Machine Learning in Medicine

Speaker: David Sontag

Summary and thoughts:

Dr. David Sontag from MIT presented at this week’s Machine Learning in Medicine series on the topic “Going Beyond Diagnosis and Prognosis: Machine Learning to Guide Treatment Suggestions”. While his MIT Clinical Machine Learning Group is involved in many areas of clinical machine learning such as clinical decision support, EHR documentation, risk stratification and more, this talk was focused on their work in guiding treatment recommendations and suggestions in the clinical workflow. Prior work in the area predominantly uses supervised learning approaches, however Dr. Sontag’s group aims to adapt algorothms developed for reinforcement learning to provide treatment suggestions. Traditional reinforcement learning algorithms are generally not robust and trustworthy enough to be deployed in a clinical environment. The goal of the project is thus to create robust algorithms that learn simple treatment policies from retrospective health data and are able to -

• understand clinician policies and characterize variation across individuals and institutions
• use machine learning to discover better treatment strategies
• do both directly on top of EMR so that decision support can be seamlessly given to clinicians and patients

Using the case study of empiric antibiotic treatment for urinary tract infection (UTI), Dr. Sontag presented a data-driven reinforcement learning approach that outperforms the traditional guidelines for treatment of UTI as well as clinician recommendations. The broad steps to the solution include learning direct individualized treatment rules (ITR) through direct policy learning that maps patient features and policy to treatment decisions, trade off with multiple competing goals (maximizing effectiveness vs avoiding broad spectrum antibiotics), evaluation with target deployment approach and simplifying the deployment process. Patient features from EHR data (demographics, comorbidities, lab tests, location, past antibiotics, past resistance) along with reward vectors (from retrospective patient antibiotic susceptibility profiles) are used for direct policy learning to maximize the expected rewards as in any reinforcement learning algorithm. The policy is then distilled to use a fraction of the original features required for machine learning to identify a simple treatment policy that achieves competitive performance.