In the midst of the buzz swirling around artificial intelligence (AI) –whether it’s overselling AI’s disease prediction capabilities or pointing out its fallibility and inherent biases—lies the knowledge that, with thorough validation and continual assessment, AI does have the potential to better predict and identify sepsis.

So argue two University of Michigan researchers: Sachin Kheterpal, MD, MBA, Professor of Anesthesiology, and Karandeep Singh, MD, MMSc, Assistant Professor of Learning Health Sciences and Internal Medicine. They, along with Eric Topol, MD, founder and director of the Scripps Research Translational Institute, have published a perspective in the Lancet that outlines key considerations for more effective AI-based prediction and management of sepsis.

One is prioritizing external validations so that variations in model performance across health centers can be better understood before an AI model is widely adopted. Another is timing: changes in documentation patterns, clinical practice patterns, and patient characteristics over time can affect how AI models behave. When the COVID-19 pandemic’s first wave arrived, it was imperative for health systems to identify patients who might have required intensive care in a timely manner, given the rapid deteriorations in health occurring among hospitalized patients. Figuring out whether deterioration models were working well in this new patient population was a top priority. In the midst of this focus on in-hospital deterioration, however, COVID-19 also led to changes in the behavior of sepsis models. Singh states, “what these changes in model behavior over time tell us is that we need to keep an eye on models through a structured model monitoring process. We cannot just set it and forget it.”

One of the other challenges in sepsis models is that they are used for multiple purposes. The authors note, “AI in sepsis spans both prediction and case identification. In some patients, the goal is to identify impending sepsis …. Conversely, some patients present to the emergency department in septic shock and the AI goal is to identify and classify presence of the disease.” Singh says that “depending on whether the intended use of a sepsis model is to start antibiotics, give intravenous fluids, start vasopressors, or identify a cohort of patients with sepsis after the fact, the time point at which a prediction needs to be relevant may differ quite a bit.”

Randomized controlled trials (RCTs) of sepsis prediction models would go a long way to improve a model’s performance and provide evidence of its effectiveness, yet RCTs of AI models in hospitalized patients have been rare. According to Singh, “The rarity of RCTs in AI models in healthcare is in part due to the technical challenges that have historically made models difficult to integrate within the electronic health record, and in part due to the close coordination that is required between researchers, informaticians, and clinicians to design and deliver a potentially effective intervention.”

Studying past mistakes, such as algorithm biases and lack of validation, and being more vigilant in the future, can make an AI-based prediction model a viable tool in preventing and treating sepsis.