A new algorithm created by Johns Hopkins scientists predicts which COVID-19 patients will become gravely ill

In late January, during the worst of the pandemic's winter surge, infectious diseases specialist Matthew Robinson pulled two extra 11-hour overnight shifts caring for several dozen people hospitalized with COVID-19. Soon after arriving at 9:30 p.m., he logged into a novel computer program he helped develop and—in an instant—real-time clinical data on every coronavirus patient on the ward flashed across the screen.

A colored circle with a number in it appeared next to the name of each patient. Most of the circles were yellow or green, with low numbers. But a handful were bright orange or red, with uncomfortably high numbers. Robinson knew those were the ones he had to worry about. The colors and percentages within them show the probability that a hospitalized COVID-19 patient will become severely ill or die, either within the next 24 hours or the coming week. Yellow or green circles mean the risk is moderate to low, while orange or red circles signal danger.

"Those with orange or red were the ones I needed to pay the most attention to overnight," says Robinson, an assistant professor of medicine in the Johns Hopkins School of Medicine. "Without that algorithm, I would have struggled to know who [the sickest patients] were."

The algorithm is a new computer program known as SCARP, for Severe COVID-19 Adaptive Risk Predictor, developed by Robinson and other scientists at the Johns Hopkins University School of Medicine and the Bloomberg School of Public Health. It alerts clinicians to the likelihood a COVID-19 patient faces an imminent bad outcome and helps them prioritize treatments. The software is available to all Hopkins clinicians and to outside clinicians using it for academic or research purposes. "It makes it easier for clinicians to anticipate what will happen to patients and helps them focus on patients who are the sickest," Robinson says.

The seven-day prediction also provides an opportunity for physicians to prepare people for the possibility they might have to go into the intensive care unit. "It's a really important predictor when you are having discussions with families and patients about what the next week may hold," says Brian Garibaldi, an associate professor of medicine in the Hopkins School of Medicine and director of the Johns Hopkins Biocontainment Unit. He also helped develop and test the tool. "Having those types of conversations early helps us know whether we're doing the right thing, and makes it a lot easier to help patients and their families navigate the uncertainties and different treatments."

SCARP was created using machine learning (a type of artificial intelligence where the computer is trained to make decisions using experiences based on data) with input from 3,163 COVID-19 patients admitted to five hospital sites within the Johns Hopkins Health System between March 5 and Dec. 4 of last year.

"While most patients with COVID-19 recover with supportive care, the course of COVID-19 is highly variable, and it is difficult to predict who will deteriorate," says Shannon Wongvibulsin, a Hopkins medical student who received a PhD in biomedical engineering from the School of Medicine in 2019. She played a major role in developing the methods and designing the particular algorithm. "Our clinical risk prediction tool allows health care providers to obtain real-time predictions for disease progression in patients hospitalized with COVID-19," she says.

Wongvibulsin, who currently conducts research in computational medicine, performed the machine learning analysis to predict the risk of a patient progressing to severe disease or death and created visualizations to help communicate how the predictions were generated and which variables were the most important.

To predict changes in the disease's course within each patient, the algorithm incorporates more than 100 patient features and clinical characteristics over time during hospitalization. In forecasting the outcome for current hospitalized patients, the most critical piece of information analyzed is the efficiency of lung function. "What we found was the most predictive variable in most scenarios is how low your oxygen level is and how much supportive oxygen you are receiving," Robinson says.

The program uses data from previous patients as well as current hospitalized patients, including blood oxygen levels, vital signs, lab measurements, and information from their electronic medical records. Machine learning models use existing datasets from previous patients where possible predictor variables and outcomes are known. Data from the electronic medical records at the five Hopkins hospitals was used for this, according to Robinson. Currently, there are two versions of the program, the first a website that allows individual clinicians to manually enter information about their patients to get a prediction; the other pulls information directly and automatically from a patient's electronic medical records.

"For the first time in the history of medicine we are capturing all the decisions made by physicians, and all of the measurements and information collected, starting from when a patient first showed up, and storing it digitally," says Scott Zeger, a professor of biostatistics and medicine in the Bloomberg School of Public Health who is also a member of the SCARP team.

As such, it is a significant adjunct to—though not a substitute for—a physician's observations, clinical background, and individual subjective judgments. "If you are sick with something, the first thing you ask the doctor is, 'Am I going to be OK?' and the doctor often relies on clinical experience and intuition," Robinson says. "But when [COVID-19] first came out, there was no clinical experience and intuition. None of us had ever seen it before."

Garibaldi says physicians who took care of COVID-19 patients early on felt a heavy responsibility to pay attention to—and remember—their patients' experiences, and apply that knowledge to other patients as the pandemic intensified. "These observations lead you to ask better questions and try to go beyond what we can see with our own eyes," he says. "Matt and Shannon's model is an incredible example of how you can learn from patients you've cared for and use machine learning to tell you something you wouldn't know just by looking at the patients."

The researchers believe the approach represents a fundamental change in the practice of medicine that will enhance how the field responds to other disease challenges, both now and in the future.

"In old medicine, the doctor observed one patient's information, then made subjective qualitative judgments and decisions, with all the reasoning going on inside the doctor's head," Zeger says. "Today that same doctor's reasoning is made more precise by the automatic collection and analysis of data from a population of similar patients, and by the application of precise logic. The COVID risk tool and others like it are transformational."

Garibaldi agrees. "We'll be ready to go for the next unknown, but why wait for the unknown?" he says. "We should be able to take what we've learned and use it on other diseases we know we'll be seeing in the hospital for years to come."