City of Hope is transforming the practice of medicine by using artificial intelligence AI) to predict — with an accuracy beyond human capability — specific events that are likely to occur in the course of a patient’s treatment.
Three in-house-built AI models are currently used at City of Hope, one for sepsis, another for surgery complications and the third for the risk of mortality within 90 days for palliative care.
Predictive AI models are based on machine learning — training an algorithm to recognize patterns by processing millions of data points drawn from anonymous case histories. When real-time information about a current patient is available — their vitals, lab tests or scans, for instance — within milliseconds recalculated predictions, risk scores and AI explanations are provided to their care team.
Nasim Eftekhari, M.S., executive director of applied AI and data science at City of Hope, explained why applying AI is such a significant advance in patient care.
Years of peer-reviewed research worldwide have confirmed AI’s predictive abilities, and City of Hope leads the way in making it an everyday part of clinical care.
“Many organizations and academic centers have developed and published research papers on machine learning predictive models,” she said. “The part that is groundbreaking about what we’ve done is closing the loop of going from data to real world evidence to action, and continuously monitoring and optimizing these models based on clinicians’ feedback and new data generated from these models in action.”
The main challenge was not in developing these algorithms, but in figuring out how to access and process real-time patient information, then provide timely and effective feedback to medical staff, and monitor and optimize the impact. The solution was integrating these AI solutions with Epic, City of Hope’s electronic medical records system, every clinician’s working environment.
All new patient information is automatically picked up from Epic by the AI model. The predictions, explanations and suggested actions are then generated, and within milliseconds EPIC displays updated results for nurses, doctors and the rest of the care team.
The Sepsis Model
Sepsis is a condition, most commonly due to infection, that can develop without warning and progress so quickly that within hours it may lead to severe organ damage, even death.
Among the most vulnerable to sepsis are bone marrow transplant patients. Before stem cells can be transplanted, patients must undergo chemotherapy or radiation, and after the transplant, their immune system is repressed to prevent rejection.
Between 5% and 10% of transplant patients develop sepsis, and the risk of poor outcomes in those patients is very high. What’s more, these immunocompromised patients may have no fever or other signs of infection before progressing into serious sepsis.
Many years ago, City of Hope hematologist-oncologist Ryotaro Nakamura, M.D., became fascinated with the possibility of using AI to predict sepsis in transplant patients — but his interest wasn’t piqued by scientific journals.
“I was reading the Wall Street Journal, and noticed that if there was, say, a volcano that erupts in Japan, people knew from a computer program which stocks in the U.S. would go up or down,” said Nakamura, the Jan & Mace Siegel Professor in Hematology & Hematopoietic Cell Transplantation. “So I suggested at a department meeting that maybe we could have an algorithm to predict the risk of sepsis. At that time, it was such an unusual idea, people just stared at me, silent.”
The idea stayed with Nakamura, though, and eventually he found an ally and partner in Sanjeet Dadwal, M.D., professor and chief of the Division of Infectious Diseases, whose expertise was vital in creating a sepsis model. Together, they worked with the applied AI and data science team and Eftekhari, who spearheaded its development. The model has now been in use at City of Hope for almost two years.
“Patients are monitored on a regular basis, and the machine gives us a score for the probability of sepsis, based on the ROC curve,” Dadwal explained. “If a yellow or red alert appears on the Epic screen, we keep a closer watch on the patient and may make changes in antimicrobial use as indicated clinically.”
Since sepsis can develop before any symptoms become apparent, the feedback from this AI model can be lifesaving.
Predicting Surgery Complications
“As a surgeon, I think a lot about possible complications, ranging from bleeding and unhealed wounds to pneumonia, a cardiac event, disabilities, even death,” said City of Hope colorectal surgeon Lily Lau Lai, M.D. “Having a better ability to predict who is at risk for postoperative complications can help with preoperative optimization of the patient, as well as improve the consenting process for the patient and family.”
This drove Lai to propose an AI model for surgery complications.
Although there are published and validated models to assess for risk of postoperative complications, they were not ideal for City of Hope's population.
“The American College of Surgeons NSQIP Surgical Risk Calculator, for example, was developed in mostly noncancer patients,” said Lai, “Also, the calculator specifically looks at risk for single operations on one organ. We often have patients who require surgery on multiple organs at the same time. The calculator just wasn’t very accurate for us.”
City of Hope's AI surgery complications model went live in November 2021, and Lai finds it an indispensable tool.
“Processing millions of bits of information, so much more than the human mind is capable of, is really an amazing thing,” Lai said. “And having that data analyzed with a risk score will only help us make better decisions.”
90-Day Mortality Prediction
When people first hear about an AI model that can predict death within 90 days, their reaction is almost always, "Whoa, what?"
The reality, though, is not as shocking as it might seem. Even without AI, predicting mortality is something practitioners must assess in order to provide the best possible medical treatment and advance-care planning.
“This is all about what the patient wants, what their values are. Honoring a patient’s preferences is our key goal, and we work hard to understand and capture these through conversations and high-quality advance directives,” said palliative care physician Finly Zachariah, M.D., who helped develop the AI model. “We encourage use of Prepare for Your Care, a tool developed by researchers at University of California San Francisco. It’s age-friendly, simple to use and will produce a detailed directive that can be entered into our electronic medical record system.”
Historically, though, such preferences have often been ignored. A California Healthcare Foundation study asked people where they wanted to be when they died. Seventy percent said they preferred to die at home — but only 32% of deaths actually took place there, while 42% occurred in a hospital and 18% in a nursing home. One reason: Although almost 80% said they would want to talk with a doctor about their end-of-life wishes, only 7% actually had.
Patients near the end of life have treatment choices to make as well, which often involve quantity-of-life versus quality-of-life decisions.
“We obviously want to maximize the availability of new treatments and clinical trials, but when time may be short, does a person want to undergo the burdens of therapy or would they rather be at home with their loved ones?” Zachariah said. “The patient is the captain of the ship, and we want to respect their values, preferences and priorities, and assure that they are aligned with treatment choices and care delivery.”
What the Future Holds
“Deploying the first three models has given us the infrastructure, processes and policies to develop and deploy many, many more models with far less time and effort,” said Eftekhari.
One project in progress is a patient experience predictive model, which will factor in many patient variables, including demographics, socioeconomic factors, language barriers and other data the care team may not have timely access to.
Also under development are AI models that could help discover what is still unknown about genetic mutations and clarify subtle details in medical images. These models might explain such mysteries as why two patients — with identical disease and demographics — respond differently to the same treatment.
But will AI replace physicians? Absolutely not.
The AI models are formidable tools to help clinicians with their decision-making — but many aspects of a patient’s condition are not quantifiable.
“The machine is not a physician with years of bedside experience, accumulated knowledge and real-life experience of each patient,” said Dadwal. “That is what a machine cannot learn. That's where medicine becomes art.”