“The only certainty is uncertainty.”
Pliny the Elder, Roman author and philosopher
Prior to the COVID-19 pandemic, the electronic health record titan Epic had introduced an algorithm called Deterioration Index that was meant to help clinicians determine which patients needed more urgent attention and care.
This index is essentially a predictive model in Epic that gathers data from patients such as age, vital signs, nursing assessments, and laboratory results and provides an overall risk score by using machine learning. This index ranges from 0 to 100 but the parameters of the model and how the model was developed have not been disclosed to the public. The index was adopted by several health systems in the chaotic days of the pandemic to help triage the sick patients that were coming in to the hospitals in such large numbers that the clinicians were overwhelmed. What is of some concern is that the model validation and peer review processes are suboptimal in this adoption and therefore vulnerable to bias and other issues.
While such a tool has potential to be valuable in the triage process, especially during the busy days of the pandemic, there still needs to be room for accommodating human insight and cognition. First, the seasoned clinicians can often “eyeball” a patient and immediately know just how dire the medical condition and stability are based on observations that are not necessarily recorded in the electronic records and/or included in the deterioration index. While respiratory rate may be useful as a vital sign in the deterioration index, the degree of retractions may not be included in the index. There is also no weighting of these factors that perhaps will increase the accuracy of prediction (oxygen saturation and trend during the acute COVID-19 illness could be the single most valuable predictor and probably much more relevant than serum potassium).
In addition, the deterioration index and its velocity of deterioration may need to be factored in the triage process. One patient that has an index of 56 that has increased to 66 in two hours may not be as critical as someone who had a lower initial index of 36 that increased to 62 in less than 30 minutes (even though the latter had a lower index). Finally, medicine, especially with unstable and unpredictable situations, is complex and a combined human cognition and artificial intelligence (clinical or medical intelligence) is most helpful for managing these patients. For example, a patient who has a paralyzed right hemidiaphragm may not have this important condition in the deterioration index but who will almost for sure be at higher risk with the same degree of pulmonary illness (compared to someone without this condition).
In short, while these indices may or may not be ready for the prime time in acute clinical medicine, these indices (no matter what the AUC may be) should not be used as the only tools in triage or management of patients especially since there can be a myriad of issues such as concept drift and model fatigue in a rapidly evolving disease like COVID-19.
