If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Performance of Early Warning Scoring Systems Regarding Adverse Events of Unanticipated Clinical Deterioration in Complementary and Alternative Medicine Hospitals
Correspondence to: Jee-In Hwang, PhD, College of Nursing Science, Kyung Hee University, 26 Kyungheedae-ro Dongdaemoon-gu, Seoul, 02447, Republic of Korea
This study aims to examine the performance of early warning scoring systems regarding adverse events of unanticipated clinical deterioration in complementary and alternative medicine hospitals.
Methods
A medical record review of 500 patients from 5-year patient data in two traditional Korean medicine hospitals was conducted. Unanticipated clinical deterioration events included unexpected in-hospital mortality, cardiac arrest, and unplanned transfers to acute-care conventional medicine hospitals. Scores of the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and National Early Warning Score 2 (NEWS2) were calculated. Their performance was evaluated by calculating areas under the receiver-operating characteristic curve for the event occurrence. Multiple logistic regression analyses were performed to determine the factors associated with event occurrence.
Results
The incidence of unanticipated clinical deterioration events was 1.1% (225/21101). The area under the curve of MEWS, NEWS, and NEWS2 was .68, .72, and .72 at 24 hours before the events, respectively. NEWS and NEWS2, with almost the same performance, were superior to MEWS (p = .009). After adjusting for other variables, patients at low-medium risk (OR = 3.28; 95% CI = 1.02–10.55) and those at medium and high risk (OR = 25.03; 95% CI = 2.78–225.46) on NEWS2 scores were more likely to experience unanticipated clinical deterioration than those at low risk. Other factors associated with the event occurrence included frailty risk scores, clinical worry scores, primary medical diagnosis, prescribed medicine administration, acupuncture treatment, and clinical department.
Conclusions
The three early warning scores demonstrated moderate-to-fair performance for clinical deterioration events. NEWS2 can be used for early identification of patients at high risk of deterioration in complementary and alternative medicine hospitals. Additionally, patient, care, and system factors need to be considered to improve patient safety.
]. Unexpected in-hospital mortality, cardiac arrest, and unplanned transfer to higher-acuity units, such as intensive care units in hospitalized patients, are serious adverse events [
]. Since physiological instability is usually preceding clinical deterioration, therefore early warning scoring systems (EWSs), which are mainly based on routinely collected vital sign observations, have been developed to identify clinically deteriorating patients and avoid preventable adverse events [
The performance of the National Early Warning Score and National Early Warning Score 2 in hospitalised patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).
The use of early warning system scores in prehospital and emergency department settings to predict clinical deterioration: a systematic review and meta-analysis.
The use of early warning system scores in prehospital and emergency department settings to predict clinical deterioration: a systematic review and meta-analysis.
]. For instance, studies have found that widely used EWSs of the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and NEWS version 2 (NEWS2) demonstrated good ability to predict clinical deterioration in acute-care conventional medicine hospitals [
A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
]. These findings may indicate the necessity of additional studies on the performance of EWSs when we consider implementing an EWS in practice.
Along with aging and increased chronic diseases, utilization of complementary and alternative medicine (CAM), also known as traditional medicine, has also increased [
]. Studies revealed that approximately 11.0% of inpatients in CAM hospitals have experienced adverse events, including unanticipated clinical deterioration [
]. Although the comprehensive plans for patient safety based on the Patient Safety Act in Korea recommend the establishment of a rapid response team in hospitals [
], there is no study on a rapid response system in CAM hospitals. Furthermore, there is a lack of data regarding the serious adverse events attributable to unanticipated clinical deterioration in CAM hospitals. In this context, a tool for predicting sudden clinical deterioration is critical for improving patient safety. Although existing EWSs have been validated, the performance of EWSs is unknown in CAM hospitals.
Researchers have suggested that patient, care, and system factors may affect clinical deterioration and patient outcomes [
]. However, there are inconsistent findings. While some studies showed that severity of illness, comorbidity, and type of care were significant factors associated with the occurrence of unanticipated clinical deterioration and patient outcomes [
]. These findings may indicate the necessity of additional study. A better understanding of the factors associated with the occurrence of unanticipated clinical deterioration will help identify high-risk patients and reduce preventable serious adverse events.
This study aimed to explore the performance of EWSs as predictors of unanticipated clinical deterioration in CAM hospitals. Specifically, we examined the performance of MEWS, NEWS, and NEWS2 in CAM hospitals. Furthermore, we investigated the factors associated with unanticipated clinical deterioration. The findings of this study will contribute to facilitating early detection and timely response to patients at risk of sudden clinical deterioration, thereby improving patient safety in CAM hospitals.
Methods
Research design
We employed a retrospective review research design. A medical record review of (1) all patients with events of unexpected clinical deterioration during the 5-year period in two TKM hospitals and (2) a random sample of patients without events during the same period was conducted.
Sample and setting
The sample comprised 500 patients from two university-affiliated TKM hospitals from January 1, 2015, to December 31, 2019. The study hospitals had electronic medical record systems with full accredited status by the Korea Institute of Healthcare Accreditation. The annual number of inpatients was 2,177 in Hospital A and 1,006 in Hospital B in 2020. Nurse staffing level was grade 3 (3.0–3.5 patients per nurse) and grade 2 (2.5–3.0 patients per nurse). TKM subspecialty includes internal medicine, gynecology, pediatrics, eyes-ear-nose-throat-dermatology, neuropsychology, rehabilitation, Sasang constitutional medicine, and acupuncture.
We included patients aged ≥19 years with a length of stay ≥1 day. Unanticipated clinical deterioration events included unexpected in-hospital mortality, cardiopulmonary resuscitation, and unplanned transfer to a higher-acuity bed outside CAM hospitals due to deteriorating conditions. Exclusion criteria were (1) cases with a do-not-resuscitate order and (2) planned transfers to other hospitals due to other reasons except for clinical deterioration. We included only transfers to university-affiliated conventional medicine hospitals or tertiary hospitals due to deteriorating conditions. If the reason for the transfer was not clearly recorded, the inclusion of the case was decided at the discretion of our research team. Patients without events were randomly sampled from patients residing in the same care unit during the same period using the Research Randomizer [
A priori sample size of approximately 500 was determined based on the recommendations of 10 to 20 cases per predictor in multiple logistic regression [
]. The final sample consisted of 500 patients (225 with events and 275 without events) (Figure 1).
Figure 1Flow Chart of the Sample Selection in the Hospital A and Hospital B. Note. ICU = intensive care unit; U-CMH = university-affiliated conventional medicine hospital; CMH = conventional medicine hospital.
]. In addition to NEWS parameters, NEWS2 includes the SpO2 scale 2 for patients with hypercapnic respiratory failure and new confusion as part of the assessment of consciousness [
]. Based on the predefined criteria in their scoring systems, a score (range = 0–3) was assigned for each parameter and then a risk stratification was determined for the summed scores. MEWS scores (range = 0–14) were categorized into low (0–2), medium (3–4), and high (≥5) risk groups [
]. NEWS and NEWS2 scores (range = 0–20 for both systems) were categorized into low (0–4), low-medium (extreme score of 3 in a single parameter), medium (5–6), and high (≥7) risk groups [
Medical records were reviewed by two nurses and three TKM physicians in the hospitals. The first author (JH), who had expertize in the application of early warning scoring systems and medical record review methodologies, trained the reviewers individually in two half-day sessions in each hospital, using the subsets of the medical records assigned to each reviewer. It was challenging to train them simultaneously due to work schedules and the COVID-19 pandemic. Inter-rater reliability between the first author and each reviewer was assessed using kappa values. The values ranged from 0.84 to 0.97, indicating the “almost perfect agreement” of 0.81–0.99 [
We calculated the MEWS, NEWS, and NEWS2 scores 24 hours before the events. For the usual group, the scores were calculated using vital sign observations at 24 hours postadmission to obtain the highest score based on previous studies [
]. For cases with no recording of certain vital sign values, we used the values at the closest time point. If SpO2 values were null, score 0 for this parameter was assigned [
]. Patient- and disease-related variables included age, gender, educational level, body mass index (BMI), frailty risk, clinical worry score (CWS), primary medical diagnosis, comorbidities, and admission route. Frailty risk was measured using the Morse fall scale (MFS) and Braden scale scores at admission [
]. If there were no recordings of these scores at admission, the values at the closest time point was used. Nine cases with no recording on the frailty risk were considered as low risk after reviewing the records. CWS was measured using the Dutch-Early-Nurse-Worry-Indicator-Score scale comprising of nine indicators of respiration change, circulation change, temperature, mentation change, agitation, pain, unexpected trajectory, patient comments, and subjective observations [
]. After the presence of each indicator was coded as 1 and the other as 0, a sum score (range = 0–9) was calculated. The Charlson comorbidity index (CCI) was calculated [
Care- and treatment-related variables included the use of prescribed medicine, herbal medication, acupuncture, moxibustion, cupping, physical therapies (e.g., infrared therapy, transcutaneous electrical nerve stimulation, interferential current therapy, and chuna), and length of hospital stay. The uses of prescribed medicine and herbal medication were reviewed for the following day postadmission to reflect the acuity, which was also consistent with the time point for EWSs’ calculation [
]. Since there was no intensive care unit, surgical operation room, and rapid response team in the CAM hospitals, variables on care intensity and types were not included. System- and organization-related variables included hospital type, nurse staffing level, and clinical department. Since the nurse staffing level was determined at the hospital level, we included only the hospital type. We also collected event-related data such as the time of event occurrence (07:01–15:00, 15:01–18:00, 18:01–22:00, and 22:01–07:00), the day of the week (weekday and weekend), and length to event occurrence postadmission.
A random sample of 52 medical records were rereviewed by the first author. The overall agreement rate was 75.0% (39/52) at the patient level and 96.5% (637/660) at the indicator level of CWS.
Ethical considerations
The study protocol was approved by the Institutional Review Boards of two study hospitals (no. 2019-12-001 and 2019-12-005).
Data analysis
Data were analyzed using the SAS program (version 9.4; Cary, NC, USA). Patients' general characteristics were summarized using descriptive statistics. Interrater reliability was calculated using Kappa values [
]. Independent t tests and chi-square tests were conducted to identify differences between the event group and the usual group according to patients’ general characteristics. We calculated the areas under the receiver operating characteristic curves (AUCs) to evaluate the performance of MEWS, NEWS, and NEWS2.
Multiple logistic regression analyses were performed to determine factors associated with the occurrence of unanticipated clinical deterioration. Model calibration was assessed using the Hosmer-Lemeshow goodness-of-fit test. Odds ratios (ORs) and 95% confidence intervals (CIs) were also calculated. For a sensitivity analysis, we performed additional analysis after excluding cases with missing data on frailty risk scores (n = 9). Statistical significance was set at a two-tailed p < .05.
Results
Patients’ general characteristics and event characteristics
The patients' general characteristics are shown in Table 1. Of the 500 patients, 56.4% were women, with a mean age of 63.72 ± 16.01 years (range = 19.00–98.00). The mean length of hospitalization was 25.27 days (95% CI = 21.57–28.97). Most patients (89.4%) were discharged with improved status. There were significant differences between the event group and the usual group by patients’ gender, age, education, BMI, fall risk, pressure ulcer risk, CWS, primary medical diagnosis, CCI, admission route, prescribed medicine use, uses of CAM treatment, length of stay, and clinical department (Table 1).
Table 1Patients’ General Characteristics and Differences Between Event Group and Usual Group.
The incidence of unanticipated clinical deterioration events was 1.1% (i.e., 225 out of 21101 patients over the 5-year period). The time spent to event occurrence since admission was a median of 14.00 days (interquartile range = 5.79–38.00). Most events occurred on weekdays (n = 195, 86.7%) and during daytime between 07:01 and 18:00 (n = 182, 80.9%).
Performance of EWSs
MEWS, NEWS, and NEWS2 well discriminated patients with events from those without (Table 2). The AUC was .68 (95% CI = .64–.72), .72 (95% CI = .67–.76), and .72 (95% CI = .67–.76), respectively (Figure 2). In pairwise comparisons, there were significant differences between NEWS2 and MEWS (p = .009), but there was no difference between NEWS and NEWS2.
Table 2Performance of the MEWS, NEWS, and NEWS2.
Variable
n
%
Event group
Usual group
χ2/t
p
AUC
n
%
n
%
MEWS, mean (95% CI)
1.79
(1.65−1.94)
1.13
(1.07−1.19)
−8.34
<.001
0.68
Low risk
446
89.2
175
35.0
271
54.2
55.50
<.001
Medium risk
50
10.0
46
9.2
4
0.8
High risk
4
0.8
4
0.8
0
0.0
NEWS, mean (95% CI)
2.50
(2.16−2.85)
0.64
(0.53−0.74)
−10.20
<.001
0.72
Low risk
413
82.6
148
29.6
265
53.0
83.85
<.001
Low-medium risk
42
8.4
33
6.6
9
1.8
Medium risk
26
5.2
25
5.0
1
0.2
High risk
19
3.8
19
3.8
0
0.0
NEWS2, mean (95% CI)
2.48
(2.14−2.82)
0.64
(0.53−0.74)
−10.16
<.001
0.72
Low risk
414
82.8
149
29.8
265
53.0
82.19
<.001
Low-medium risk
42
8.4
33
6.6
9
1.8
Medium risk
26
5.2
25
5.0
1
0.2
High risk
18
3.6
18
3.6
0
0.0
Note. AUC = area under the receiver operating characteristic curve; CI = confidence interval.
Factors associated with unanticipated deterioration event occurrence
Based on the findings of the univariate analyses, multiple logistic regression analyses were performed. The “medium (5–6)” and “high (≥7)” risk stratification on NEWS2 scores were merged into one category (≥5) due to low frequency of the categories. The results showed that NEWS2, fall risk, pressure ulcer risk, CWS, primary medical diagnosis, use of prescribed medicine, acupuncture treatment, and clinical department were significant factors associated with event occurrence (Max-rescaled R2 = 59.0%, p < .001; c-statistic = 0.90; Hosmer-Lemeshow goodness-of-fit test, p = .057).
Specifically, patients at low-medium risk (OR = 3.28; 95% CI = 1.02–10.55) and those at medium and high risk (OR = 25.03; 95% CI = 2.78–225.46) on NEWS2 scores were more likely to experience unanticipated deterioration events than patients at low risk. In addition, patients at medium risk on the MFS were more likely to experience the events than those at low risk on the MFS (OR = 2.98; 95% CI = 1.48–6.02). Those at a high risk of pressure ulcer on the Braden scale were more likely to experience the events than others (OR = 4.33; 95% CI = 2.12–8.84). Patients with higher CWSs were more likely to experience the events (OR = 1.91; 95% CI = 1.29–2.84). Patients with “circulatory system diseases” (OR = 3.44; 95% CI = 1.64–7.23), “neoplasm” (OR = 12.98; 95% CI = 3.92–43.00), and “injuries and other consequences of external causes” (OR = 5.21; 95% CI = 1.43–18.94) were more likely to experience the events than those with “others” diseases (Table 3).
Table 3Logistic Regression Results for Unanticipated Clinical Deterioration Event Occurrence.
Variable
Odds ratio
95% Confidence interval
NEWS2
Medium and high risk
25.03
(2.78–225.46)∗
Low-medium risk
3.28
(1.02–10.55)∗
Low risk
reference
Gender
Men
1.28
(0.72–2.27)
Women
reference
Age (years)
1.00
(0.98–1.03)
Education level
College or higher
0.82
(0.37–1.84)
High school
0.55
(0.26–1.20)
Others
0.67
(0.27–1.64)
Middle school or lower
reference
Body mass index
0.96
(0.89–1.04)
Fall risk
High
2.07
(0.86–4.49)
Medium
2.98
(1.48–6.02)∗
Low
reference
Pressure ulcer risk
High
4.33
(2.12–8.84)∗
Low
reference
Clinical worry score
1.91
(1.29–2.84)∗
Primary medical diagnosis
Circulatory system disease
3.44
(1.64–7.23)∗
Nervous system disease
1.80
(0.69–4.74)
Neoplasm
12.98
(3.92–43.00)∗
Musculoskeletal system disease
1.37
(0.43–4.41)
Injuries, other consequences of external causes
5.21
(1.43–18.94)∗
Others
reference
Charlson comorbidity index
1.07
(0.95–1.20)
Admission route
Outpatient department
0.98
(0.37–2.58)
Others
reference
Use of prescribed medicine
4.34
(1.42–13.28)∗
Complementary and alternative medicine treatment
Acupuncture
0.09
(0.01–0.97)∗
No acupuncture
reference
Moxibustion
0.74
(0.39–1.41)
No moxibustion
reference
Cupping
1.03
(0.57–1.86)
No cupping
reference
Physical therapy
0.79
(0.44–1.43)
No physical therapy
reference
Length of stay (days)
1.00
(0.99–1.01)
Clinical department
Rehabilitation
0.26
(0.12–0.60)∗
Acupuncture
0.73
(0.32–1.65)
Others
0.71
(0.24–2.07)
Internal medicine
reference
Note. This analysis was performed using 435 patient data due to the missing values of body mass index.
Patients treated with prescribed medicine in CAM hospitals were more likely to experience the events than those without (OR = 4.34; 95% CI = 1.42–13.28). In addition, those receiving acupuncture treatment (OR = 0.09; 95% CI = 0.01–0.97) and those who were admitted to CAM rehabilitation department (OR = 0.26; 95% CI = 0.12–0.60) were less likely to experience unanticipated deterioration events (Table 3).
Discussion
Improving patient safety in CAM practices is a global concern [
]. Early identification of patients at high risk of unanticipated clinical deterioration and appropriate responses are important to avoid unnecessary harm to patients. This is the first study to explore the performance of MEWS, NEWS, and NEWS2 for unanticipated clinical deterioration in CAM hospitals. Moreover, this is the first report on the incidence of unanticipated clinical deterioration events in CAM hospitals.
In this study, the EWSs showed moderate to fair ability in predicting unanticipated clinical deterioration in CAM hospitals. The study found that patient, care, and system factors, along with EWSs, were significantly associated with the occurrence of unanticipated clinical deterioration. These findings indicate that using EWSs can help in the early identification of patients at risk of deterioration in CAM hospitals. Furthermore, patient, care, and system factors (i.e., frailty risk scores, CWS, primary medical diagnosis, treatment modality, and clinical department) should be considered along with EWS scores to better identify patients at high risk of clinical deterioration and to avoid preventable adverse events in CAM hospitals.
Although three EWSs showed reasonable discrimination ability, their AUCs were lower than .8, indicating a threshold of good performance [
The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
]. In this study, most events were transfer cases, and nearly half scored 0 on the NEWS2. In addition, routine vital sign measurements are sometimes performed twice a day in CAM hospitals. Thus, vital sign values for EWSs’ calculation can be distant from the time point of 24 hours before the event. These might result in low prognostic accuracy of the EWSs. Specifically, NEWS and NEWS2, showing better performance than MEWS, demonstrated similar performance. This was different from the findings of previous studies [
A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
]. A possible reason can be due to the limited number of patients with hypercapnic respiratory failure. Since NEWS2 has the benefit of a separate oxygen saturation scoring system for COPD patients, the use of NEWS2 is suggested.
Additionally, we examined the improvement in EWSs’ models by including other variables. Based on existing research [
Early warning score adjusted for age to predict the composite outcome of mortality, cardiac arrest or unplanned intensive care unit admission using observational vital-sign data: a multicentre development and validation.
], adding an age variable to the models improved their performance, indicating fair discrimination (AUC = .76, .77, and .77 for MEWS, NEWS, and NEWS2, respectively). However, the values were still lower than .8. Therefore, future studies are necessary to further improve the prognostic accuracy of EWSs in CAM hospitals.
Significant factors to predict unanticipated clinical deterioration included NEWS2 scores, frailty risk scores, CWS, primary medical diagnosis, treatment modality, and clinical department. After controlling for other characteristics, patients with NEWS2 scores of ≥5 were approximately 25 times more likely to experience unanticipated clinical deterioration events than those in the low-risk group on NEWS2 scores of 0–4. Additionally, those with extreme scores of any single parameter were slightly over three times more likely to experience the events. This finding supports the relevance of NEWS2 as a monitoring tool for clinical deterioration.
Among patient factors, MFS scores were positively associated with clinical deterioration event occurrence. In particular, patients at medium-level fall risk experienced unanticipated clinical deterioration more frequently than those at low-level fall risk. In addition, patients at high risk of pressure ulcers were more likely to experience unanticipated clinical deterioration. This was similar to the finding that the Braden score was a predictor of adverse events in geriatric surgical patients [
]. However, these findings were different from the finding that the MFS and Braden scale scores were not significant independent factors for predicting mortality in patients with heart failure [
]. Since recordings on fall and pressure ulcer risk assessments were not mandatory in the hospitals during the study period, we used frailty risk scores at admission. In addition, we used MFS and Braden scale scores as readily available data on frailty risk [
]. We recommend that future studies on the predictive abilities of frailty risk scores 24 hours before adverse events be conducted using other measures developed for frailty risk assessments [
Development and validation of a Hospital Frailty Risk Score focusing on older people in acute care settings using electronic hospital records: an observational study.
]. These findings indicate the importance of nurses’ assessment on subjective and objective signs and symptoms of patients. Therefore, the use of a systematic tool to measure clinical concerns needs to be considered for early detection of unanticipated deterioration events. Patients with circulatory system diseases, neoplasms, and injuries experienced unanticipated clinical deterioration more frequently than others. This is similar to the findings of previous studies [
]. These findings might reflect disease characteristics. Therefore, patients with such diseases need to be monitored closely.
In relation to care and treatment, patients who needed prescribed medicine were more likely to experience unanticipated deterioration events. Studies have recommended the use of integrated therapies in CAM hospitals, rather than CAM therapies alone, especially for inpatients whose conditions are severe and highly complex, such as intractable cancer patients [
]. In this regard, the use of prescribed medicine during hospitalization in CAM hospitals may reflect patient conditions with high morbidity, which can easily contribute to sudden deterioration. Therefore, patients who require prescribed medication to be administered in addition to CAM therapies need to be closely observed for deterioration risk. In addition, patients who received acupuncture treatment were less likely to experience unanticipated deterioration events. This finding may relate to the fact that acupuncture treatment is not recommended for patients with septic conditions, acute hemorrhagic stroke, unstable seizures, or confusion. Generally, acupuncture therapy has been applied as a general treatment mode for common conditions and diseases such as acute and chronic noncancer pain, nausea, and vomiting, rather than complex and severe diseases [
]. However, additional study on the relationship between clinical deterioration risk and acupuncture treatment is suggested.
Patients who were hospitalized in the rehabilitation department were less likely to experience unanticipated clinical deterioration. It may reflect that patients who were admitted to CAM hospitals for rehabilitation have relatively stable conditions. Therefore, clinical departments will be considered in managing high-risk patients.
Overall, the findings of this study indicate that NEWS2 can be used to predict the risk of clinical deterioration in CAM practices. NEWS2 will be utilized as a common language for communicating the clinical deterioration risk between healthcare teams comprising of various healthcare professionals, which will facilitate timely interdisciplinary interventions. Therefore, hospital executives and nurse managers should support the use of EWSs in CAM practice. Setting a protocol including the escalation of care in CAM hospitals will be a fundamental step. Furthermore, a rapid response system between CAM hospitals and conventional medicine hospitals needs to be implemented.
The findings of this study also indicate that EWSs' predictive abilities need to be further improved. Based on the findings of this study, patient, care, and system factors can be used to detect patients at high risk of clinical deterioration. Using multiple predictors will contribute to reducing false alarms and increasing EWSs’ predictive ability.
However, this study has several limitations. First, this study was conducted only in two CAM hospitals. Thus, the generalizability of the findings is limited. However, we analyzed 5-year data, which also met the recommendation of including at least 100 event cases [
]. Second, most event cases were transfers to other conventional medicine hospitals. Thus, we could not analyze patient outcomes after transfer. Third, we collected data based on recordings using a retrospective study design. Since documentation could not be prioritized in busy clinical environments, EWSs' performance can be different in real-world practice. In addition, we did not include variables such as the degrees of vital signs’ derangement over time and their temporal characteristics, laboratory test results, and care team factors that may affect the performance of EWSs and patient outcomes. Lastly, we assigned risk stratification values to the cases with missing data of frailty risk through medical record reviews (n = 9). It can cause bias. However, additional analyses, after excluding such cases, demonstrated that the results were consistent. Thus, we suggest future studies with more cases of mortality and cardiac arrest, including various CAM hospitals. Furthermore, a prospective study on the impact of using EWSs on patient outcomes is recommended.
Conclusion
This study provides evidence on the performance of EWSs in CAM hospitals for predicting high-risk patients who especially require rapid transfer to acute-care conventional medicine hospitals due to unanticipated clinical deterioration. Using NEWS2 will help identify high-risk patients in CAM hospitals. At the same time, patient, care, and system factors should be considered to avoid preventable adverse events and improve patient safety. This study finding will contribute to expanding the applicability of EWSs to CAM hospitals.
Conflict of interest
The authors declared no conflict of interest.
Funding
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government Ministry of Science and ICT (MSIT) (grant number: NRF-2019R1F1A1060844).
Ethical approval
This research was approved by the Institutional Review Boards of the study hospitals (no. 2019-12-001 and 2019-12-005).
Data availability
The data presented in this study are available from the corresponding author upon reasonable request and with permission of the institutional review boards of the study hospitals.
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1F1A1060844).
References
World Health Organization
Global patient safety action plan 2021–2030: towards eliminating avoidable harm in health care.
The performance of the National Early Warning Score and National Early Warning Score 2 in hospitalised patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).
The use of early warning system scores in prehospital and emergency department settings to predict clinical deterioration: a systematic review and meta-analysis.
A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.
The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
Early warning score adjusted for age to predict the composite outcome of mortality, cardiac arrest or unplanned intensive care unit admission using observational vital-sign data: a multicentre development and validation.
Development and validation of a Hospital Frailty Risk Score focusing on older people in acute care settings using electronic hospital records: an observational study.
00018-X&id=gr1.jpg){kind=link}
00018-X&id=gr2.jpg){kind=link}