Heterogeneous treatment effects of stress ulcer prophylaxis among ICU patients at risk for gastrointestinal bleeding

Abstract

Background

While randomized clinical trials of stress ulcer prophylaxis (SUP) have generally shown no overall benefit, subgroup analyses suggest the benefit or harm of SUP in specific patients, indicating heterogeneity of treatment effects (HTE). Understanding HTE is crucial for tailoring SUP to individual treatment.

Methods

This cohort study included patients admitted to intensive care unit (ICU) with at least one risk factor for clinically important gastrointestinal bleeding (GIB). The primary exposure was the use of SUP within 48 h after ICU entry; the primary outcome was 28-day mortality. We employed conventional subgroup analysis, risk-based analysis, and effect-based analysis to explore the HTE of SUP.

Results

A total of 25,475 patients were included, of whom 6199 (24.3%) received SUP, with famotidine being the most commonly prescribed (53.7%). Baseline characteristics were well-balanced between treatment groups after weighting. SUP was not associated with the 28-day mortality in the overall population (median value for the posterior distribution of the odds ratio (OR), 1.03; 95% credible interval (CrI), 0.96–1.11). In conventional subgroups, the impact of SUP on 28-day mortality varied substantially between patients with an age of higher than or equal to 77 years in comparison with other age subgroups (posterior probability of difference in OR, 99.3%), between patients with and without chronic liver disease (posterior probability of difference in OR, 99.9%), between patients with and without coagulopathy (posterior probability of difference in OR, 92.1%), and between patients with and without malignant cancer (posterior probability of difference in OR, 100%). In risk-based analysis, patients at high risk of death exhibited the highest propensity for benefit from SUP (posterior probability of an OR > 1, 1.9%). In effect-based analysis, patients with malignant cancer and a higher Charlson comorbidity index identified at high probability of benefit.

Conclusions

Among ICU patients with at least one risk factor for clinically important GIB, those who are younger, have chronic liver disease, coagulopathy, or malignant cancer are more likely to benefit from SUP.

Background

Gastrointestinal stress ulceration is a well-recognized condition that can progress to clinically important gastrointestinal bleeding (GIB) and potentially contributes to critical illness or death in patients admitted to intensive care unit (ICU) [1]. The prevalence of GIB in critically ill adult patients in the ICU ranges from 0.6 to 2.8% [2]. Although international guidelines offer conflicting recommendations [3, 4], stress ulcer prophylaxis (SUP) with proton pump inhibitors (PPIs) or histamine-2 receptor blockers (H2RAs) is a primary strategy for preventing GIB in the ICU [5].

The provision of SUP in the ICU remains a topic of ongoing debate. While SUP is associated with a lower rate of GIB, it might increase the rate of adverse events that counterbalance their potential benefits. The SUP-ICU trial included patients with at least one risk factor for clinically important GIB and found that PPIs decrease both overt GI bleeding and GIB without affecting mortality and infectious complications [6]. Subsequent meta-analyses confirmed these findings [7, 8], The effect of a treatment on outcomes can vary among patients based on their individual characteristics, also known as heterogeneity of treatment effect (HTE). Subgroup analyses of the SUP-ICU trial revealed increased 90-day mortality with pantoprazole in patients with a Simplified Acute Physiology Score (SAPS) II higher than 53 points [6]. A further post hoc study indicated that 90-day mortality and infectious adverse events may be increased with the use of pantoprazole in patients with greater illness severity and in those with more risk factors for GIB [9]. These findings suggested the presence of HTE, whereas the HTE of SUP was not fully explored. Understanding the HTE of SUP is essential to maximize its efficacy while minimizing its adverse effects.

Various approaches have been proposed to estimate the HTE. Subgroup analysis is a conventional method but has significant limitations [10, 11], whereas risk-based analysis and effect-based analysis are newly developed machine-learning techniques that estimate HTE more robustly [12, 13]. In the present study, we employed these three methods to estimate the HTE of SUP in patients with at least one risk factor for clinically important GIB, aiming to identify which patients can benefit most from SUP.

Methods

Study design and participants

The study utilized the electronic health records from the Medical Information Mart for Intensive Care (MIMIC-IV V2.2) [14], a comprehensive database containing detailed, high-quality data on ICU patients admitted to Beth Israel Deaconess Medical Center, Boston, from 2008 to 2019. Authorization for data access was obtained. Given the de-identified nature of the data, informed consent was waived. Our study was conducted according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement (Additional file 1).

Patients with at least one risk factor for clinically important GIB were enrolled, including the use of vasopressor or inotropes, renal-replacement therapy, invasive mechanical ventilation (IMV), coagulopathy, use of anticoagulants, or any history of chronic liver disease (Additional file 2). Exclusion criteria were as follows: an ICU or hospital stay of 24 h or less, length of ICU stay more than 100 days, patients with peptic ulcer disease, patients with gastrointestinal bleeding during current ICU admission. Additionally, we excluded patients who received SUP prior to ICU admission. Only the first ICU stay was analyzed for patients admitted more than once.

Variable extraction and outcomes

Details on data collection are presented in the supplementary materials (Additional file 3: Table S1). The primary exposure was the use of SUP within 48 h after ICU entry, including PPIs and H2RAs, and administered consecutively twice. The primary outcome was 28-day mortality after ICU entry. Secondary outcomes were upper gastrointestinal bleeding, hospital-acquired or ventilator-associated pneumonia, Clostridium difficile infection, and length of ICU and hospital stays.

Overlap weighting

We constructed a propensity score using overlap weighting to balance observed baseline characteristics between treatment groups [15]. A propensity score for using SUP was estimated from a multivariable logistic regression model incorporating variables including age, sex, ethnicity, admission type, lactate, sequential organ failure assessment (SOFA) score, SAPS II, Charlson comorbidity index (CCI), use of vasopressors or inotropes, IMV, coagulopathy, use of anticoagulants, use of renal replacement therapy, chronic pulmonary disease, myocardial infarction, congestive heart failure, renal disease, chronic liver disease, and malignant cancer. The overlap propensity score weighting method was then employed, in which each patient’s weight is the probability of that patient being assigned to the opposite medication group. The standardized mean differences (SMDs) were calculated among groups to evaluate the effectiveness of the weighting, with an SMD less than 0.1 indicating a covariate balance. The weighted cohort was generated to explore the HTE of SUP.

Statistical analysis

Values were presented as mean (standard deviation) or median [interquartile range (IQR)] for continuous variables as appropriate and categorical variables as total number and percentage. Comparisons between groups were made using the X² test or Fisher’s exact test for categorical variables and Student’s t-test or Mann–Whitney U test for continuous variables as appropriate. HTE was evaluated using three strategies based on the weighted cohort.

Conventional subgroup analysis approach

We initially evaluated the HTE across subgroups defined by clinical variables, including age, sex, SAPS II, the number of risk factors, IMV, coagulopathy, and chronic liver disease, which were considered potential moderators of the treatment effect. Bayesian proportional odds ordinal logistical models with statistical interaction were applied to assess the impact of SUP on 28-day mortality in each subgroup, adjusting for age, sex, SAPS II, and CCI. The Bayesian posterior probability determined the treatment effect in each subgroup, indicating whether the treatment odds ratio (OR) was likely to be less than or greater than 1.0. An OR greater than 1.0 indicated treatment harm, and an OR less than 1.0 indicated treatment benefit. The magnitude of statistical evidence for differences in treatment effects across subgroups was quantified by calculating the posterior probability that the odds ratio (OR) was higher in one subgroup than others. Bayesian ordinal analysis was conducted using the “rmsb” package in R software (version 4.0.2).

Risk-based approach

We further assessed the HTE across varying levels of mortality risk using previously described methods [16, 17]. Initially, a logistic regression model was developed to predict 28-day mortality using candidate risk predictors (Additional file 4). The ultimate selection of variables was determined by identifying the model with the lowest Bayes information criterion among models with all possible combinations. The effectiveness of the risk model was evaluated by calculating the area under the receiver operating characteristic curve (AUROC). The derived model was used to calculate each patient’s risk score, determined by the linear combination of their covariate values and the final model coefficients (log ORs), with a constant added to guarantee the positivity of all scores. Patients were ranked based on these risk scores and divided into risk deciles. The impact of SUP on mortality in each risk decile was assessed using the described Bayesian logistical regression model.

Effect-based approach

We finally examined the HTE across levels of predicted treatment effect using a nonparametric causal forest method. The treatment effect in this approach was conceptualized as a conditional absolute rate difference (cARD), representing the disparity in weighted mortality averages between control and treatment groups among patients with similar values for potential effect modifiers. Positive cARD values indicate a predicted improvement in survival attributed to SUP, while negative values indicate an expected reduction in survival.

Before applying the causal forest method, patients were randomly allocated into training and validation cohorts, each comprising half of the patients. In the training cohort, a predictive model of the probability of 28-day mortality was constructed by gradient boosting machine modeling using the same set of candidate risk predictors used to construct the risk-based model, aimed to diminish computational complexity when identifying effect modifiers within a modest sample size. Subsequently, the causal forest model assessing the treatment effect on 28-day mortality was developed, incorporating each patient’s predicted probability of survival alongside all potential effect modifiers, the importance of each variable was determined by the number of times a potential effect modifier was chosen to be in the first splits of a tree in the causal forest model. The model estimated the cARD for each patient in the validation cohort, and we tested whether the observed absolute differences in the survival rate increased monotonically across deciles of the cARD predicted for each patient. The causal forest analysis was conducted using the “grf” package in R software (version 4.0.2).

Results

The study population comprised 25,475 patients with ≥ 1 risk factors for clinically significant GIB (Fig. 1). Of those, 57.8% were men, the median age was 66 years [IQR, 55–77], the median CCI was 5.0 [IQR: 3.0–7.0], and the median SOFA score was 6.0 [IQR: 3.0–8.0]. The predominant risk factor is the use of vasopressors or inotropes (51.3%), followed by coagulopathy (40.5%) and the use of anticoagulants (24.6%). In the overall population, 6199 (24.3%) patients received SUP within 48 h of ICU admission, and famotidine (53.7%) is the most commonly prescribed medication, the time from ICU admission to the first dose of drug was 9.8 h [IQR, 4.8–18.7], and to the second dose was 29.6 h [IQR, 19.7–42.3] (Additional file 3: Tables S2–S3). Upper gastrointestinal bleeding was observed in 301 (1.2%) patients. There were no patients lost to follow-up, and the 28-day all-cause mortality was 10.9% (Table 1).

Fig. 1

Study flowchart

Table 1 Clinical characteristics and outcomes of patients before and after overlap weighting

Before weighting, substantial disparities exist across a majority of the clinical characteristics between SUP and non-SUP groups. Patients in SUP group were younger, more likely to have higher CCI, SOFA scores and SAPS II, and more likely to receive IMV than those in the non-SUP group. The 28-day mortality was significantly higher in patients who received SUP than in those who did not. After weighting, the patient population was well-balanced across all clinical characteristics (all SMD < 0.1) (Table 1), and SUP was not associated with the 28-day mortality (median value for the posterior distribution of the OR, 1.03; 95% credible interval (CrI), 0.96–1.11).

Conventional subgroup analysis

The impact of SUP on 28-day mortality differed substantially between patients with an age of higher than or equal to 77 years in comparison with other age subgroups (posterior probability of difference in OR, 99.3%), between patients with and without chronic liver disease (median OR, 0.87 vs 1.07; posterior probability of difference in OR, 99.9%), between patients with and without coagulopathy (median OR, 0.95 vs 1.09; posterior probability of difference in OR, 92.1%), and between patients with and without malignant cancer (median OR 0.57 vs 1.20, posterior probability of difference in OR, 100%) (Fig. 2).

Fig. 2

Heterogeneity of treatment effect of SUP evaluated using conventional subgroup analysis. SUP, stress ulcer prophylaxis; OR, odds ratio; Crl, credible interval; SAPS II, Simplified Acute Physiology Score II; IMV, invasive mechanical ventilation. ^aPosterior probability of an OR greater than 1

Treatment effects on 28-day mortality did not demonstrate significant differences between females and males (median OR, 0.99 vs 1.03; posterior probability of difference in OR, 56.7%), between the subgroup with a SAPS II of less than or equal to 31 in comparison with subgroups with a SAPS II of 32 or higher (posterior probability of difference in OR, < 90% for all comparisons), between patients with one risk factor and those with two or more risk factors (posterior probability of difference in OR, < 90% for all comparisons), and between patients with and without IMV (median OR, 0.94 vs 1.09; posterior probability of difference in OR, 75.4%) (Fig. 2).

Risk-based approach

Table S4 (Additional file 3) delineates the estimated ORs and log ORs for each variable incorporated into the final model, which predicts the cumulative odds of 28-day mortality. The distribution of risk score is shown in Fig. S1 (Additional file 3). The calibration curve generated from the risk model exhibits robust predictive accuracy (Additional file 3: Fig. S2), with an AUROC of 0.808 (95% CI 0.801–0.814) (Additional file 3: Fig. S3). Patients were ranked by risk score and grouped by decile; increasing risk score was associated with progressively higher risk for 28-day death.

The effect of SUP on 28-day mortality for each risk score decile was displayed in Fig. 3. Despite the incrementally 28-day mortality observed from the first to the tenth decile, the efficacy of SUP exhibited no uniform variation until group 7. Patients in group 10 with the highest risk score had a posterior probability of an OR greater than 1 at 1.9%, while did not differ significantly in comparison to other groups (posterior probability of difference in OR, < 90% for all comparisons).

Fig. 3

Risk-based heterogeneity of treatment effect of SUP on 28-day mortality. Risk group 1 means the lowest risk of death, and group 10 means the highest risk of death. Clinical benefit was deemed substantially more probable than not (posterior probability of an OR > 1, 1.9%) in the highest risk group. SUP, stress ulcer prophylaxis; OR, odds ratio; Crl credible interval. ^aPosterior probability of an OR greater than 1

Effect-based approach

Table S5 (Additional file 3) presents the clinical characteristics between patients in the training and validation cohorts. The distribution of cARD in rates of 28-day survival obtained by validation cohort was illustrated in Fig. S4 (Additional file 3). Observed 28-day survival was monotonically increase with deciles of predicted benefit (P < 0.05) (Fig. 4). In the lowest 50% of the cohort (deciles 1–5), the observed treatment effect indicated potential harm from SUP. Conversely, in the highest 40% of the cohort (deciles 6–10), the observed treatment effect suggested a potential benefit from SUP. Notably, the effect of SUP on 28-day mortality differed between patients in the highest decile (group 10) in comparison with all others (observed cARD, 9.0%, 95% CI, 5.9 to 12.1%, post hoc P < 0.05 for statistical interaction).

Fig. 4

Effect-based heterogeneity of treatment effect of SUP on 28-day mortality. Positive cARD values indicate a predicted improvement in survival attributed to SUP and negative values indicate an expected reduction in survival. Patients in group 10 can benefit most from SUP. cARD, conditional absolute rate difference; SUP, stress ulcer prophylaxis; CI, confidence interval

In the model, the most important variables in determining the treatment effect of SUP were malignant cancer, CCI, and SOFA score (Additional file 3: Fig. S5). The comparison between a group consisting of the highest 10% of predicted cARD and a group consisting of all patients was presented in Table S6 (Additional file 3). Patients in the highest cARD decile group tended to have high CCI (SMD = 1.127) and were more likely to have malignant cancer (SMD = 2.171).

Discussion

Using three analytic approaches, this study demonstrated significant variability in the treatment effects of SUP among patients with at least one risk factor for clinically important GIB. Specifically, the treatment effect differed markedly across subgroups defined by risk factors or comorbidities, regardless of the baseline risk of mortality. Patients admitted to the ICU with malignant cancer benefitted the most from SUP, independent of the analytic approaches. The study verified the HTE of SUP and highlighted the necessity of individualized SUP treatment, even among patients with GIB risk factors.

The effectiveness of SUP depends on various factors, including the disease severity and comorbid illness. The SUP-ICU and REVISE trials are two large-scale clinical trials designed to evaluate the impact of pantoprazole on critically ill patients [6, 18]. Although the trials found no significant impact on overall mortality, subgroup analyses suggested that pantoprazole may increase or decrease mortality in specific patient populations. The meta-analysis also identified varying effects of SUP based on different risks of GIB [8]. All these findings underscore the necessity of exploring the HTE associated with using SUP.

The HTE results from this study differ from the subgroup and post hoc analyses of the SUP-ICU trial [6, 9], which detected no HTE of pantoprazole on 90-day mortality among patients stratified by history of liver disease, coagulopathy, and mechanical ventilation at randomization, but observed potential HTE according to illness severity with higher risk in the most severely ill. There are some explanations for the differences. First, compared to patients in this study, patients in the SUP-ICU trial had higher SOFA scores and higher mortality rates, but a lower proportion of patients exhibited risk factors for GIB, which could potentially attenuate the beneficial effect of SUP while highlighting its adverse effects. Second, the SUP-ICU trial focused solely on pantoprazole, while our study included both PPIs and H2RAs. PPIs have been demonstrated to exert a range of immunosuppressive effects that could potentially increase the risk of death from common infection-related complications in ICU patients [19]. Previous studies also suggested that PPIs were associated with greater risk of nosocomial pneumonia and Clostridioides difficile infection compared to H2RAs [20,21,22], Our results identified targeted patients who can benefit from SUP, while due to the nature of the cohort study, these results warrant further investigation in prospective studies.

Malignant cancer was the most significant influential contributor to HTE in both the subgroup and effect-based analyses. None of the previous studies has elucidated the treatment effect of SUP concentrated on ICU patients with cancer, while critically ill cancer patients theoretically represent a high-risk population for the development of GIB, since the common presence of an underlying coagulopathy related to disease or treatment, and potentially in addition to other established risk factors [23]. Additionally, the infection-related complications of SUP also represent a significant concern for cancer patients [24]. The beneficial and adverse effects of SUP in critically ill cancer patients need more research.

This study has several strengths. First, contrary to randomized controlled trials using stringent inclusion and exclusion criteria, we utilized an extensive database to elucidate the actual effect of SUP in real-world clinical practice and confirm its beneficial effects in specific patients. Second, we employed three approaches to comprehensively identify the HTE of SUP, resulting in more informative and robust findings. Subgroups defined by a single characteristic are often more alike than unlike with respect to the effect of the therapy, whereas risk-based and effect-based approaches are more efficient in the search across all possible combinations of potential predictor variables and interactions to predict variability in treatment response [25]. Third, the Bayesian analyses we used provided full posterior probability distributions, superior to dichotomized tests using arbitrary thresholds for statistical significance. Conventional subgroup analyses, which increase the risk of type II errors due to lower power, can benefit from probability distributions as they offer more valuable insights than traditional null-hypothesis significance testing.

Several limitations in the present study should also be considered. First, our study was a retrospective cohort study based on electronic healthcare records; although we applied overlap weighting to adjust for the confounders that could impact the association between SUP and mortality, multiple measured and unmeasured confounders persist. Additionally, the current study lacks a uniform regimen for SUP therapy; we included patients who initiated the SUP within 48 h after ICU entry and administered consecutively twice to mitigate this limitation. Second, the 28-day mortality follow-up began after ICU admission, which could introduce an immortal time bias. We performed Landmark analyses to address this bias, and the results were consistent with the overall population (Additional file 3: Table S7), suggesting that the potential impact of immortal time bias on the study outcomes is minimal. Third, although we defined risk factors of clinically important GIB based on a published study [6], we did not account for all reported risk factors, and the impact of included risk factors on GIB remains questioned. We also did not consider the weight of different risk factors, while the risk-based and effect-based approaches can address this issue. Fourth, we included PPIs and H2RAs in the present study without distinguishing their individual effects. This approach aligns more closely with real-world clinical practice, as some patients may use both medications sequentially. Besides, the PEPTIC trial concluded that the hospital mortality rates for those receiving PPIs versus H2RAs showed no significant difference among ICU patients requiring mechanical ventilation [26]. Finally, the effect-based analysis identified malignant cancer as the most significant influential contributor, while the mechanism remains uncertain, and this finding might not always be applicable to other patients. Further studies are required to examine the generalizability of our findings.

Conclusions

Among patients with at least one risk factor for clinically important GIB, those who are younger, have chronic liver disease, coagulopathy, or malignant cancer are more likely to benefit from SUP. While the treatment effect did not vary significantly according to baseline risk of mortality, malignant cancer appears to be the most pivotal factor influencing treatment effect. These findings are hypothesis-generating and warrant further investigation.