Home
JournalsCollections
For Authors For Reviewers For Editorial Board Members
Article Processing Charges Open Access
Ethics Advertising Policy
Editorial Policy Resource Center
Company Information Contact Us
OPEN ACCESS

Prognostic Nomogram for Patients with Hepatitis E Virus-related Acute Liver Failure: A Multicenter Study in China

  • Jian Wu1,2,# ,
  • Cuifen Shi3,#,
  • Xinyu Sheng1 ,
  • Yanping Xu1 ,
  • Jinrong Zhang4,
  • Xinguo Zhao5,
  • Jiong Yu1 ,
  • Xinhui Shi6,
  • Gongqi Li7,
  • Hongcui Cao1,8,*  and
  • Lanjuan Li1 
Journal of Clinical and Translational Hepatology   2021;9(6):828-837

doi: 10.14218/JCTH.2020.00117

Received:

Revised:

Accepted:

Published online:

 Author information

Citation: Wu J, Shi C, Sheng X, Xu Y, Zhang J, Zhao X, et al. Prognostic Nomogram for Patients with Hepatitis E Virus-related Acute Liver Failure: A Multicenter Study in China. J Clin Transl Hepatol. 2021;9(6):828-837. doi: 10.14218/JCTH.2020.00117.

Abstract

Background and Aims

Timely and effective assessment scoring systems for predicting the mortality of patients with hepatitis E virus-related acute liver failure (HEV-ALF) are urgently needed. The present study aimed to establish an effective nomogram for predicting the mortality of HEV-ALF patients.

Methods

The nomogram was based on a cross-sectional set of 404 HEV-ALF patients who were identified and enrolled from a cohort of 650 patients with liver failure. To compare the performance with that of the model for end-stage liver disease (MELD) scoring and CLIF-Consortium-acute-on-chronic liver failure score (CLIF-C-ACLFs) models, we assessed the predictive accuracy of the nomogram using the concordance index (C-index), and its discriminative ability using time-dependent receiver operating characteristics (td-ROC) analysis, respectively.

Results

Multivariate logistic regression analysis of the development set carried out to predict mortality revealed that γ-glutamyl transpeptidase, albumin, total bilirubin, urea nitrogen, creatinine, international normalized ratio, and neutrophil-to-lymphocyte ratio were independent factors, all of which were incorporated into the new nomogram to predict the mortality of HEV-ALF patients. The area under the curve of this nomogram for mortality prediction was 0.671 (95% confidence interval: 0.602–0.740), which was higher than that of the MELD and CLIF-C-ACLFs models. Moreover, the td-ROC and decision curves analysis showed that both discriminative ability and threshold probabilities of the nomogram were superior to those of the MELD and CLIF-C-ACLFs models. A similar trend was observed in the validation set.

Conclusions

The novel nomogram is an accurate and efficient mortality prediction method for HEV-ALF patients.

Keywords

Hepatitis E, Acute liver failure, Nomogram, Mortality prediction, Scoring model

Introduction

Hepatitis E virus (HEV) is endemic in many developing countries because of poor sanitation. The virus is predominantly transmitted through fecal and oral routes, which is also a main cause of acute viral hepatitis.1,2 About 20.1 million HEV infection-related hepatitis cases occur worldwide, resulting in 70,000 deaths and 3,000 stillbirths in the past.3 Although hepatitis E usually causes asymptomatic and self-limiting diseases with low mortality, fulminant hepatitis that leads to acute liver failure (ALF) or acute-on-chronic liver failure (ACLF) are possible. Of all acute HEV cases, only a small fraction (0.5–4%) progress to ALF. The rate of progression to ALF may be as high as 10–22% in pregnant women.4 Notably, the fact that HEV plays an important role in the development of ALF has also been frequently reported in Europe.5,6 All of these could lead to high mortality rates, ranging from 0–67%. Hence, diagnosing HEV-related ALF (HEV-ALF) patients in a timely manner is extremely important.

To date, a few scoring systems have been established for the diagnosis and prediction of prognosis in patients with different kinds of liver diseases. The model for end-stage liver disease (MELD) score,7 the integrated MELD (also known as iMELD) score,8 Child-Turcotte-Pugh score,9 and CLIF-Consortium-ACLF score (CLIF-C-ACLFs)10 have been reported for predicting prognosis in patients with liver cirrhosis. The MELD11 and the CLIF-C-ACLFs model12 have been used to assess the degree of liver damage and the prognosis of patients. Although various models have been used to predict mortality and transplant-free survival in ALF patients of both acetaminophen-induced and virus-related, a scoring model for predicting the mortality of HEV-ALF patients has not yet been reported, to the best of our knowledge.

A nomogram is a graphical representation, which can be used to diagnose or predict disease occurrence or progression with multiple indicators.13 Moreover, nomograms can provide a user-friendly interface, which has a demonstrated advantage over the traditional staging systems used to predict patient outcomes for many critical diseases.14,15 As a result, nomogram has been proposed as an alternative method, or even as the new standard. Hence, this study aimed to develop a nomogram for predicting the mortality of HEV-ALF patients, and to compare the performance of this nomogram with that of the CLIF-C-ACLFs and MELD models.

Methods

Patients

A total of 404 eligible HEV-ALF patients were recruited from among 650 patients with liver failure from five hospitals in different regions of China. The patient enrollment flow chart is shown in Supplementary Figure 1. All diagnosed HEV-ALF patients, who were referred to The First Affiliated Hospital (Zhejiang University School of Medicine), The Fifth People’s Hospital of Wuxi, The First People’s Hospital of Yancheng City, The People’s Hospital of Dafeng City, and The Linyi Traditional Hospital between 1 January 2010 and 30 May 2019, were retrospectively and consecutively analyzed as the development set (n=249) and the validation set (n=155) of the study.

The selection criteria for HEV-ALF patients have been based on the King’s College criteria.16 Diagnosis of HEV infection made by testing for anti-HEV immunoglobulin (Ig)M and IgG using enzyme-linked immunosorbent assays. A hepatitis E case in this study was defined by positive serum anti-HEV IgM, and/or a greater than 2-fold increase in the anti-HEV IgG titer, and/or detectable HEV RNA with clinical presentation of acute hepatitis, which showed elevated liver enzymes and/or jaundice and/or non-specific symptoms such as fatigue, itching and nausea. The inclusion and exclusion criteria for the enrolled HEV-ALF patients are both described in the supplemental material. The test methods for anti-HEV IgM, IgG antibodies and HEV RNA quantification are provided in the supplemental material.

The criteria for diagnosing ALF was as follows: (1) evidence of abnormal liver synthetic function (prothrombin activity ≤40% or international normalized ratio [INR] ≥1.5), jaundice and hepatic atrophy in 2 weeks in patients; (2) presence of stage 2 or 3 encephalopathy complicating end-stage disease manifestations; and (3) no chronic liver disease.

The exclusion criteria for the enrolled HEV-ALF patients was as follows: (1) co-infection with hepatitis B virus or hepatitis C virus, or alcoholic and non-alcoholic fatty liver disease (NAFLD); (2) drug-induced liver disease; (3) autoimmune liver disease; (4) liver cancer; (5) co-infection with cytomegalovirus or Epstein-Barr virus; (6) metabolic liver diseases; (7) previous kidney diseases; (8) accepted liver transplantation; (9) Wilson’s disease; (10) Budd-Chiari syndrome; (11) treatment with an immunosuppressive; (12) incomplete data; or (13) loss to follow up.

Patients were followed up every 7 days and the survival data were collected through medical records or by direct contact with the patients or their families, with death or LT as a composite endpoint. During the follow-up, two of the total four hundred and four HEV-ALF patients were treated with immunosuppressives. One was to address sarcoidosis (prednisone 20 mg/day), and the other giant cell arteritis (tocilizumab 8 mg/kg body weight per month). The present study was performed in accordance with the Helsinki Declaration and was approved by the Ethics Committee of the First Affiliated Hospital, Zhejiang University (reference number: 2011013). Informed consent was obtained from all participants or their families.

Data collection and scoring model calculation

We collected all enrolled patients’ clinical, demographic information and laboratory variables, including age, sex, coagulation parameters, hepatic encephalopathy (HE), arterial blood ammonia, laboratory parameters, length of hospitalization and intensive care unit stay, and prognosis. The diagnosis of HE met the West Haven criteria.17 The MELD7 and CLIF-C-ACLFs10 scoring model calculations are described in the supplemental material. Patients with HE of grade I and II were defined as mild, while those with grade III and above were defined as severe.

Scoring model calculation

The MELD score (range: 6–40) was calculated as follows:

9.6 * loge [creatinine (mg/dL)] + 3.8 * loge [bilirubin (mg/dL)] + 11.2 * loge (INR) + 6.43 * (etiology: 0 if cholestatic or alcoholic, 1 otherwise).

The CLIF-C-ACLFs was derived from a modification of the CLIF-sequential organ failure assessment (SOFA) scale and was calculated as follows:

10 * [0.33 * CLIF-SOFAs + 0.04 * age + 0.63 * loge (white-cell count)-2].

In general, the CLIF-SOFA score (range: 0-24) comprises the same six organ systems as the SOFA and is used to evaluate organ failure in HEV-liver failure patients. As such, in our study, the SOFA score (range: 0–24) was calculated as the sum of scores for six organ systems: respiratory, cardiovascular, renal, neurological systems, liver, and coagulation.

HEV-specific antibody detection

All serum samples were tested for the presence of anti-HEV IgM and IgG antibodies using commercially available HEV enzyme-linked immunosorbent assay kit (Wantai, Beijing, China) according to manufacturer’s instructions. Samples with optical density >1.1 were considered positive. Samples with optical density ≤1.1 were considered negative.

HEV RNA detection

HEV RNA was tested by means of internally controlled quantitative real-time reverse transcription PCR as described.3 Total RNA was extracted from serum using a virus nucleic acid purification kit (Aikang, Hangzhou, China) according to the manufacturer’s instructions. A 348-nucleotide fragment of the HEV open reading frame 2 was amplified using a nested PCR technique and sequenced to identify the genotype. The viral infection of each sample was estimated using qualitative PCR according to the CT value using a diagnostic kit for HEV RNA (Aikang) according to the manufacturer’s instruction.

Statistical analysis

Statistical analyses were performed with SPSS (v18.0; IBM Corp., Armonk, NY, USA) and R software (v3.1.2; Institute for Statistics and Mathematics, Vienna, VIC, Austria). Categorical data are showed as numbers (percentages) and were compared using the chi-square test. Univariate and multivariate logistic regression analyses were performed to identify independent prognostic factors for HEV-ALF patients’ 7-day, 28-day and 90-day mortality. Orthogonal partial least squares-discriminant analysis (OPLS-DA) was used to evaluate and rank the ability of the parameters to predict the mortality of HEV-ALF patients using SIMCA software (Sartorius, Gottingen, Germany). The performance of the nomogram was evaluated by calibration and discrimination, and assessed by comparing nomogram-predicted vs. observed Kaplan-Meier estimates of survival probability.18,19 The rcorrp.cens package in Hmisc in R software was performed to compare the concordance index (C-index). The time receiver operating characteristic package in R software20 was tested to compare the time-dependent area under the receiver operating characteristic curve (td-AUC). The decision curve analysis (DCA)21 was also used to assess the net benefits of the nomogram.

Results

Patient characteristics and follow-up

The majority of the HEV-ALF patients were males (71.5%), with 115 (28.5%) patients being female. The mean patient age was 57.25 years (range: 43.33–69.17 years). Nine of the four hundred and four total eligible HEV-ALF patients were pregnant women. In addition to the liver, the most frequent failure organ was the kidney (14.4%), followed by cerebral (7.9%), coagulation (5.9%) and lung failure (4.0%). Among all the HEV-ALF patients, 83.9% exhibited just failure in liver, followed by 9.2% with failure in two organs, and 6.9% with failure in three or more. The mean follow-up times were 5.7 months (range: 3.2 to 9.6 months) and 5.5 months (range: 3.1 to 9.2 months) for the development and validation sets, respectively. The 7-day, 28-day and 90-day overall survival rates of the HEV-ALF patients were 201 (49.8%), 157 (38.9%) and 155 (38.4%), respectively. The characteristics of all recruited patients are summarized in Table 1, and show that there was no significant difference among all variables between the development and validation sets.

Table 1

Characteristics of the enrolled patients

VariableTotal (n=404)Development set (n=249)Validation set (n=155)pUnivariate analysis
Multivariate analysis
OR (95% CI)pOR (95% CI)p
Clinical characteristics
Age, years57.25±12.9256.11±12.0757.54±12.880.1771.02 (0.97–1.12)0.892
Sex, F/M115/28973/17642/1130.6311.39 (0.42–5.06)0.601
BMI23.74±2.9823.55±2.3723.91±2.990.5991.07 (0.92–1.08)0.794
PH7.42 (7.32–7.48)7.41 (7.33–7.47)7.42 (7.31–7.49)0.5542.19 (0.94–4.99)0.877
MAP, mm Hg88.56±12.3990.59±13.3985.86±10.770.5921.07 (0.91–1.04)0.293
HE mild3019110.8422.84 (1.09–7.41)0.0333.35 (1.04–10.78)0.042
HE severe6840280.6012.28 (1.14–4.55)0.0191.77 (0.71–4.40)0.221
Muscle and/or joint pain (mild/serve)81/3448/2233/120.5851.09 (0.79–1.09)0.322
Abdominal pain and/or vomiting(mild/serve)106/6462/3944/250.7531.12(0.89–1.51)0.554
Ascites, mild/serve94/5158/3136/200.9141.05 (0.81–1.17)0.488
Bacterial infection62(15.3%)39(15.7%)23(14.8%)0.8231.51(1.12–2.19)0.0211.06 (0.41–2.74)0.906
Laboratory parameters
WBC, 109/L6.29 (5.46–8.91)6.21 (5.44–8.90)6.44 (5.48–8.95)0.6872.25 (0.97–5.65)0.897
RBC, 1012/L4.10±0.674.12±0.694.09±0.610.6551.19 (0.87–1.38)0.503
ALT, U/L401.00 (133.00–1,134.00)418.00 (145.00–1,240.00)396.00 (124.00–1,042.00)0.5991.00 (1.00–1.00)0.0431.00 (1.00–1.00)0.512
AST, U/L219.50 (84.50–617.00)223.00 (92.00–636.00)217.00 (82.00–610.00)0.6731.00 (1.00–1.00)0.477
GGT, U/L98.00 (56.00–176.00)99.00 (57.00–177.00)97.00 (55.00–172.00)0.9880.99 (0.99–1.00)0.0031.00 (0.99–1.00)0.029
TP, g/L)57.21±8.8157.87±8.5156.63±9.120.1640.99 (0.95–1.05)0.331
ALB, g/L31.72±5.7732.12±5.7131.42±5.660.2270.95 (0.91–1.00)0.0390.93 (0.88–0.99)0.019
TBIL, mol/L299.54±165.81297.76±156.64305.12±176.410.6601.00 (1.00–1.00)0.0161.00 (1.00–1.01)0.030
DBIL, mol/L217.60±116.82213.87±110.12223.87±124.320.3961.00 (1.00–1.01)0.035
UREA, mmol/L4.58 (3.62–6.79)4.49 (3.51–6.61)4.67 (3.81–7.30)0.5512.48 (1.33–4.64)0.0041.08 (1.01–1.15)0.017
CR, mol/L76.91 (66.00–94.00)77.20 (66.50–95.00)76.50 (64.00–94.50)0.9561.00 (1.00–1.01)0.0161.01 (1.00–1.01)0.030
PT, s17.45 (15.50–23.55)17.35 (14.50–24.20)17.65 (14.50–23.90)0.9121.05 (1.02–1.07)<0.001
INR1.50 (1.30–2.12)1.50 (1.32–2.15)1.51 (1.30–2.07)0.8817.02 (2.59–19.01)<0.0017.72 (2.35–25.16)0.001
Ammonia, µmol/L150.25 (87.56–185.55)143.50 (78.22–178.25)152.55 (80.20–182.50)0.6741.01 (1.00–1.02)0.0081.01 (1.00–1.02)0.186
CRP, mg/L9.92 (7.15–14.87)9.77 (7.04–13.98)10.21 (7.19–14.97)0.6970.98 (0.95–1.09)0.912
TG, mmol/L1.02 (0.82–1.54)1.04 (0.87–1.55)0.99 (0.80–1.32)0.8120.42 (0.25–0.68)<0.0010.59 (0.21–1.65)0.310
TCH, mmol/L2.26±0.782.30±0.672.21±0.790.2180.71 (0.52–0.90)0.0050.80 (0.41–1.54)0.496
GLU, mmol/L3.67 (2.96–5.98)3.64 (2.99–5.98)3.78 (2.97–5.46)0.5120.89 (0.82–0.97)0.0080.92 (0.84–1.01)0.087
Potassium, mmol/L4.55±0.734.59±0.774.49±0.690.1841.31 (0.96–1.72)0.156
Sodium, mmol/L138.98±65.09139.76±65.18137.12±64.810.2341.09 (0.92–1.21)0.542
Total T3, nmol/L103.97 (57.07–136.79)102.50 (58.25–132.22)105.50 (60.05–139.50)0.6621.01 (1.00–1.01)0.0041.00 (1.00–1.01)0.226
Total T4, nmol/L0.94 (0.69–1.41)0.92 (0.75–1.32)0.96 (0.74–1.45)0.8920.98 (0.71–1.31)0.552
TSH, mIU/L1.73 (1.05–3.12)1.67 (0.97–2.41)1.81 (1.05–3.09)0.4460.92 (0.79–1.32)0.152
RDW14.88 (13.30–17.90)14.85 (13.20–16.00)14.95 (13.35–18.25)0.8751.07 (0.98–1.17)0.132
RLR0.77 (0.59–1.42)0.77 (0.59–1.29)0.78 (0.59–1.41)0.5962.46 (1.21–4.57)0.0121.34 (0.71–2.52)0.367
AFP, ng/mL38.37 (6.62–119.26)37.55 (6.00–112.00)39.90 (6.40–125.20)0.9050.99 (0.79–1.22)0.812
CHE, U/L2,693.24 (2,412.50–3,312.09)2,695.80 (2,363.60–3,155.12)2,681.20 (2,450.30–3,486.80)0.4721.00 (1.00–1.00)<0.0011.00 (1.00–1.00)0.228
FER, ng/mL2,912.54 (1,395.43–4,957.72)2,907.79 (1,391.20–4,922.12)2,932.60 (1,399.20–4,997.54)0.9761.00 (1.00–1.00)0.497
Other organ failure except for liver and cerebral
Kidney, n (%)58 (14.4%)35 (14.1%)23 (14.8%)0.8274.08 (2.79–7.93)<0.0011.81 (0.62–5.30)0.279
Coagulation, n (%)24 (5.9%)15 (6.0%)9 (5.8%)0.9285.69 (4.02–7.97)<0.0012.39 (0.27–21.22)0.433
Lung, n (%)16 (4.0%)10 (4.0%)6 (3.9%)0.9421.60 (1.09–2.29)<0.0013.18 (0.18–55.80)0.428
2 organs failure (%)37 (9.2%)23 (9.2%)14 (9.0%)0.9453.23 (2.79–4.29)<0.001
≥3 organs failure (%)28 (6.9%)17 (6.8%)11 (7.1%)0.9173.48 (1.24–9.77)0.018

Prognostic factors for HEV-ALF patients’ 7-day, 28-day and 90-day mortality

A univariate Cox analysis was firstly performed to observe the influences of clinical and laboratory parameters on HEV-ALF patients’ 7-day, 28-day and 90-day mortality, which indicated that HE, bacterial infection, alanine aminotransferase (ALT), γ-glutamyl transpeptidase (GGT), albumin (ALB), urea nitrogen (UREA), creatinine (CR), total bilirubin (TBIL), direct bilirubin (DBIL), the INR, prothrombin time (PT), cholinesterase (CHE), triglyceride (TG), total cholesterol, glucose, total triiodothyronine), neutrophil-to-lymphocyte ratio (NLR), RDW to lymphocyte ratio (RLR), platelet (PLT) count, and organ failure were all prognostic factors for HEV-ALF patients’ survival. Subsequently, multivariable analyses continued to demonstrate that GGT, ALB, TBIL, UREA, CR, INR, and NLR levels were independent risk factors for HEV-ALF patients’ survival (Table 1).

To further evaluate and rank the ability of the parameters to predict the mortality of HEV-ALF patients, OPLS-DA was next used. Nonsurvivors could be unambiguously distinguished from survivors using OPLS-DA (Fig. 1A, B). The top seven predictors were ln (UREA), ln (NLR), ln (GGT), ln (TBIL), ln (INR), ln (ALB), and ln (CR) (Fig. 1C,D). UREA, NLR, GGT, TBIL, INR, ALB, and CR were finally identified as the seven best prognostic indicators, since they influenced the mortality of HEV-ALF patients independent from other parameters (identified by Cox regression) and were the top seven indicators with highest predictive capability (identified by OPLS-DA).

OPLS-DA was used to evaluate and rank the ability of the parameters to predict the mortality of HEV-ALF patients.
Fig. 1  OPLS-DA was used to evaluate and rank the ability of the parameters to predict the mortality of HEV-ALF patients.

(A) ROC of OPLS-DA. (B) In the three-dimensional scatter plot of all samples in the OPLS-DA model, the predictive component was used to distinguish survivors and nonsurvivors. (C) Loading plot showing the relation of each parameter to the predictive component (x) and the first orthogonal component (y); parameters that deviated from zero on the x-axis were considered potentially predictive. (D) The higher predictive VIP (VIP pred) value.

Prognostic nomogram for HEV-ALF patients

A prognostic nomogram was created to predict HEV-ALF patients’ 7-day, 28-day and 90-day survival using the significantly independent risk factors for HEV-ALF patients’ survival (Fig. 2). The prognostic nomogram allows the user to predict the mortality of HEV-ALF patients, corresponding to a patient’s particular combination of covariates. For example, we can locate the patient’s GGT level and draw a line straight upward to the ‘points’ axis to determine the score associated with that GGT level. The same process was applied for ALB, TBIL, UREA, CR, INR, and NLR levels, and then we summed the scores achieved for each covariate, and located this sum on the ‘total points’ axis. Then, we drew a line straight down to determine the probability of mortality at each time point.

The nomogram for HEV-ALF patients’ 7-day, 28-day and 90-day mortality, including UREA, NLR, GGT, TBIL, INR, ALB, and CR levels.
Fig. 2  The nomogram for HEV-ALF patients’ 7-day, 28-day and 90-day mortality, including UREA, NLR, GGT, TBIL, INR, ALB, and CR levels.

The nomogram allows the user to obtain a probability of 7-day, 28-day and 90-day mortality corresponding to a patient’s particular combination of covariates. To use the nomogram, locate the patient’s value and draw a line straight upward to determine the score received for the variable. The sum of these scores is obtained for each covariate, which is then located on the ‘Total Points’ axis. A line is drawn downward to determine the likelihood of 7-day, 28-day and 90-day mortality on the survival axis.

Comparison of predictive accuracy for HEV-ALF patients’ 7-day, 28-day and 90-day mortality between the nomogram, MELD score, and CLIF-C-ACLFs in the development set

Calibration tests were used to compare the predictive accuracy for the mortality of HEV-ALF patients between the nomogram, MELD score, and CLIF-C-ACLFs. The rcorrp.cens package in Hmisc in R software was used to compare the C-index between the nomogram, MELD and CLIF-C-ACLFs scores. The C-index for predicting HEV-ALF patients’ survival using the nomogram was 0.671 (95% CI: 0.602–0.740), which was statistically significantly greater than that of the MELD score at 0.557 (95% CI: 0.489–0.624) and the CLIF-C-ACLFs at 0.540 (95% CI: 0.467–0.612) (all p<0.05). The calibration curve had an optimal agreement between the prognostic nomogram and the actual observation (Fig. 3A–C; Supplementary Table 1).

Calibration curves of the nomogram, MELD score, and CLIF-C-ACLF<sub>s</sub> for predicting HEV-ALF patients’ 7-day, 28-day and 90-day mortality in the development and validation sets.
Fig. 3  Calibration curves of the nomogram, MELD score, and CLIF-C-ACLFs for predicting HEV-ALF patients’ 7-day, 28-day and 90-day mortality in the development and validation sets.

The average predicted probability (predicted overall survival; x-axis) was plotted against the Kaplan-Meier estimate(observed overall survival; y-axis). 95% CIs of the Kaplan-Meier estimates are indicated with vertical lines. The dashed line indicates the reference line, where an ideal would lie.

To estimate the prognostic efficiency of the nomogram, we compared the td-AUC between the nomogram, MELD and CLIF-C-ACLFs scores. Figure 4A–C shows the time-dependent receiver operating characteristics (td-ROC) curves of the nomogram, MELD score, and CLIF-C-ACLFs for predicting HEV-ALF patients’ mortality. The td-AUC for predicting 7-day mortality using the nomogram was 0.921 (0.872–0.970), and was statistically significantly greater than that that obtained using the MELD score (0.474 [0.363–0.586]), and the CLIF-C-ACLFs (0.489 [0.376–0.603]) (both p<0.05). The td-AUC within 28-day using the nomogram was 0.809 (0.710–0.907), and was statistically significantly greater than that using the MELD score, which was 0.683 (0.559–0.807), and the CLIF-C-ACLFs, which was 0.632 (0.498–0.766) (both p<0.05). A similar trend was seen with 90-day predictions. Comparisons of the td-AUC of all models for predicting HEV-ALF patients’ mortality are shown in Supplementary Table 2.

Comparisons of the td-ROC between the nomogram, MELD score, and CLIF-C-ACLFs in the development and validation sets.
Fig. 4  Comparisons of the td-ROC between the nomogram, MELD score, and CLIF-C-ACLFs in the development and validation sets.

Moreover, DCA was used to further assess the net benefits of nomogram, MELD score, and CLIF-C-ACLFs assisted decisions at different threshold probabilities. Supplementary Figure 2A–C shows that the nomogram gave a better performance than the MELD score and CLIF-C-ACLFs over the entire range of threshold probabilities.

Validation of the predictive accuracy of the nomogram in the validation set

The clinical characteristics and laboratory parameters of the validation set are shown in Table 1. A good agreement was shown using the nomogram and the calibration curve between the prediction and actual observation of the probability of HEV-ALF patients’ 7-day, 28-day and 90-day survival (Fig. 3D–F). The C-index for the established nomogram was 0.671 (95% CI: 0.608–0.735), which was significantly greater than that of the MELD score, 0.578 (95% CI: 0.504–0.651), and the CLIF-C-ACLFs, 0.604 (95% CI: 0.530–0.675) (Supplementary Table 2). Notably, the performance of the established nomogram was also superior to that of MELD score and CLIF-C-ACLFs, which was confirmed by td-AUC (Fig. 4D–F; Supplementary Table 2) and DCA (Supplementary Fig. 2D–F).

Performance of the nomogram in stratifying risk among HEV-ALF patients

We determined the cut-off value by grouping the patients in the development set into two groups, on average after sorting according to the total score (low risk: 0–200, and high risk: ≥201); each group showed a different mortality (p<0.0001; Supplementary Fig. 3A). Similar results were obtained in the validation set. The nomogram performed well, allowing a remarkable distinction between the Kaplan-Meier curves for survival outcomes when stratifying into two risk subgroups (p<0.0001; Supplementary Fig. 3B).

Discussion

In the current study, a multicenter and multisample design was used with HEV-ALF patients. A new nomogram model was established and compared with traditional liver disease models to prognosticate the mortality of HEV-ALF patients. The nomogram integrated UREA, NLR, GGT, TBIL, INR, ALB, and CR levels, which are all significant independent risk factors for HEV-ALF patient survival. Notably, the nomogram had better predictive accuracy than the current conventional prognostic prediction scoring systems for liver failure.

The nomogram generated from the development set had a C-index that was superior to that of MELD score and the CLIF-C-ACLFs models. The calibration curves for the probability of 7-day, 28-day and 90-day overall survival showed optimal agreement between the nomogram prediction and actual observation values. Moreover, the td-ROC and DCA also showed that the nomogram was superior to the MELD and CLIF-C-ACLFs models. In addition, stratification into two risk subgroups (low-risk and high-risk) allowed remarkable distinction between Kaplan-Meier curves for survival outcomes. Similar results were also confirmed in the validation set.

Both multivariate logistic regression and OPLS-DA revealed that UREA, NLR, GGT, TBIL, INR, ALB, and CR levels are all independent risk factors for HEV-ALF patients’ survival. Both UREA and CR are important indicators for evaluating renal function. Consistent with previous studies,22,23 HEV infection and the associated renal injury is likely to be a causal factor. Cases of membranoproliferative glomerulonephritis with and without cryoglobulinemia, and membranous glomerulonephritis in HEV patients have been reported.24–27 A case of renal impairment during acute HEV infection in a solid organ transplant recipient has also been reported.28

INR is an important index to evaluate the coagulation function of patients. HEV infection is associated with certain hematological diseases. Severe thrombocytopenia has been reported in patients with acute HEV infection.29 All these symptoms are further aggravated with the development of HEV, especially for HEV-ALF patients. NLR, which was combined with neutrophils and lymphocytes, two inflammation indicators, has been reported to predict the prognosis of patients with stable cirrhosis,30 NAFLD31 and hepatitis B virus-related decompensated cirrhosis.32 Several other extrahepatic disorders, such as myocarditis,33 thyroiditis34 and myasthenia gravis,35 have been described with HEV infection.

Jiang et al.36 revealed that hypoalbuminemia was associated with an increased risk of ALF in patients with acute hepatitis A and B. In addition, Manka et al.37 reported that ALB levels were inversely correlated with the MELD score, INR, and bilirubin. Our study also confirmed ALB was an independent risk factor for HEV-ALF patient survival.

Compared with the majority of ALF-cohorts in the worldwide literature,38,39 the mean age of our cohort was 57.25±12.92 years, being significantly older. We consider that this is related to the high incidence of hepatitis E failure in the elderly, the mechanism of which remains to be further studied. The 7-day, 28-day and 90-day overall survival rates of the HEV-ALF patients were significantly better than patients of other etiologies. All of these are consistent with the report by Shalimar et al.40

This was a retrospective study, which inherently limits the generalization of its findings. First, all HEV-ALF patients were enrolled from five hospitals located in different regions of China. Therefore, the study was easily subject to selection bias and there was considerable heterogeneity likely between units. Second, the nomogram may not be useful for pregnant females, as this cohort only include nine pregnant females. Third, the role of nomogram in HEV-related ACLF patients has not been discussed in this study and requires further focused investigation.

Conclusions

In summary, the noninvasive nomogram may serve as an important method of HEV-ALF mortality evaluation for clinicians, and also enhance patient stratification in clinical trials.

Supporting information

Supplementary Fig. 1

Screening flow chart for HEV-ALF patients. HBV, hepatitis B virus; HCV, hepatitis C virus.

(TIF)

Supplementary Fig. 2

Comparisons of the net benefit associated with the use of the nomogram, MELD score, and CLIF-C-ACLFs in the development and validation sets.

(TIF)

Supplementary Fig. 3

Kaplan-Meier survival curves of the development and validation sets, categorized by the nomogram. The curves showed different survival rates of patients with different nomogram scoring range (low risk: 0–200, and high risk: ≥201). (A) Development set. (B) Validation set.

(TIF)

Supplementary Table 1

Comparison of C-index of nomogram, MELD score, and CLIF-C-ACLFs for prognosis in the development set and validation set.

(DOC)

Supplementary Table 2

Comparison of td-AUC of nomogram, MELD score, and CLIF-C-ACLFs score for prognosis in the development set and validation set.

(DOC)

Abbreviations

ACLF: 

acute-on-chronic liver failure

AFP: 

alpha fetoprotein

ALB: 

albumin

ALF: 

acute liver failure

ALT: 

alanine aminotransferase

AST: 

aspartate aminotransferase

C-Index: 

concordance index

CHE: 

cholinesterase

CI: 

confidence interval

CLIF-C-ACLFs: 

CLIF-Consortium score

CR: 

creatinine

DBIL: 

direct bilirubin

DCA: 

decision curve analysis

GGT: 

γ-glutamyl transpeptidase

HE: 

hepatic encephalopathy

HEV: 

hepatitis E virus

HEV-ALF: 

HEV patients with acute liver failure

Ig: 

immunoglobulin

iMELD: 

integrated MELD

INR: 

international normalized ratio

MELD: 

model for end-stage liver disease

NAFLD: 

non-alcoholic fatty liver disease

NLR: 

neutrophil-to-lymphocyte ratio

OPLS-DA: 

orthogonal partial least squares-discriminant analysis

PT: 

prothrombin time

RBC: 

red blood count

RDW: 

red cell distribution width

RLR: 

RDW to lymphocyte ratio

SOFA: 

sequential organ failure assessment

TBIL: 

total bilirubin

td-AUC: 

time-dependent area under the receiver operating characteristic curve

td-ROC: 

time-dependent receiver operating characteristics

UREA: 

urea nitrogen

WBC: 

white blood cell

Declarations

Acknowledgement

We thank the authors of the primary studies for their timely and helpful responses to our information requests.

Data sharing statement

All data are available upon request.

Funding

This study was supported by a grant from the National Science and Technology Major Project for Infectious Diseases (2012ZX10002004).

Conflict of interest

The authors have no conflict of interests related to this publication.

Authors’ contributions

Conception and design (JW, CS, HC, LL), conduct of the literature search and writing of the manuscript (JW, CS), collection of patients’ samples and medical information (JZ, XS, GL, XZ), data analysis and generation of the tables and figures (XS, YX), statistical analysis (JY); obtained funding and critically revised the manuscript (HC, LL).

References

  1. Westhölter D, Hiller J, Denzer U, Polywka S, Ayuk F, Rybczynski M, et al. HEV-positive blood donations represent a relevant infection risk for immunosuppressed recipients. J Hepatol 2018;69(1):36-42 View Article PubMed/NCBI
  2. Wu J, Guo N, Zhang X, Xiong C, Liu J, Xu Y, et al. HEV-LFS : A novel scoring model for patients with hepatitis E virus-related liver failure. J Viral Hepat 2019;26(11):1334-1343 View Article PubMed/NCBI
  3. Rein DB, Stevens GA, Theaker J, Wittenborn JS, Wiersma ST. The global burden of hepatitis E virus genotypes 1 and 2 in 2005. Hepatology 2012;55(4):988-997 View Article PubMed/NCBI
  4. Shalimar, Acharya SK. Hepatitis e and acute liver failure in pregnancy. J Clin Exp Hepatol 2013;3(3):213-224 View Article PubMed/NCBI
  5. Manka P, Bechmann LP, Coombes JD, Thodou V, Schlattjan M, Kahraman A, et al. Hepatitis E virus infection as a possible cause of acute liver failure in Europe. Clin Gastroenterol Hepatol 2015;13:1836-1842.e2. quiz e157-e158 View Article PubMed/NCBI
  6. Dalton HR, Fellows HJ, Stableforth W, Joseph M, Thurairajah PH, Warshow U, et al. The role of hepatitis E virus testing in drug-induced liver injury. Aliment Pharmacol Ther 2007;26(10):1429-1435 View Article PubMed/NCBI
  7. Asrani SK, Kamath PS. Model for end-stage liver disease score and MELD exceptions: 15 years later. Hepatol Int 2015;9(3):346-354 View Article PubMed/NCBI
  8. Luca A, Angermayr B, Bertolini G, Koenig F, Vizzini G, Ploner M, et al. An integrated MELD model including serum sodium and age improves the prediction of early mortality in patients with cirrhosis. Liver Transpl 2007;13(8):1174-1180 View Article PubMed/NCBI
  9. Giannini E, Botta F, Fumagalli A, Malfatti F, Testa E, Chiarbonello B, et al. Can inclusion of serum creatinine values improve the Child-Turcotte-Pugh score and challenge the prognostic yield of the model for end-stage liver disease score in the short-term prognostic assessment of cirrhotic patients?. Liver Int 2004;24(5):465-470 View Article PubMed/NCBI
  10. Gustot T, Fernandez J, Garcia E, Morando F, Caraceni P, Alessandria C, et al. Clinical Course of acute-on-chronic liver failure syndrome and effects on prognosis. Hepatology 2015;62(1):243-252 View Article PubMed/NCBI
  11. Sundaram V, Jalan R, Wu T, Volk ML, Asrani SK, Klein AS, et al. Factors associated with survival of patients with severe acute-on-chronic liver failure before and after liver transplantation. Gastroenterology 2019;156(5):1381-1391.e3 View Article PubMed/NCBI
  12. Jalan R, Saliba F, Pavesi M, Amoros A, Moreau R, Ginès P, et al. Development and validation of a prognostic score to predict mortality in patients with acute-on-chronic liver failure. J Hepatol 2014;61(5):1038-1047 View Article PubMed/NCBI
  13. Guan YJ, Fang SY, Chen LL, Li ZD. Development and validation of prognostic nomograms for medullary thyroid cancer. Onco Targets Ther 2019;12:2299-2309 View Article PubMed/NCBI
  14. Zeigler-Johnson C, Hudson A, Glanz K, Spangler E, Morales KH. Performance of prostate cancer recurrence nomograms by obesity status: a retrospective analysis of a radical prostatectomy cohort. BMC Cancer 2018;18(1):1061 View Article PubMed/NCBI
  15. Zhou YJ, Zheng JN, Zhou YF, Han YJ, Zou TT, Liu WY, et al. Development of a prognostic nomogram for cirrhotic patients with upper gastrointestinal bleeding. Eur J Gastroenterol Hepatol 2017;29(10):1166-1173 View Article PubMed/NCBI
  16. McPhail MJ, Farne H, Senvar N, Wendon JA, Bernal W. Ability of king’s college criteria and model for end-stage liver disease scores to predict mortality of patients with acute liver failure: A meta-analysis. Clin Gastroenterol Hepatol 2016;14(4):516-525.e5. quiz e43-e45 View Article PubMed/NCBI
  17. Weissenborn K. Hepatic encephalopathy: Definition, clinical grading and diagnostic principles. Drugs 2019;79(Suppl 1):5-9 View Article PubMed/NCBI
  18. Choi HJ, Ryu JM, Kim I, Nam SJ, Kim SW, Yu J, et al. Nomogram for accurate prediction of breast and axillary pathologic response after neoadjuvant chemotherapy in node positive patients with breast cancer. Ann Surg Treat Res 2019;96(4):169-176 View Article PubMed/NCBI
  19. Zhong H, Chen J, Cheng S, Chen S, Shen R, Shi Q, et al. Prognostic nomogram incorporating inflammatory cytokines for overall survival in patients with aggressive non-Hodgkin’s lymphoma. EBioMedicine 2019;41:167-174 View Article PubMed/NCBI
  20. Cai YJ, Dong JJ, Dong JZ, Chen Y, Lin Z, Song M, et al. A nomogram for predicting prognostic value of inflammatory response biomarkers in decompensated cirrhotic patients without acute-on-chronic liver failure. Aliment Pharmacol Ther 2017;45(11):1413-1426 View Article PubMed/NCBI
  21. Gong J, Zhou W, Xiao C, Jie Y, Zhu S, Zheng J, et al. A nomogram for predicting prognostic value of inflammatory biomarkers in patients with acute-on-chronic liver failure. Clin Chim Acta 2018;478:7-12 View Article PubMed/NCBI
  22. Choi M, Hofmann J, Köhler A, Wang B, Bock CT, Schott E, et al. Prevalence and clinical correlates of chronic hepatitis E infection in German renal transplant recipients with elevated liver enzymes. Transplant Direct 2018;4(2):e341 View Article PubMed/NCBI
  23. Tomar LR, Aggarwal A, Jain P, Rajpal S, Agarwal MP. Acute viral hepatitis E presenting with haemolytic anaemia and acute renal failure in a patient with glucose-6-phosphate dehydrogenase deficiency. Trop Doct 2015;45(4):245-246 View Article PubMed/NCBI
  24. Marion O, Abravanel F, Del Bello A, Esposito L, Lhomme S, Puissant-Lubrano B, et al. Hepatitis E virus-associated cryoglobulinemia in solid-organ-transplant recipients. Liver Int 2018;38(12):2178-2189 View Article PubMed/NCBI
  25. Del Bello A, Guilbeau-Frugier C, Josse AG, Rostaing L, Izopet J, Kamar N. Successful treatment of hepatitis E virus-associated cryoglobulinemic membranoproliferative glomerulonephritis with ribavirin. Transpl Infect Dis 2015;17(2):279-283 View Article PubMed/NCBI
  26. Taton B, Moreau K, Lepreux S, Bachelet T, Trimoulet P, De Ledinghen V, et al. Hepatitis E virus infection as a new probable cause of de novo membranous nephropathy after kidney transplantation. Transpl Infect Dis 2013;15(6):E211-E215 View Article PubMed/NCBI
  27. Guinault D, Ribes D, Delas A, Milongo D, Abravanel F, Puissant-Lubrano B, et al. Hepatitis E virus-induced cryoglobulinemic glomerulonephritis in a nonimmunocompromised person. Am J Kidney Dis 2016;67(4):660-663 View Article PubMed/NCBI
  28. Mikolašević I, Sladoje-Martinović B, Orlić L, Milić S, Lukenda V, Župan Ž, et al. Evaluation of viral hepatitis in solid organ transplantation. Acta Med Croatica 2014;68(2):151-159 View Article PubMed/NCBI
  29. Bazerbachi F, Haffar S, Garg SK, Lake JR. Extra-hepatic manifestations associated with hepatitis E virus infection: a comprehensive review of the literature. Gastroenterol Rep (Oxf) 2016;4(1):1-15 View Article PubMed/NCBI
  30. Sarin SK, Kedarisetty CK, Abbas Z, Amarapurkar D, Bihari C, Chan AC, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific Association for the Study of the Liver (APASL) 2014. Hepatol Int 2014;8(4):453-471 View Article PubMed/NCBI
  31. Khoury T, Mari A, Nseir W, Kadah A, Sbeit W, Mahamid M. Neutrophil-to-lymphocyte ratio is independently associated with inflammatory activity and fibrosis grade in nonalcoholic fatty liver disease. Eur J Gastroenterol Hepatol 2019;31(9):1110-1115 View Article PubMed/NCBI
  32. Lin L, Yang F, Wang Y, Su S, Su Z, Jiang X, et al. Prognostic nomogram incorporating neutrophil-to-lymphocyte ratio for early mortality in decompensated liver cirrhosis. Int Immunopharmacol 2018;56:58-64 View Article PubMed/NCBI
  33. Kitano Y, Yamashita YI, Yamamura K, Arima K, Kaida T, Miyata T, et al. Effects of preoperative neutrophil-to-lymphocyte and platelet-to-lymphocyte ratios on survival in patients with extrahepatic cholangiocarcinoma. Anticancer Res 2017;37(6):3229-3237 View Article PubMed/NCBI
  34. Okamura Y, Sugiura T, Ito T, Yamamoto Y, Ashida R, Mori K, et al. Neutrophil to lymphocyte ratio as an indicator of the malignant behaviour of hepatocellular carcinoma. Br J Surg 2016;103(7):891-898 View Article PubMed/NCBI
  35. Zhang Y, Peng Z, Chen M, Liu F, Huang J, Xu L, et al. Elevated neutrophil to lymphocyte ratio might predict poor prognosis for colorectal liver metastasis after percutaneous radiofrequency ablation. Int J Hyperthermia 2012;28(2):132-140 View Article PubMed/NCBI
  36. Jiang AA, Greenwald HS, Sheikh L, Wooten DA, Malhotra A, Schooley RT, et al. Predictors of acute liver failure in patients with acute hepatitis A: An analysis of the 2016-2018 San Diego County hepatitis A outbreak. Open Forum Infect Dis 2019;6(11):ofz467 View Article PubMed/NCBI
  37. Manka P, Olliges V, Bechmann LP, Schlattjan M, Jochum C, Treckmann JW, et al. Low levels of blood lipids are associated with etiology and lethal outcome in acute liver failure. PLoS One 2014;9(7):e102351 View Article PubMed/NCBI
  38. Dao DY, Hynan LS, Yuan HJ, Sanders C, Balko J, Attar N, et al. Two distinct subtypes of hepatitis B virus-related acute liver failure are separable by quantitative serum immunoglobulin M anti-hepatitis B core antibody and hepatitis B virus DNA levels. Hepatology 2012;55(3):676-684 View Article PubMed/NCBI
  39. Dao DY, Seremba E, Ajmera V, Sanders C, Hynan LS, Lee WM. Use of nucleoside (tide) analogues in patients with hepatitis B-related acute liver failure. Dig Dis Sci 2012;57(5):1349-1357 View Article PubMed/NCBI
  40. Shalimar, Kedia S, Gunjan D, Sonika U, Mahapatra SJ, Nayak B, et al. Acute liver failure due to hepatitis E virus infection is associated with better survival than other etiologies in Indian patients. Dig Dis Sci 2017;62(4):1058-1066 View Article PubMed/NCBI