Home
JournalsCollections
For Authors For Reviewers For Editorial Board Members
Article Processing Charges Open Access
Ethics Advertising Policy
Editorial Policy Resource Center
Company Information Contact Us
OPEN ACCESS

Clinical Prediction Models for Hepatitis B Virus-related Acute-on-chronic Liver Failure: A Technical Report

  • Xia Yu# ,
  • Yi Lu# ,
  • Shanshan Sun ,
  • Huilan Tu ,
  • Xianbin Xu ,
  • Kai Gong ,
  • Junjie Yao ,
  • Yu Shi*  and
  • Jifang Sheng* 
Journal of Clinical and Translational Hepatology   2021;9(6):838-849

doi: 10.14218/JCTH.2021.00005

Received:

Revised:

Accepted:

Published online:

 Author information

Citation: Yu X, Lu Y, Sun S, Tu H, Xu X, Gong K, et al. Clinical Prediction Models for Hepatitis B Virus-related Acute-on-chronic Liver Failure: A Technical Report. J Clin Transl Hepatol. 2021;9(6):838-849. doi: 10.14218/JCTH.2021.00005.

Abstract

Background and Aims

It is critical but challenging to predict the prognosis of hepatitis B virus-related acute-on-chronic liver failure (HBV-ACLF). This study systematically summarized and evaluated the quality and performance of available clinical prediction models (CPMs).

Methods

A keyword search of articles on HBV-ACLF CPMs published in PubMed from January 1995 to April 2020 was performed. Both the quality and performance of the CPMs were assessed.

Results

Fifty-two CPMs were identified, of which 31 were HBV-ACLF specific. The modeling data were mostly derived from retrospective (83.87%) and single-center (96.77%) cohorts, with sample sizes ranging from 46 to 1,202. Three-month mortality was the most common endpoint. The Asian Pacific Association for the Study of the Liver consensus (51.92%) and Chinese Medical Association liver failure guidelines (40.38%) were commonly used for HBV-ACLF diagnosis. Serum bilirubin (67.74%), the international normalized ratio (54.84%), and hepatic encephalopathy (51.61%) were the most frequent variables used in models. Model discrimination was commonly evaluated (88.46%), but model calibration was seldom performed. The model for end-stage liver disease score was the most widely used (84.62%); however, varying performance was reported among the studies.

Conclusions

Substantial limitations lie in the quality of HBV-ACLF-specific CPMs. Disease severity of study populations may impact model performance. The clinical utility of CPMs in predicting short-term prognosis of HBV-ACLF remains to be undefined.

Keywords

Hepatitis B virus, Acute-on-chronic liver failure, Clinical prediction models, Quality and performance

Introduction

Acute-on-chronic liver failure (ACLF) is a clinically critical illness characterized by acute exacerbations of underlying chronic liver diseases with short-term high mortality.1,2 The etiology of underlying chronic liver diseases and precipitating events are distinct between Eastern and Western ACLF, which contributes to the heterogeneity of this syndrome.3 In Eastern ACLF, especially in China, hepatitis B virus related acute-on-chronic liver failure (HBV-ACLF) is the most common type.4

There are a variety of emerging therapies for HBV-ACLF, such as extracorporeal liver support device,5,6 glucocorticoid,7,8 granulocyte colony-stimulating factor (G-CSF),9 and cell therapies,10,11 but their efficacy requires further validation. Liver transplantation (LT) remains the only definite treatment to reduce the mortality of advanced HBV-ACLF12 but is limited by a lack of organ donors, huge financial cost of the procedure, and high mortality on the waiting list. In the European Association for the Study of the Liver–Chronic Liver Failure (EASL-CLIF) Acute-on-Chronic Liver Failure in Cirrhosis (CANONIC) study, ACLF patients had a 28-day mortality of 33.9%, and only 7.6% received LT.13 As a result, it is critical to precisely predict the short-term outcome of HBV-ACLF at the early stage of disease to make an accurate and prompt clinical decision of LT.

A number of clinical prediction models (CPMs) have been used to predict the short-term prognosis of HBV-ACLF utilizing laboratory and clinical variables that can be easily obtained in clinical practice. Some were specifically developed for HBV-ACLF, while others were originally developed for end-stage liver diseases [for instance, the model for end-stage liver disease (MELD) score,14 MELD-sodium (MELD-Na) score15 and Child-Turcotte-Pugh (CTP) score16], acute liver failure [King’s College Criteria (KCC)17], and other critical illness with organ failures [sequential organ failure assessment (SOFA)18]. Despite the number of available CPMs, there is no consensus on the use of optimal models to predict HBV-ACLF outcome. In addition, there are major concerns about the heterogeneity of study populations as well as model quality. Therefore, in the study, we systematically assessed both the performance and quality of available HBV-ACLF CPMs. We also analyzed the factors associated with heterogeneity and their predictive performance among different studies.

Methods

Study search and selection

A keyword search was carried out on articles related to HBV-ACLF published in PubMed from January 1995 to April 2020. The search strategy was developed as follows: (HBV OR hepatitis B) AND (severe flares of chronic hepatitis B OR chronic severe hepatitis B OR severe flare-up, chronic hepatitis B OR hepatic failure OR severe hepatitis B OR severe acute chronic hepatitis B (CHB) exacerbation OR hepatic decompensation OR severe acute exacerbation OR liver failure OR acute-on-chronic liver failure OR ACLF OR acute liver failure) AND (mortality OR prognosis OR outcome). Two reviewers (YX and LY) independently screened the searched articles based on the title, abstract, and full text sequentially. Disputes were resolved by negotiation between the two reviewers.

We included articles reporting the development of an HBV-ACLF-specific CPM or those assessing the predictive performance of previously established CPMs in non-HBV-ACLF-specific patients.

In addition, the included studies had clearly defined endpoints and reported the statistical modeling approaches if an HBV-ACLF-specific CPM was developed. For inclusion, the CPM had to contain at least two independent variables.

The exclusion criteria were as follows: (1) other types of publications, such as letters and reviews; (2) samples including patients younger than 18 years of age or pregnant women; (3) reports of biomarker-based prediction models; (4) reports of cost-benefit models; (5) experimental studies; or (6) decision-analysis studies.

Data extraction

We extracted the following information for each of the included articles: (1) year of publication; (2) study design; (3) study registration if reported; (4) diagnostic criteria for HBV-ACLF; (5) baseline characteristics of the study population; (6) sample size; (7) number of deaths or LT if reported; (8) variables included in the new CPMs; (9) statistical approaches for model development; and (10) model validation.

All information was independently extracted by the two reviewers, and disputes were resolved by negotiation between them.

Model assessment

Quality of HBV-ACLF-specific models

As shown in Supplementary Table 1, a scoring system was established by weighting study design, number of patients recruiting centers, sample size, adjustment of confounding factors, reporting of LT, and model validation. Studies with scores of 5–6 were considered high quality, 3–4 medium quality, and 1–2 low quality.

Performance of the CPMs

The performance of the CPMs was evaluated by discrimination and calibration.19 Discrimination referred to how well the model distinguished individuals at high risk of an event from those at low risk of an event.19 Calibration referred to the accuracy of absolute risk estimation.19 To measure model discrimination, we extracted the area under the receiver operating characteristic curve (AUROC) from each study. Quantitative pooled analysis of the discrimination performance of a specific model reported in several studies was performed by summary receiver operating characteristic (SROC) curves using Review Manager 5.3. To measure calibration, information on the Hosmer-Lemeshow test was extracted.

Ethics approval and consent to participate

The ethics committee of the First Affiliated Hospital of Zhejiang University reviewed and approved this study. Written consent from patients or their authorized representatives was waived.

Results

Characteristics of all CPMs

A total of 4,261 related studies were retrieved from PubMed based on the keyword search. According to the inclusion and exclusion criteria, 52 studies were selected after being screened by the title, abstract, and full text (Fig. 1). A total of 52 articles were extracted, of which 31 developed HBV-ACLF-specific CPMs and the other 21 assessed previously established CPMs. As shown in Figure 2, the number of publications is rapidly increasing each year. The studies were published in a number of academic journals (n=30), the most frequent being Chinese Journal of Hepatology [5 (9.62%)], followed by Medicine (Baltimore) [n=4 (7.69%)].

Flow chart of study selection.
Fig. 1  Flow chart of study selection.
Cumulative growth in relevant publications on PubMed by April 14, 2020.
Fig. 2  Cumulative growth in relevant publications on PubMed by April 14, 2020.

The diagnosis of HBV-ACLF in these studies was made mainly based on the Asian Pacific Association for the Study of the Liver (APASL) consensus for ACLF (51.92%) or the Chinese Medical Association (CMA) liver failure guidelines (40.38%). Among all studies, the sample size ranged from 46 to 1,202 patients. Significant heterogeneity was observed in patient characteristics among the different studies, as shown by the sex proportion (male/female) (ranging from 2.96 to 12.19), incidence of cirrhosis (24–100%), incidence of hepatic encephalopathy (10–51%), incidence of ascites (36–91%), and mean MELD score (20.97–29.00). The type of precipitating event was reported in seven studies (13.5%), with flare-up of hepatitis B being the major event in each study. Mortality varied among the different studies, with 3-month mortality ranging from 26% to 87%.

Regarding reporting of LT, 18 studies did not mention LT (34.62%), 21 excluded patients receiving LT (40.38%), and 5 defined LT and death as a composite endpoint (9.62%). LT was regarded as the censored event in six studies (11.54%). Patients with LT were defined as survivors in one study (1.92%). In one study, patients who received LT within 3 months were considered dead and more than 3 months as surviving.

In 8 studies (15.4%), dynamic parameters were used for modeling. ΔMELD or ΔMELD-Na calculated as the difference between MELD or MELD-Na at two time points was most frequent. One parameter was constructed based on the daily levels of predictive variables for 7 days after diagnosis combined with baseline risk factors. In the other studies, only baseline parameters were used.

Characteristics of HBV-ACLF-specific CPMs

Thirty-one CPMs were established specifically for HBV-ACLF (Table 1).

Table 1

Patient characteristics of HBV-ACLF-specific CPMs

ReferencesModelACLF diagnostic criteriaSampleDeath eventsEndpoint timeBasic characteristics of the study population at admission
Age in yearsSex, male/femaleCirrhosis, n/totalAscites, n/totalHE, n/totalTB in mmol/LINRMELD score
[1]Ke’s modelCMA205104NANANANANANANANANA
[2]Li’s modelCMA409215NA42±12378/31NANANANANANA
[3]Sun’s modelCMA20411890-day46.8±13.2170/34110/204NA86/204318.6±175.8NA26.0±9.0
[4]LRMAPASL45217590-day45.6±11.5361/91138/452334/452119/452NANANA
[5]He’s modelCMA1727590-day45.16±11.21144/28132/17296/172NA297.8±109.32.4±0.726.4±4.2
[6]TPPMAPASL24813390-day42.27±11.98225/2368/248152/24895/248270.9±140.32.0±0. 520.97±5.83
[7]Zheng’s modelAPASL72637190-day43.5±11.6635/91NA530/726251/726NANANA
[8]ALPH-QAPASL2148190-dayNA160/5499/214123/21445/214NANANA
[9]Yan’s modelAPASL43220990-day46.9±13.3329/103239/432348/432115/432351 (210)2.8 (1.6)27.8 (8.3)
[10]Yi’s modelAPASL39221890-dayNA323/69NANA165/392NANANA
[11]Li’s modelCMA33812990-day44.7±10.1268/70222/338220/33854/338NANANA
[12]HBV-ACLFsEASL-ACLF30015028-day46.5±11.3233/67300/300229/30071/300453.2±278.73.2±2.1NA
[13]HAMAPASL53019090-day41 (median)489/41246/530264/53095/530NANANA
[14]Chen’s modelAPASL55124190-dayNA465/86217/551NANANANANA
[15]MELD-LACAASL23610690-dayNA197/39131 / 236NANANANANA
[16]HINAT ACLFAPASL573153 (28-day), 219 (90-day)28-day, 90-day43.5±11.5478/98NA374/573117/573313.0±144.72.3±0.8NA
[17]Lei’s modelCMA138NAthe time of discharge or in-hospital death of the patient45.80±11.01111/2751/13896/138NANANANA
[18]Lin’s modelAPASL45617690-dayNA383/73NA228/45646/456NANANA
[19]Shi’s modelAPASL38475 (30-day), 106 (60-day), 125 (90-day), 127 (180-day)30-day, 60-day, 90-day, 180-dayNA303/81177/384236/38493/384NANANA
[20]Xue’s modelAPASL3058730-dayNA257/4889/305212/30592/305NANANA
[21]Gong’s modelCMA1847590-dayNA157/27NA122/184NANANANA
[22]Lin’s modelAPASL37011090-dayNA314/5688/370248/370103/370NANANA
[23]HINTAPASL63520430-day46.31±11.87538/97455/635239/635108/635319.1 (220.9, 421.0)2.02 (1.71, 2.55)23.07±5.95
[24]COSSH-ACLFEASL-ACLF657233 (28-day), 313 (90-day)28-day, 90-dayNA586/71466/657366/657130/657NANANA
[25]CTP-ABICCMA2228090-dayNA197/25168/222151/22244/222NANANA
[26]Gao’s modelAPASL1,202329 (28-day), 456 (90-day)28-day, 90-dayNA980/222382/1,202772/1,202282/1,202NANANA
[27]APMAPASL405NA28-dayNA358/47176/405144/40552/405NANANA
[28]ANNAPASL40216090-day47.2±13.3316/86NANANA297.5±169.32.9±1.728.2±6.2
[29]ANNAPASL684175 (28-day), 251 (90-day)28-day, 90-day43.9±11.6582/102NA405/684122/684323.5±148.42.3±0.822.9 (20.0, 26.5)
[30]CARTNA77731690-dayNA610/167371/777NANANANANA
[31]CARTEASL-CLIF489191 (28-day)28-dayNA424/65234/489234/48963/489NANANA

The diagnosis of HBV-ACLF in these studies was made mainly based on the APASL consensus [n=18 (58.06%)] or the CMA liver failure guidelines [n=8 (25.81%)]. EASL-ACLF criteria were used in four studies [n=2 (6.45%)] and Chinese Group on the Study of Severe Hepatitis B-acute-on-chronic liver failure (COSSH-ACLF) in one study [n=1 (3.23%)]. One study [n=1 (3.23%)] adopted the diagnostic criteria of acute liver failure proposed by the American Association for the Study of Liver Disease (AASLD). One study did not mention specific diagnostic criteria [n=1 (3.23%)].

As shown in Supplementary Table 1, 17 studies had a quality score of 0–2 (low quality), 12 had a score of 3–5 (medium quality), and only 2 had a score of 6–8 (high quality). Most were retrospective [n=26 (83.87%)] and single-center [n=30 (96.77%)], and only one was pre-registered. In terms of variable screening, most studies used regression approaches [n=26 (83.87%)]. The logistic regression model [n=14 (45.16%)] and the Cox hazard proportional model [n=12 (38.71%)] were the two methods most frequently used to identify risk variables. Two studies (6.45%) did not mention a clear variable screening method. Among the clinical variables consisting of CPMs, serum bilirubin (67.74%), international normalized ratio (INR) (54.84%), and hepatic encephalopathy (51.61%) were most frequent (Table 2). In terms of model formula, most CPMs were calculated as the results of multivariate logistic regression or Cox proportional hazard model as follows: (regression coefficients β1)×(variable 1)+(regression coefficients β2)×(variable 2)+(regression coefficients β3)×(variable 3)+….+constant (if logistic regression) (n=19 (61.29%). Three (9.68%) were calculated based on the sum of a series of categorical variables, the values of which were equally assigned [such as the Child-Turcotte-Pugh (CTP) score]; moreover, 5 (16.13%) were represented in the form of a nomogram, 2 (0.06%) were represented as an artificial neural network, and 2 (0.06%) were represented as a classification and regression tree.

Table 2

Variables consisting of model and screening approaches

New CPMsVariablesMethods
Ke’s modelTB; PTA; WBC; serum creatinine; maximum depth of ascites; HE score; singultus score; digestive tract hemorrhage scoreNot mentioned
Li’s modelHE; serum creatinine; PTA; TB; infection; liver size; ascites fluid levelClinical experience
Sun’s modelHR; LC; hepatitis B e antigen; ALB; PTALogistic regression
LRMHE; HR; LC; hepatitis B e antigen; PTA; AgeLogistic regression
He’s modelHE; serum creatinine; INR; TB at the end of 2 weeks of treatment; cholinesteraseLogistic regression
TPPMTB; INR; complications; HBV DNALogistic regression
Zheng’s modelTB; serum creatinine; PTA; HE; the maximum depth of ascites; WBCNot mentioned
ALPH-Qage; LC; PT; HE; QTcCOX regression
Yan’s modelage; HE score; MELDCOX regression
Yi’s modelHE; lnPTA2; lnINR2; lnTB2 (PTA2, INR2 and TB2 corresponded to those parameters at two weeks of treatment).Logistic regression
Li’s modelage; Family history of HBV; HE; HR; WBC; PLT; INR; TB; TBA; CHE; serum creatinine; serum sodium; HBV DNA; hepatitis B e antigenLogistic regression
HBV-ACLFsage; serum creatinine; WBCCOX regression
HAMMELD; HE; AFP; WBC; ageLogistic regression
Chen’s modelMELD, age, sodiumLogistic regression
MELD-LACLAC, MELDLogistic regression
HINAT ACLFHE, INR, NLRCOX regression
Lei’s modelNLR; serum levels of gamma-glutamyltransferase; ALB; sodium; artificial liver support therapyLogistic regression
Lin’s modelage; LAAR; MELDCOX regression
Shi’s modelage; TB; serum sodium; PTACOX regression
Xue’s modelTB; ALB; INR; Blood neutrophils percentage count; HE; Suspicion of infectionLogistic regression
Gong’s modelNLR; age; TBCOX regression
Lin’s modelTB; evolution of bilirubin; PTA; PLT; anti-HBeLogistic regression
HINTHE; INR; neutrophil count; TSHCOX regression
COSSH-ACLFINR; HBV-SOFA; Age; TBCOX regression
CTP-ABICCTP; ABICCOX regression
Gao’s modelage; TB; ALB; INR; HECOX regression
APMAFP; HE score; serum sodium; INRCOX regression
ANNserum sodium; TB; age; PTA; Hb; hepatitis B e antigenUnivariate analysis and Artificial neural network
ANNTB, PTA, serum sodium, HE, hepatitis B e antigen, GGT, ALP, ageUnivariate analysis and Artificial neural network
CARTTB, age, serum sodium, INRUnivariate Logistic regression and Classification and regression tree
CARTHE, PT, TBLogistic regression and Classification and regression tree

A total of 19 CPMs (61.29%) were validated, including 1 model that was validated by two cohorts. Single-center and multicenter validation cohorts were used in 14 and 6 studies, respectively (a single-center cohort and a multicenter cohort were used for the CPM with two validation cohorts). Eight of fourteen single-center validation cohorts were derived from the same center as the modeling cohorts, and the other six cohorts were derived from external centers. The validation cohort was prospective in five studies (26.32%) and retrospective in fourteen studies (73.68%). The patients in the model cohort and validation cohort were recruited during the same period in two studies but not in the other sixteen studies; one study did not mention the timing of recruitment. The sample size of the validation cohort was generally smaller than the derivation cohort and ranged from 88 to 300 patients.

Characteristics of non-HBV-ACLF-specific CPMs

A total of 21 studies evaluated the performance of CPMs that were non-specific for HBV-ACLF. Eighteen were single-center studies (85.7%) and three were multicenter studies (14.3%). Ten models developed for other diseases were evaluated, including KCC for acute liver failure, age-bilirubin-INR-creatinine (ABIC) score for alcohol liver diseases, albumin-bilirubin (ALBI) score for liver cancer, CTP, modified Child-Turcotte-Pugh (mCTP) score, MELD, MELD-Na, updated MELD (UpMELD), and MELD excluding the international normalized ratio (MELD-XI) score for end-stage liver diseases.

Model performance

Among the 52 selected studies, 50 evaluated model predictive performance. Forty-six studies reported the AUROC, four studies reported the C-Index, and only five studies reported the Hosmer-Lemeshow test to assess model calibration.

Table 3 presents the discriminative performance of each CPM. The AUROC of all CPMs varied between 0.521 and 0.970, the sensitivity between 34% and 100%, and the specificity between 2.60% and 93.31%. The AUROC of 31 CPMs specific for HBV-ACLF ranged from 0.63 to 0.97, the sensitivity from 44.44% to 92.6%, and the specificity from 42.3% to 95.31%. As shown in Table 2, the MELD score was the most widely used CPM (44 studies), followed by the MELD-Na score (21 studies) and the CTP score (19 studies). The capacity of discrimination of MELD varied widely among different studies, as indicated by the AUROC (between 0.58 and 0.94), sensitivity (between 43.70% and 100%), specificity (between 63.8% and 90.2%), and optimal cut-off point (between 21 and 32 points). Likewise, a large variation in predictive performance was seen in the MELD-Na score [AUROC (between 0.563 and 0.922), sensitivity (between 41.90% and 86.4%), specificity (between 61.9% and 86.7%), and optimal cut-off point (between 22.35 and 34.28)] and the CTP score [AUROC (between 0.553 and 0.878), sensitivity (between 34% and 99.35%), specificity (between 39.71% and 84%), and optimal cut-off point (between 9 and 12.5 points)].

Table 3

Discriminative performance of CPMs

ModelAUROC/C-IndexSensitivitySpecificityCut-offReferences
MELD0.58–0.9443.70–100%63.8–90.2%21–32[3–6,8–10,12,13,15–46,51],
Ke’s modelNANANANA[1]
KCC0.642–0.78341–59%2.6–87.7%0–0.5[32,36]
CTP0.553–0.87834–99.35%39.71–84%9–12.5[4,8–10,16–18,20,23,24,29,32,36,42,45–48],
MELD-Na0.563–0.92241.9–86.4%61.9–86.7%22.35–34.28[5,13,14,16–18,20,22,24–29,34,37,39,46,47,49,52]
Li’s model0.95397%82%9.5[2]
Sun’s model0.647–0.89168.6–72.3%52.1–52.5%−2.554[3,4,13]
Zhang’s model(LRM)0.68–0.91464–92.6%42.3–95.1%–0.3264–0.5176[3,4,8,13,30,36,41]
MELD-Na0.521–0.88641.9–78.21%50.5–90.16%25.6–32[10,12,13,14,28,36,42,49,50]
He’s model0.85±0.03NANANA[5]
iMELD0.540–0.86454.7–89.58%56.16–85%34.705–52[5,10,13,14,17,28,31,36,37,39,42]
MESO0.571–0.90538.7–80.77%75.25–91.80%1.986–21.61[5,10,13,28,42]
TPPM0.786–0.97084.09–89.6%61.54–94.7%0.22[6,25,38]
Zheng’s model0.900–0.970NANANA[7]
UpMELD0.68744.7%87.2%5.5[39]
MELD-XI0.64755.3%71.8%20.5[39]
UKMELD0.76657.6%81.6%45.5[39]
ALPH-Q0.837–0.89678–78.7%85.1%6.778[8]
Yan’s model0.853–0.86772–76%84.8–89.2%4.66[9]
SOFA0.705–0.75154.2–60%80.4–84.7%6.5[9,16]
CLIF-SOFA0.711–0.87654.3–80.14%64.56–91.1%7–8.5[9,16,23,44,50]
Yi’s model0.930±0.016NANANA[10]
iMELD-C0.776–0.86269.23–89.58%78.71–80.33%49.306–52.157[10]
LRM0.9386%87.1%3.16[11]
HBV-ACLFs0.704 (C-Index)NANANA[12]
CLIF-C ACLFs0.632–0.87361.86–93.65%63.7–78.6%36.78–43.76[12,16,23–27,29,31,44,46]
HAM0.868–0.89484.9–91.5%70.9–75%−1.191[13]
mCTP0.7491%48.8%14[42]
ALBI0.583–0.78462.2–65.9%67.2–81.4%–1.119–0.95[17,43,45]
ALBI+MELD0.91276.7%90.9%NA[43]
Chen’s model0.867NANANA[14]
MELD-LAC0.85991.5%80.1%−0.4741[15]
HINAT ACLF0.839–0.85582%74.5%4.6[16]
CLIF-C OF0.656–0.90653.9–92.6%72.9–78.8%8.5–10.5[16,24,25,44,45,46,50]
Lei’s model0.65662.2%64.1%NA[17]
Lin’s model0.854–0.890NANANA[18]
Shi’s model0.790–0.799 (C-Index)NANANA[19]
Xue’s model0.813–0.84844.44%93.63%NA[20]
ABIC0.695–0.82954.4–73.8%81.7%9.16–9.44[45,48]
Gong’s model0.63–0.742NANANA[21]
Lin’s model0.79–0.8667.3%91%−0.73[22]
HINT0.889–0.91774.60–79.43%84.56–95.31%−0.77[23]
COSSH-ACLF0.718–0.89854.9–89.04%55.56–91.78%3.7–6.4[23–27,31,50]
CLIF AD0.775NANANA[46]
CTP-ABIC0.92790%80.3%9.08[48]
AARC-ACLFs0.790NANANA[25]
Gao’s model0.58–0.80 (C-Index)NANANA[26]
APM0.747–0.79073.2%71.5%2.56[27]
ANN0.765–0.869NANANA[28]
ANN0.754–0.913NANANA[29]
CART0.896–0.90569.7–85.2%80.1–93.5%NA[30]
CART0.820–0.82488.2–88.6%62.7–68.5%NA[31]

In addition, we performed a pooled analysis of diagnostic accuracy of several common CPMs. As shown by the summary receiver operating characteristic (SROC) curves in Figure 3, the overall discriminative performance of the MELD score and chronic liver failure-sequential organ failure assessment (CLIF-SOFA) score seemed to be higher than those of the CTP score and MELD-Na score.

Relationship between MELD score on admission and AUROC values. MELD, model for end-stage liver disease; AUROC, area under the receiver operating characteristic curve.
Fig. 3  Relationship between MELD score on admission and AUROC values. MELD, model for end-stage liver disease; AUROC, area under the receiver operating characteristic curve.

AUROC, area under the receiver operating characteristic curve; MELD, model for end-stage liver disease.

Impact of ACLF severity and diagnostic criteria on model performance

To further analyze the factors contributing to the large variation in the predictive performance of a specific model among different studies, we compared the accuracy of MELD in HBV-ACLF defined by different diagnostic criteria. In APASL-defined ACLF patients, the AUROC of the MELD score was between 0.580 and 0.940, the sensitivity was between 43.7% and 88.9%, the specificity was between 67.2% and 90.2%, and the best cut-off point was between 21.57 and 29.6 points. In CMA-defined ACLF patients, the AUROC was between 0.612 and 0.906, the sensitivity was between 51% and 100%, the specificity was between 70.2% and 91.4%, and the best cut-off point was between 21 and 32 points.

Next, we assessed the relationship between the mean MELD value of patients at admission and the AUROC value of the MELD score. As shown in Figure 4, we found that the lower the mean MELD value of HBV-ACLF patients at admission, the greater the AUROC value. This suggested a negative correlation between disease severity at admission and the discriminative capacity of the MELD score.

SROC for MELD score, CTP score, MELD-Na score, iMELD score, LRM score and CLIF-SOFA score.
Fig. 4  SROC for MELD score, CTP score, MELD-Na score, iMELD score, LRM score and CLIF-SOFA score.

SROC, summary receiver operating characteristic curve; MELD, model for end-stage liver disease; CTP, Child-Turcotte-Pugh; MELD-Na, MELD-sodium; iMELD, integrated MELD; LRM, logistic regression model; CLIF-SOFA, chronic liver failure-sequential organ failure assessment.

Discussion

In this study, we systematically summarized the available clinical prediction models for HBV-ACLF and performed an extensive review of each study with regard to modeling data, modeling approach and model performance. Although the number of HBV-ACLF-specific CPMs has increased rapidly in the past 10 years, there are major concerns about the quality and reproducibility of most of them. Our analysis showed that the development of most HBV-ACLF-specific CPMs was flawed in the quality of modeling data. Most studies were retrospective in nature, recruited patients from a single center, and had limited sample sizes. The model proposed by the Chinese Group on the Study of Severe Hepatitis B (COSSH) consortium is the only CPM that was developed on the basis of national, multicenter, and prospective cohort data. Nevertheless, the COSSH HBV-ACLF model is not fully validated, as the validation cohort is single center and not from external study centers. Another frequent weakness is the absence of information on LT or inappropriate handling of LT data. Generally, LT is regarded as a competing event with death. However, a competing risk model in survival analysis has seldom been used. Few of the studies reported the indication of LT when adopting the use of a composite endpoint that combined death and LT. Either using an LT-free cohort or defining LT as a censored event may underestimate the mortality of the overall population and introduce bias in model development.

The MELD score is recognized as the mainstay for evaluating end-stage liver disease.20 It was originally developed to predict the short-term prognosis of cirrhotic patients undergoing transjugular intrahepatic portosystemic shunt (TIPS).14 The present analysis showed that MELD is the most commonly used CPM for predicting HBV-ACLF outcome. However, a large variation in the discriminative performance of MELD as indicated by AUROC, sensitivity and specificity was observed in different studies. This variation raises the concern that the heterogeneity of the study populations may impact model performance. The population heterogeneity may be due to the use of different diagnostic criteria in various studies (Table 4). The current analysis suggests that the use of MELD in APASL- and CMA-defined HBV-ACLF patients can obtain comparable discriminative performance because both diagnostic criteria identify ACLF patients characterized by high bilirubin and coagulopathy. On the other hand, our findings reveal a wide range of AUROC values for the MELD score despite using the same inclusion criteria for HBV-ACLF. Even when specific criteria are used, HBV-ACLF cases represent a heterogeneous population. Defining the population is confounded by the type of precipitating events (for instance, flare-up of hepatitis, use of hepatotoxic drugs, large alcohol consumption and so on) and the severity of underlying chronic liver diseases (non-cirrhotic chronic liver disease or compensated cirrhosis).3,21,22 Our findings showed that a lower MELD at admission has higher predictive power in HBV-ACLF, and the use of MELD in those with ultra-high MELD scores achieves high predictive performance as well.23 These findings suggest that the severity of HBV-ACLF is another important confounding factor of model performance and that preferential inclusion of patients at both ends of the severity spectrum would overestimate the predictive capacity of models. In addition, both 28-day and 90-day mortality were used as primary endpoints in different studies, thus contributing to varying degrees of predictive performance. Death events occurred frequently between 28 days and 90 days post-admission but were less frequent after 90 days in APASL-defined ACLF.24–26 The CANONIC study, which defined 28-day mortality as the primary endpoint, also reported much higher mortality at 90 days in patients with ACLF grade 1 or 2.13 Therefore, the use of 90-day mortality as the primary endpoint better fits the natural history of ACLF.

The present study identified common variables used in CPMs, in addition to the components of MELD. The presence of hepatic encephalopathy (HE) was frequently reported to be an independent variable associated with poor outcome.27 In addition, indicators of systemic inflammation, such as white blood cells (WBC) count, neutrophil percentage, and neutrophil-to-lymphocyte ratio (NLR), are common risk factors for short-term death.28 Other common variables included age, presence of ascites, serum sodium and hepatitis B e antigen presence. On the other hand, one of the MELD parameters, serum creatinine, was less frequently reported as an independent risk factor in HBV-ACLF. As a result, the overall predictive performance of MELD in HBV-ACLF is not satisfactory, and consistent with this finding, recent studies have shown limited capacity of MELD-Na in identifying ACLF patients at high risk of death on LT waiting lists.29–31 By contrast, a MELD-based scoring system that integrates HE and age outperforms the MELD score in predicting 90-day mortality of HBV-ACLF.32 In addition to the variables constituting the CPMs, model performance is determined by the weighting of specific variables. For example, although MELD does not include important criteria such as HE and ascites, the CTP with these parameters performed less well overall than the MELD score in which each variable is equally weighted.

In conclusion, a growing number of HBV-specific CPMs have been developed in recent years, but most are flawed in either the quality of the modeling data, the integrity of the modeling approach, or external validation. The MELD score is the most commonly used CPM, although it is non-HBV-specific. However, there is significant heterogeneity in the predictive performance of the MELD score among different studies due to the confounding effect of disease severity. Therefore, the clinical utility of CPMs in predicting the short-term prognosis of HBV-ACLF remains to be undefined. There is redundancy in the current HBV-ACLF CPMs, and there is an urgent need to establish high-quality prognostic models to better guide clinical practice. The development of future HBV-ACLF-specific CPMs should include the following elements to ensure the reliability of the model: (1) unified HBV-ACLF diagnostic criteria with a defined endpoint; (2) high-quality and unbiased modeling and validation data from prospective, large-sample, multicenter cohorts, as well as real-world validation; (3) selection of a couple of non-redundant and easily accessible variables for inclusion in the model via a well-adjusted process; (4) appropriate handling of events competing with death; (5) assessment of model discrimination and calibration; and (6) appropriate presentation of clinical utility.

Table 4

Similarities and differences of ACLF diagnostic criteria

CMAAPASLEASL-CLIFNACSELDCOSSH
DefinitionSevere liver damage caused by various insults on the basis of chronic liver disease, representing a clinical syndromes mainly manifesting as coagulopathy, jaundice, hepatic encephalopathy, ascites, etc.Acute hepatic insult manifesting as jaundice and coagulopathy. Complicated within 4 weeks by ascites and/or encephalopathy in a patient with previously diagnosed or undiagnosed chronic liver disease associated with high mortality.An acute deterioration of pre-existing chronic liver disease usually related to a precipitating event and associated with increased mortality at 3 months due to multisystem organ failure.A syndrome characterized by acute deterioration in a patient of cirrhosis due to infection presenting with two or more extrahepatic organ failure.A complicated syndrome with a high short-term mortality rate that develops in patients with HBV-related chronic liver disease regardless of the presence of cirrhosis and is characterized by acute deterioration of liver function and hepatic and/or extrahepatic organ failure.
Proposing time2006 (updated on 2014)2009 (updated on 2019)201320142017
Chronic liver diseasecompensated chronic liver diseaseNon-cirrhotic chronic liver disease and previously compensated cirrhosisDecompensated cirrhosisDecompensated cirrhosisNon-cirrhotic chronic liver disease and cirrhosis
Acute precipitating eventsAcute hepatic insultsAcute hepatic insultsAny and frequently without identifiable eventsInfectionAny and frequently without identifiable events
EtiologyAllAllAllAllHBV
Definition of liver failurePTA ≤40% and serum bilirubin ≥10 mg/dL or daily rise ≥1 mg/dLINR ≥1.5 and serum bilirubin ≥5 mg/dLSerum bilirubin ≥12 mg/dLNoneSerum bilirubin ≥12 mg/dL

Supporting information

Supplementary Table 1

(A) Scoring system criteria for assessing quality of HBV-ACLF-specific models; (B) Quality score of each HBV-ACLF-specific model.

(DOCX)

Supplementary File 1

Supplementary reference list.

(DOCX)

Abbreviations

AARC-ACLFs: 

APASL ACLF research consortium-ACLF

AASL: 

American Association for the Study of Liver Failure

ABIC: 

age-bilirubin-INR-creatinine

ACLF: 

acute-on-chronic liver failure

AFP: 

alpha-fetoprotein

ALB: 

albumin

ALBI: 

albumin-bilirubin

ALP: 

alkaline phosphatase

ANN: 

artificial neural network

APASL: 

Asian Pacific Association for the Study of the Liver

APLH-Q: 

age-PT-LC-HE-QTc

APM: 

artificial liver support system prognosis model

AUROC: 

area under the receiver operating characteristic curve

CART: 

classification and regression tree

CHE: 

cholinesterase

CLIF: 

chronic liver failure

CLIF AD: 

chronic liver failure-consortium acute decompensation

CLIF-C ACLFs: 

chronic liver failure-consortium acute-on chronic liver failure score

CLIF-C OF: 

chronic liver failure-consortium organ failure

CLIF-SOFA: 

chronic liver failure-sequential organ failure assessment

CMA: 

Chinese Medical Association

COSSH: 

Chinese Group on the Study of Severe Hepatitis B

CPMs: 

clinical prediction model

CTP: 

Child-Turcotte-Pugh

EASL-CLIF: 

European Association for the Study of the Liver–Chronic Liver Failure

G-CSF: 

granulocyte colony-stimulating factor

GGT: 

γ-glutamyltransferase

HAM: 

HBV-ACLF MELD

HB: 

hemoglobin

HBV: 

hepatitis B virus

HBV-ACLF: 

hepatitis B virus related acute-on-chronic liver failure

HE: 

hepatic encephalopathy

HINAT ACLF: 

HE-INR-NLR -age-TB ACLF

HINT: 

HE-INR-neutrophil count-thyroid stimulating hormone

HR: 

hepatorenal syndrome

ICU: 

intensive care unit

iMELD: 

integrated MELD model

iMELD-C: 

iMELD plus complications

INR: 

international normalized ratio

KCC: 

King’s College Criteria

LAAR: 

liver to abdominal area ratio

LAC: 

lactic acid

LC: 

liver cirrhosis

LRM: 

logistic regression model

LRM-Z: 

Z logistic regression model

LT: 

liver transplantation

mCTP: 

modified Child-Turcotte-Pugh

MELD: 

model for end-stage liver disease

MELD-LAC: 

MELD-lactate

MELD-Na: 

MELD-sodium

MELD-XI: 

MELD excluding the international normalized ratio

MESO: 

model for end-stage liver disease score to serum sodium ratio index

NACSELD: 

North American Consortium for the Study of End-Stage Liver Disease

NLR: 

neutrophil–lymphocyte ratio

PLT: 

platelet

PT: 

prothrombin time

PTA: 

prothrombin activity

QTc: 

QT interval corrected for heart rate

SOFA: 

sequential organ failure assessment

SROC: 

summary receiver operating characteristic curve

TB: 

total bilirubin

TBA: 

total bile acid

TPPM: 

Tongji prognostic predictor model

TSH: 

thyroid-stimulating hormone

UpMELD: 

updated MELD

UKMELD: 

United Kingdom MELD

WBC: 

white blood cells

Declarations

Data sharing statement

No additional data are available.

Funding

This work was supported by grants from the Chinese National Natural Science Foundation (Nos. 81670567 and 81870425) and the Fundamental Research Funds for the Central Universities.

Conflict of interest

The authors have no conflict of interests related to this publication.

Authors’ contributions

Conceptualization of the idea and design of the study (JS, YS), drafting of the manuscript (XY, YL), revision of the manuscript (YS, JS), and search and selection, data extraction, analysis, and interpretation (XY, YL, HT, XX, KG, JY). All authors read and approved the final manuscript.

References

  1. Hernaez R, Solà E, Moreau R, Ginès P. Acute-on-chronic liver failure: an update. Gut 2017;66(3):541-553 View Article PubMed/NCBI
  2. Bernal W, Jalan R, Quaglia A, Simpson K, Wendon J, Burroughs A. Acute-on-chronic liver failure. Lancet 2015;386(10003):1576-1587 View Article PubMed/NCBI
  3. Shi Y, Yang Y, Hu Y, Wu W, Yang Q, Zheng M, et al. Acute-on-chronic liver failure precipitated by hepatic injury is distinct from that precipitated by extrahepatic insults. Hepatology 2015;62(1):232-242 View Article PubMed/NCBI
  4. Shi Y, Zheng MH, Yang Y, Wei W, Yang Q, Hu A, et al. Increased delayed mortality in patients with acute-on-chronic liver failure who have prior decompensation. J Gastroenterol Hepatol 2015;30(4):712-718 View Article PubMed/NCBI
  5. Li H, Chen HS, Nyberg SL. Extracorporeal liver support and liver transplant for patients with acute-on-chronic liver failure. Semin Liver Dis 2016;36(2):153-160 View Article PubMed/NCBI
  6. Chen JJ, Huang JR, Yang Q, Xu XW, Liu XL, Hao SR, et al. Plasma exchange-centered artificial liver support system in hepatitis B virus-related acute-on-chronic liver failure: a nationwide prospective multicenter study in China. Hepatobiliary Pancreat Dis Int 2016;15(3):275-281 View Article PubMed/NCBI
  7. Fujiwara K, Yasui S, Yonemitsu Y, Fukai K, Arai M, Imazeki F, et al. Efficacy of combination therapy of antiviral and immunosuppressive drugs for the treatment of severe acute exacerbation of chronic hepatitis B. J Gastroenterol 2008;43(9):711-719 View Article PubMed/NCBI
  8. Chen JF, Wang KW, Zhang SQ, Lei ZY, Xie JQ, Zhu JY, et al. Dexamethasone in outcome of patients with hepatitis B virus-related acute-on-chronic liver failure. J Gastroenterol Hepatol 2014;29(4):800-806 View Article PubMed/NCBI
  9. Chavez-Tapia NC, Mendiola-Pastrana I, Ornelas-Arroyo VJ, Noreña-Herrera C, Vidaña-Perez D, Delgado-Sanchez G, et al. Granulocyte-colony stimulating factor for acute-on-chronic liver failure: systematic review and meta-analysis. Ann Hepatol 2015;14(5):631-641 View Article PubMed/NCBI
  10. Shi M, Zhang Z, Xu R, Lin H, Fu J, Zou Z, et al. Human mesenchymal stem cell transfusion is safe and improves liver function in acute-on-chronic liver failure patients. Stem Cells Transl Med 2012;1(10):725-731 View Article PubMed/NCBI
  11. Lin BL, Chen JF, Qiu WH, Wang KW, Xie DY, Chen XY, et al. Allogeneic bone marrow-derived mesenchymal stromal cells for hepatitis B virus-related acute-on-chronic liver failure: a randomized controlled trial. Hepatology 2017;66(1):209-219 View Article PubMed/NCBI
  12. Chan AC, Fan ST, Lo CM, Liu CL, Chan SC, Ng KK, et al. Liver transplantation for acute-on-chronic liver failure. Hepatol Int 2009;3(4):571-581 View Article PubMed/NCBI
  13. Moreau R, Jalan R, Gines P, Pavesi M, Angeli P, Cordoba J, et al. Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis. Gastroenterology 2013;144(7):1426-1437, e1421-1429 View Article PubMed/NCBI
  14. Malinchoc M, Kamath PS, Gordon FD, Peine CJ, Rank J, ter Borg PC. A model to predict poor survival in patients undergoing transjugular intrahepatic portosystemic shunts. Hepatology 2000;31(4):864-871 View Article PubMed/NCBI
  15. Biggins SW, Kim WR, Terrault NA, Saab S, Balan V, Schiano T, et al. Evidence-based incorporation of serum sodium concentration into MELD. Gastroenterology 2006;130(6):1652-1660 View Article PubMed/NCBI
  16. Pugh RN, Murray-Lyon IM, Dawson JL, Pietroni MC, Williams R. Transection of the oesophagus for bleeding oesophageal varices. Br J Surg 1973;60(8):646-649 View Article PubMed/NCBI
  17. O’Grady JG, Alexander GJ, Hayllar KM, Williams R. Early indicators of prognosis in fulminant hepatic failure. Gastroenterology 1989;97(2):439-445 View Article PubMed/NCBI
  18. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22(7):707-710 View Article PubMed/NCBI
  19. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. Jama 2017;318(14):1377-1384 View Article PubMed/NCBI
  20. Wong ZH, Cai SQ. Analysis of prognosis on patients with severe viral hepatitis using the model for end-stage liver disease. Zhonghua Gan Zang Bing Za Zhi 2005;13(4):249-251 View Article PubMed/NCBI
  21. Zhao RH, Shi Y, Zhao H, Wu W, Sheng JF. Acute-on-chronic liver failure in chronic hepatitis B: an update. Expert Rev Gastroenterol Hepatol 2018;12(4):341-350 View Article PubMed/NCBI
  22. Solà E, Fernandez J, Ginès P. Acute-on-chronic liver failure: the role of precipitating illness. Semin Liver Dis 2016;36(2):117-122 View Article PubMed/NCBI
  23. Fung J, Mak LY, Chan AC, Chok KS, Wong TC, Cheung TT, et al. Model for end-stage liver disease with additional criteria to predict short-term mortality in severe flares of chronic hepatitis B. Hepatology 2020;72(3):818-828 View Article PubMed/NCBI
  24. Sarin SK, Choudhury A, Sharma MK, Maiwall R, Al Mahtab M, Rahman S, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian pacific association for the study of the liver (APASL): an update. Hepatol Int 2019;13(4):353-390 View Article PubMed/NCBI
  25. Sarin SK, Kedarisetty CK, Abbas Z, Amarapurkar D, Bihari C, Chan AC, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian pacific association for the study of the liver (APASL) 2014. Hepatol Int 2014;8(4):453-471 View Article PubMed/NCBI
  26. Sarin SK, Kumar A, Almeida JA, Chawla YK, Fan ST, Garg H, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian pacific association for the study of the liver (APASL). Hepatol Int 2009;3(1):269-282 View Article PubMed/NCBI
  27. Zhang J, Liu X, Ye B, Chen H. Ammonia predicts poor outcomes in patients with hepatitis B virus-related acute-on-chronic liver failure. J Clin Lab Anal 2020;34(12):e23553 View Article PubMed/NCBI
  28. Wu W, Yan H, Zhao H, Sun W, Yang Q, Sheng J, et al. Characteristics of systemic inflammation in hepatitis B-precipitated ACLF: differentiate it from No-ACLF. Liver Int 2018;38(2):248-257 View Article PubMed/NCBI
  29. Hernaez R, Liu Y, Kramer JR, Rana A, El-Serag HB, Kanwal F. Model for end-stage liver disease-sodium underestimates 90-day mortality risk in patients with acute-on-chronic liver failure. J Hepatol 2020;73(6):1425-1433 View Article PubMed/NCBI
  30. Sundaram V, Jalan R, Wu T, Volk ML, Asrani SK, Klein AS, et al. Factors Associated with survival of patients with severe acute-on-chronic liver failure before and after liver transplantation. Gastroenterology 2019;156(5):1381-1391.e1383 View Article PubMed/NCBI
  31. Sundaram V, Shah P, Wong RJ, Karvellas CJ, Fortune BE, Mahmud N, et al. Patients with acute on chronic liver failure grade 3 have greater 14-day waitlist mortality than status-1a patients. Hepatology 2019;70(1):334-345 View Article PubMed/NCBI
  32. Yan H, Wu W, Yang Y, Wu Y, Yang Q, Shi Y. A novel integrated model for end-stage liver disease model predicts short-term prognosis of hepatitis B virus-related acute-on-chronic liver failure patients. Hepatol Res 2015;45(4):405-414 View Article PubMed/NCBI