Denoised recurrence label-based deep learning for prediction of postoperative recurrence risk and sorafenib response in HCC

Abstract

Background

Pathological images of hepatocellular carcinoma (HCC) contain abundant tumor information that can be used to stratify patients. However, the links between histology images and the treatment response have not been fully unveiled.

Methods

We trained and evaluated a model by predicting the prognosis of 287 non-treated HCC patients postoperatively, and further explored the model’s treatment response predictive ability in 79 sorafenib-treated patients. Based on prognostic relevant pathological signatures (PPS) extracted from CNN-SASM, which was trained by denoised recurrence label (DRL) under different thresholds, the PPS-based prognostic model was formulated. A total of 78 HCC patients from TCGA-LIHC were used for the external validation.

Results

We proposed the CNN-SASM based on tumor pathology and extracted PPS. Survival analysis revealed that the PPS-based prognostic model yielded the AUROC of 0.818 and 0.811 for predicting recurrence at 1 and 2 years after surgery, with an external validation reaching 0.713 and 0.707. Furthermore, the predictive ability of the PPS-based prognostic model was superior to clinical risk indicators, and it could stratify patients with significantly different prognoses. Importantly, our model can also stratify sorafenib-treated patients into two groups associated with significantly different survival situations, which could effectively predict survival benefits from sorafenib.

Conclusions

Our prognostic model based on pathology deep learning provided a valuable means for predicting HCC patient recurrence condition, and it could also improve patient stratification to sorafenib treatment, which help clinical decision-making in HCC.

Background

Hepatocellular carcinoma (HCC) accounts for 75–85% of primary liver cancers with high incidence and mortality, and the 5-year survival rates of HCC patients with hepatectomy are only 21% [1]. Sorafenib is the first FDA-approved first-line targeted therapy for advanced HCC [2] due to its good efficacy and manageable safety [3]. However, only a fraction of patients derive long-term benefits [4, 5]. More recently, the FDA approved many molecular targeted therapies [6, 9] and immune checkpoint inhibitors (ICIs) [10] for the treatment of advanced HCC, but the efficacy of these therapies in HCC is still limited [11]. A standardized risk‐stratification system for HCC patients is essential to improve the benefits of adjuvant systemic therapies after curative resection/ablation. Thus, it is urgent to identify biomarkers that predict response to therapy or individuals most likely to develop resistance.

Clinical pathological analysis is crucial and essential for diagnosing and stratifying cancer patients [12, 13]. Pathological images contain numerous phenotypic descriptions and pathological patterns that reflect the comprehensive evaluation of tumor diagnosis, prognosis, and clinical management [14,15,16,17]. With the advent of whole-slide imaging (WSI) [18] in digital pathology, deep learning-based pathological analysis exhibits the capacity to dynamically extract and decipher intricate tumor image features [19, 20], thereby presenting an approach for automated diagnostic procedures and prognostic modeling in patient care. Accumulating studies have demonstrated that pathological images can be used for the detection of early-stage [21] and microvascular invasion [22] of HCC based on convolutional neural networks (CNNs). Risk stratification to predict the survival of HCC based on pathological images has been reported recently [23, 24]. However, histopathological images that contain fundamental information on phenotype underlying HCC treatment response and outcomes remain largely unknown. Therefore, it is urgent to translate the histopathological features of HCC into predictive algorithms to inform clinical management and personalized cancer therapy.

Compared to the diagnosis tasks, the performance of deep learning on prognosis tasks is undesirable. Therefore, little research focuses on further investigation of the response to a specific treatment. Besides, it is more challenging to evaluate prognosis based on unreliable labels in the real world because patients’ outcomes encompass numerous variables beyond the disease itself. This poses challenges for executing end-to-end prognostic prediction utilizing pathological images. To address this issue, specific research indirectly forecasts outcomes by utilizing intermediate understandable pathological markers like tumor mutation burden (TMB) [25] and microvascular invasion [26]. In our study, we proposed to denoise the recurrence labels for the first time and use them to train the pathological model, which was more effective in extracting pathological features and predicting the prognosis. In constructing the pathological model, different from the conventional attention and self-attention mechanisms, we considered the mutual correlation between channels or spatial locations of the input data by the cross-self-attention module. It could achieve favorable classification ability by limited parameters.

In this work, we developed a CNN-SASM architecture by integrating CNN with channel and spatial self-attention mechanism modules to extract prognostic relevant pathological signatures (PPS), training by denoised recurrence label (DRL). PPS turned out to be associated with specific clinical histological regions. Furthermore, we constructed a PPS-based prognostic model to explore its crucial role in predicting prognosis of HCC patients compared to existing clinical risk indicators, and then explored the prognostic model’s survival benefits predictive capability to sorafenib.

Methods

Patients and follow-up

A total of 372 patients with 744 pathological slides who underwent curative resection for HCC at Zhongshan Hospital, Fudan University (Shanghai, China) were enrolled in the present study. Among them, 292 patients in the non-treatment group did not receive postoperative medication and 80 patients in the sorafenib group were treated with sorafenib postoperatively. Pathological tissue specimens were sampled during the surgery twice per patient. During the follow-up, patients were monitored every 2–3 months after surgery including serum a-fetoprotein (AFP), ultrasonography, and chest X-ray. If tumor recurrence was suspected, computed tomography (CT) scanning or magnetic resonance imaging (MRI) was conducted. The experiment was carried out in line with the principles of the Declaration of Helsinki. The study was reviewed and approved by the medical ethics committee of Zhongshan Hospital, Fudan University, with the approval number B2021-143R. We further excluded patients who followed up less than 2 years. The exclusion criteria of pathological images were (1) loss of tissue structure and (2) an extensive area of out-of-focus. Ultimately, 366 patients with 724 pathological images were included for the subsequent analysis after the screening. The same preprocessing was conducted on the TCGA-LIHC external validation set [27], and 78 patients were incorporated.

Data preprocessing

Limited by the computer’s memory and the large size of a single pathological image, we cut all pathological images into $224\times 224$ sizes without overlap in the tissue area. After that, patches with more than 50% background area were excluded, and the remaining patches were stain normalized by Vahadane’s approach [28], resulting in an average of 65 patches per image. Data augmentation was also carried out, including random flipping, rotating, adding Gaussian noise, and randomly erasing with a certain probability. Images from TCGA were in WSI format, so we processed them with the Python DeepZooming package, and patches were obtained by undersampling to the same number as the training set from all patches to identify the same distribution. Other preprocessing operations were similar to the internal dataset.

Denoising recurrence label

The real-world prognostic labels are less reliable than diagnostic labels because of the complicated non-disease factors, such as surgical outcomes, postoperative medication types and cycles, postoperative care, and physical or psychological conditions, which make it challenging to predict recurrence directly. To settle this issue, we proposed using DRL, recurrence labels with confidence levels higher than the threshold, instead of raw recurrence labels without filtering to train a pathological model, specifically the CNN-SASM mentioned below. DRL could be further categorized into denoised recurrence (DR) and denoised recurrence-free (DRF). The agreement between patients’ recurrence status and overall survival (OS) time determined the confidence of DRL, and we then divided the training set into various subsets according to various confidence levels. For instance, the 1/4 subset consisted of non-recurrence and recurrence patients with OS in the top and bottom quartile, respectively. With the increase in data sizes, DRL’s confidence worsened, and we iterated this sampling operation until all training data was included. The subsequent formula defines the subset split criteria:

$$\text{Subset}_t = \text{Patients} \quad \text{which} \left\{\begin{array}{ll}OS> OS_t \quad \text{and} \quad \text{label} = \text{Relapse-Free} \\OS < OS_t' \quad \text{and} \quad \text{label} = \text{Relapse}\end{array}\right.$$

where ${\text{Patients}}_{OS}$ represents patients with overall survival of OS, t represents the tth subset of training set, and ${OS}_{t}$ and ${OS}_{t}^{{\prime}}$ are the t-quantiles of OS for relapse-free and relapse patients, respectively. We obtained 6 subsets, including 1/4, 1/3, 1/2, 2/3, 3/4, and 1/1 ratios (Table S1). The external test set of this part was comprised of 14 patients (7 cases in each category) from TCGA following a similar principle.

Model design and construction

Construction of CNN-SASM

The CNN-SASM framework depicted in Fig. 1C chiefly consists of three components: the convolution (Conv) module, the channel self-attention (CSA) and spatial self-attention (SSA) modules, and the classification (Cls) module.

Fig. 1

Workflow diagram of the study. A Process of the pre-processing of pathological images and the criteria for subset selection based on confidence level. Patches were deemed non-qualified if they had an excessive background or lacked cellular structures. The pathological model was trained using DRL for labeling, with recurrence status as a pseudo-label, and the confidence was determined by the consistency between the patient’s recurrence status and overall survival. B Dataset structure. All subsets were selected from the training set, and the test set contained three sections: non-treatment internal test set, non-treatment TCGA, and sorafenib-treated patients in the test set, respectively. C The framework of the CNN-SASM detailed the data’s forward propagation and the model’s architecture. D Selection of the best subset with the corresponding optimum parameters CNN-SASM. E Training process of the prognostic model and the corresponding prognostic tasks

Conv module: Considering the limited data size, we employed ResNet18 pre-trained on ImageNet, except for the last two layers, as the foundational architecture for the convolution module by transfer learning. Assuming the input data ${{X}_{\text{input}}\in {\mathbb{R}}}^{B\times 3\times H\times W}$, it could produce the output ${{X}_{\text{output}}\in {\mathbb{R}}}^{B\times C\times {H}^{\prime}\times {W}^{\prime}}$ after one iteration. Here, B refers to the batch size; three corresponds to the RGB color channels; H and W represent the height and width of the input data, while ${H}^{\prime}$ and ${W}^{\prime}$ correspond to the output feature map. C describes the number of output channels.

CSA and SSA modules: We introduced cross multi-head self-attention mechanisms in both channel and spatial dimensions to capture interactions across channels and spatial locations, followed by the softmax layer to allocate more weight to vital channels and spatial locations. In the CSA computation, the number of channels corresponded to the number of tokens in sequence models. Specifically, the feature map ${{Fea}_{i}\in {\mathbb{R}}}^{{H}^{\prime}\times {W}^{\prime}}$ was reshaped into a vector ${Vec}_{i}\in {\mathbb{R}}^{{H}^{\prime}{\times W}^{\prime}}$ in the ith channel, which acted as the embedding representation for that particular channel. The vectors ${q}_{\text{m}}$, ${k}_{\text{m}}$, and ${v}_{\text{m}}$ were derived from the linear projection of ${Vec}_{i}$ and could be decomposed into multiple single attention head vectors ${q}_{\text{s}}^{n}$, ${k}_{\text{s}}^{n}$, and ${v}_{\text{s}}^{n}$, where $\text{m}$ stands for multi-heads, s denotes the single-attention head, and $n$ represents the nth attention head. The self-attention scores, termed attention-score (AS), and corresponding weighted vectors were independently calculated for each single-attention head. Then, the final output was formed by concatenating the weighted results from all attention heads.

$${q}_{\text{m}}={w}_{q}\times {Vec}_{i}+{b}_{q}=\text{Concat}({q}_{\text{s}}^{n})$$

$${k}_{\text{m}}={w}_{k}\times {Vec}_{i}+{b}_{k}=\text{Concat}({k}_{\text{s}}^{n})$$

$${v}_{\text{m}}={w}_{v}\times {Vec}_{i}+{b}_{v}=\text{Concat}({v}_{\text{s}}^{n})$$

$${AS}^{n}=\text{Dropout}(\text{SoftMax}(\frac{{q}_{\text{s}}^{n}\times {({k}_{\text{s}}^{n})}^{\text{T}}}{\sqrt{{d}_{\text{s}}}}))$$

$$\text{CSA}\left({X}_{\text{Conv}}\right)=\text{Dropout}(W\times \text{Concat}\left({AS}^{n}\times {v}_{\text{s}}^{n}\right)+b)$$

$${X}_{\text{CSA}}=\text{SoftMax}(\text{LayerNorm}({X}_{\text{Conv}}+\text{CSA}({X}_{\text{Conv}})))$$

which $n=\left[{n}_{1},{n}_{2}, {n}_{3},\cdots {,n}_{nu{m}_{\text{heads}}}\right]$. The softmax layer was incorporated, followed by self-attention, allowing the model to distinguish the significant features across the sequence dimension and normalize the trainable parameters. The design of SSA mirrored CSA but differed in that it treated spatial locations as tokens and recognized channel features as embeddings.

$${X}_{\text{SSA}}=\text{SoftMax}(\text{LayerNorm}({\text{Output}}_{\text{CSA}}+\text{SSA}(\text{Input})))$$

Cls module: This module consisted of a max-pooling layer and a multi-layer perceptron (MLP). Their primary function was to extract feature vectors and then execute the final classification. Max-pooling was performed on each channel dimension so that the feature dimension changed from ${\mathbb{R}}^{B\times C\times {H}{\prime}{W}{\prime}}$ to ${\mathbb{R}}^{B\times C}$. The MLP was the combination of two fully connected layers incorporating activation functions. Finally, the embedding features of patches were encoded into 128-dimensional patch-level feature vectors and binary classification outputs after Cls, and the former was ${\text{PPS}}_{\text{patch}}.$

$${\text{PPS}}_{\text{patch}}=\text{Maxpooling}({\text{Output}}_{\text{SSA}})$$

$${\text{DRS}}_{\text{patch}}=\text{Mlp}({\text{PPS}}_{\text{patch}})$$

During the training phase, we employed the cross-entropy loss function with a learning rate of 0.0001, the batch size was set to 128, and the training encompassed 50 epochs. Early termination occurs if the loss decreases for 5 consecutive epochs. The goal of CNN-SASM convergence was it could distinguish DR and DRF.

The aggregation of ${\text{PPS}}_{\text{image}}$: After being well-trained on the subset, CNN-SASM could be applied to inference, and we could extract the ${\text{PPS}}_{\text{patch}}$ directly from it. The image-level PPS was derived from averaging the corresponding ${\text{PPS}}_{\text{patch}}$ belong to the same image.

$${\text{PPS}}_{\text{image}}^{t}=\frac{1}{N}{\sum }_{i=1}^{N}{\text{PPS}}_{\text{patch}}^{i}$$

where N denotes the patch number of the tth image.

We took the CNN architecture ResNet18 and DenseNet121 as the pathological baseline models, along with the current SOTA model, Swin Transformer, based on self-attention mechanisms. All model parameters were initialized with pre-trained weights via transfer learning.

Construction of PPS-based prognostic model

The prognostic model was designed to evaluate the recurrence risk of postoperative HCC patients based on PPS extracted from CNN-SASM. We applied a MLP similar to the Cls in CNN-SASM to generate the prediction. Specifically, the input was the ${\text{PPS}}_{\text{image}}\in {\mathbb{R}}^{1\times 128}$, and the output was binary classification nodes with a dimension ${\mathbb{R}}^{1\times 2}$, where the first value represented the probability of non-recurrence risk for the image, and the second value denoted the probability of recurrence risk, with the sum of the two equal to one. The recurrence risk probabilities would be averaged if the patient had more than one pathological image. At this stage, the learning rate was fixed at 0.001, with all other parameters maintained consistently with the CNN-SASM training process.

Software and statistical analysis

This study utilized Python 3.9.12 and Pytorch 2.0.1. It evaluated the performance of CNN-SASM and the prognostic model primarily using AUROC. Patch-level clustering visualization was achieved through PCA and t-SNE. The significance tests were conducted by independent t-tests if two groups of data both conformed to the normal distribution or else we applied Wilcoxon rank-sum tests, which were both used in our study. The correlation test used the Pearson correlation coefficient. Significance scoring in survival analysis was conducted using the log-rank test, and prognostic ability was quantified through the concordance index (C-index) and hazard ratio (HR). All significance thresholds were set to 5%.

Results

Study design and patient characteristics

The overall study design is shown in Fig. 1. We first trained and validated the pathology model CNN-SASM by a series of pathological image subsets derived from the training set, then selected the optimal CNN-SASM in both performance and generalizability from all subset models as the PPS exactor. The rationale for splitting training subsets is that we believe complicated factors beyond the disease lead to the recurrence label in the real world, so we need to denoise the recurrence label and make it more associated with HCC itself; the new label was named DRL. We finally constructed a simple MLP using PPS to predict HCC patients’ postoperative recurrence at different points of time and survival benefits from sorafenib treatment.

A total of 366 patients met the inclusion criterion from 372 recruited patients and were divided into training and test sets in a ratio of 7:3. The detailed clinicopathological characteristics are listed in Table 1. Among these, 79 patients were treated with sorafenib postoperatively (Additional file 1: Fig. S1). The detailed dataset structure can be found in Additional file 1: Fig. S2. A total of 78 patients from the TCGA-LIHC project were applied to test the generalizability of the prognostic model.

Table 1 Clinical characteristics of the subjects

CNN-SASM accurately captures PPS of patches

We trained CNN-SASM on each subset to select the optimal subset with the best model parameters. CNN-SASM’s classification performance in DRL declined in both patch and TMA core levels as the samples of subsets increased (Fig. 2B), and a similar trend was also observed for accuracy (Additional file 1: Fig. S4B), proving effective in denoising recurrence labels. To trade off the classification performance and the sample size, we carried out generalizability tests in a TCGA subset with strictly DRL, containing 910 patches. The result suggested that the generalizability increased steadily until the 1/2 subset, then fluctuated, which might be due to the instability of the added samples. Therefore, we selected the CNN-SASM trained on the 1/2 subset as the PPS exactor, which achieved an AUROC of 0.912 and an AUPR of 0.921 at the patch level on the corresponding test set, with an accuracy of 0.830 computing from the confusion matrix (Fig. 2D), indicating CNN-SASM captured prognostic relevant signatures from pathological images.

Fig. 2

Selection and performance of CNN-SASM. A Five-fold cross-validation results of CNN-SASM compared with the baseline models on different subsets. B–C The AUROC scores of CNN-SASM for all subsets, along with the model’s generalization performance on the TCGA dataset. D–F The confusion matrix, ROC curve, and PR curve for the internal test set at the patch level. G–H The PCA and t-SNE clustering visualizations at the patch level for the internal test set and the TCGA dataset. ROC, receiver-operating characteristic

To visualize the representation ability of PPS qualitatively, we further displayed the PCA and t-SNE results in the internal test set and TCGA subset (Fig. 2G, H). DRF and DR patches could be separated well according to PPS despite slight mixing existing and that might be caused by noised labels. In this way, clustering results on the rest subsets could also be conducted (Additional file 1: Figs. S6 and S7), from clear boundaries to blending, they followed the same trend as AUROC and accuracy.

ResNet18 [29], DenseNet121 [30], and Swin-Transformer [31] represent classical model architectures within the domain of deep learning, frequently employed for image-related tasks. We found that our proposed pathological model, CNN-SASM, was superior to ResNet18, DenseNet121, and even Swin-Transformer in classifying DRF and DR on 1/2 subset and average performance with just little more trainable parameters than ResNet18 (Fig. 2A).

To further explore the interpretability of the CSA and SSA modules, we employed Grad-CAM to visualize their respective attention regions (Additional file 1: Fig. S5). As expected, the CSA module assigned varying attention weights to different channels, with channel 10 and 257 receiving the highest attention, indicating their importance for the model, while channel 39 and 154 had minimal impact on the prediction results. Similarly, the SSA module generated distinct attention weights across spatial locations of the image. The activation map of the DR image predicted as recurrence exhibited more prominent activation, whereas the DRF image showed the opposite pattern. By assigning higher weights to the most important channels and spatial regions, the CSA and SSA modules enable CNN-SASM to achieve substantial prediction performance.

Additionally, ablation experiments were performed to ascertain the efficacy of the CSA, SSA modules, and the softmax layer in CNN-SASM quantitatively. Eliminating either the CSA or SSA module resulted in a decline in model performance. Among them, the CSA module played a more significant role in CNN-SASM. Specifically, after removing the CSA module, the model’s AUROC on 1/2 subset dropped from 0.951 to 0.932 (Table 2). The softmax layer could function analogously as the conventional attention layer within limited samples (Table 2, Additional file 1: Fig. S4A).

Table 2 The performance of various models

Correlation between the PPS and known pathologic characteristics

In order to have a better understanding of PPS in morphological and biological interpretation, we visualized the attention map of the convolutional layer using the Grad-CAM method. CNN-SASM concentrated on the cell nucleus for both DR and DRF patches. It was correlated with clinical benchmarks for tumor assessment, demonstrating the significance and reliability of CNN-SASM in distinguishing DR and DRF patches (Fig. 3A). Regarding patch morphology, tumor cell nuclei on DRF patches were smaller with noticeable glandular structures. In contrast, nuclei on DR patches were larger with visible nucleoli. The biological significance of these morphological differences in prognosis needs to be further investigated.

Fig. 3

Visualization of CNN-SASM prediction results and the correlation with pathologic characteristics. A The malignant and benign patches with overlaid heatmaps, indicated areas of high attention as identified by Grad-CAM. B–C WSIs from TCGA patients with and without postoperative recurrence, including visual differentiation between high- and low-risk patches. In B, annotations are color-coded: green for tumor tissue, pink for the collagenous stroma, yellow for adjacent liver tissue, red for blood vessels, and blue for lymphoid tissue. C uses a similar color scheme: green for tumor tissue, yellow for adjacent liver tissue, blue for lymphocytes, and red for blood vessels (four diagrams with a gradation of thresholds define high and low risk, becoming increasingly strict from left to right and top to bottom). TCL, threshold cutoff level for high-risk and low-risk tissues

Additionally, we displayed heatmaps of different denoised recurrence risk levels using the output of CNN-SASM to explore the connections between PPS and biological tissues. Compared to the non-recurrence WSI, there was a larger area of high risk in the recrudescent WSI, and almost all high-risk regions under the extreme threshold were distributed along the tumor edges or junctions. In contrast, low-risk regions resembled lymphocyte-rich regions (Fig. 3B, C). These suggested that the interface between tumor tissue, adjacent tissues, and lymphoid tissue regions were potential prognostic indicators. Further magnified the tissues, the high-risk tissues contained dispersed lymphocytes among collagenous fibrous tissue and remnants of small bile duct structures. The low-risk tissues primarily featured densely arranged cancer cells in cord-like structures with abundant cytoplasm and numerous lymphocytes infiltrating the interstitial space, potentially accounting for the favorable prognosis associated with this region.

The PPS-based prognostic model predicts outcomes of HCC patients

Based on qualitative and quantitative evaluations, PPS had been identified to be reliable and could represent prognostic relevant pathological signatures. We then developed a prognostic model to predict the postoperative recurrence of HCC patients using PPS at different time points. In the internal test set, the prognostic model achieved a favorable performance AUROC of 0.818, 0.811, and 0.709 at 1-year, 2-year, and final recurrence prediction tasks, respectively (Fig. 4A). We then divided patients of the test set into two groups, which were predicted non-recurrence and recurrence, by comparing their predicted risk scores with the medium value of predicted recurrence risk score in the training set to investigate the association between patients’ PFS and predicted results. We found that, except for the classification ability, the correlation analysis illustrated that the predicted results were also associated with patients’ PFS. Specifically, patients predicted recurrence had significantly worse PFS than those predicted non-recurrence (P < 0.0001 for all three prognostic models) (Fig. 4B). The regression analysis results also showed that patients with higher recurrence risk scores had worse PFS (Additional file 1: Fig. S8B). We then conducted the survival analysis of patients in the test set, with the additional endpoint events defined as the recurrence status at different time points. Differences in PFS at 1-year, 2-year, and final time points were observed significantly between patients with predicted non-recurrence and recurrence groups (Fig. 4C), indicating the PPS-based prognostic model could discern high recurrence risk patients from all HCC patients by pathological images.

Fig. 4

Performance of the prognostic models in the internal test set. A The ROC curves of 1-year, 2-year, and final prognostic models. B The significance tests between patients’ PFS and the prediction results of 1-year, 2-year, and final prognostic models. The binary classification results were derived by comparing the predicted risk scores of individuals in the test set with the medium value of predicted risk scores in the training set. C The survival curves of patients in the non-treatment test set. The endpoints were defined as the recurrence status at 1-year, 2-year, and final time points, and the log-rank test was used to assess the significance level between the two groups (from left to right panels are 1-year, 2-year, and final prognostic models, respectively)

We validated the generalizability of the PPS-based prognostic model in the external dataset. To ensure the reliability of outcomes, we excluded patients with less than 2 years of follow-up data and applied 1-year and 2-year prognostic models to validate. The AUROC at 1 and 2 years after surgery were 0.713 and 0.707 (Fig. 5A), and the corresponding AUPR were 0.785 and 0.776 (Additional file 1: Fig. S9), suggesting the models had good generalizability and binary classification ability on recurrence. Furthermore, we also divided the external patients into predicted recurrence and non-recurrence groups according to the medium value of risk scores in the training set. The correlation analyses between patients’ PFS and the predicted results were both significant (Fig. 5B). Patients who predicted recurrence tended to have a worse PFS compared to those who predicted non-recurrence. In addition, similar results were also observed that there were significant survival differences between the two groups on the external test set, and the P values of 1-year and 2-year models were 0.003 and 0.0014, respectively (Fig. 5C). In conclusion, these results showed that the PPS-based prognostic models could predict the outcomes of HCC patients at 1 and 2 years after surgery in both the internal and external test sets, exhibiting the powerful predictable ability of tumor pathological signatures.

Fig. 5

Performance of the prognostic model in the external test set TCGA-LIHC. A The ROC curves of the 1-year and 2-year prognostic model. B The significance tests between patients’ PFS and the prediction results of 1-year and 2-year prognostic models. Patients were divided into predicted recurrence and non-recurrence groups by comparing their risk score to the medium value of the training set. C The survival curves of patients at 1-year and 2-year time points in the non-treatment TCGA set, and the log-rank test was used to assess the significance level of the difference between the two groups (from left to right panels are 1-year and 2-year prognostic models, respectively)

To prove it is more effective to extract PPS by DRL rather than real-world recurrence labels, we compared the performance of our method and the end-to-end method under the same configuration (Table 3). Our method exceeded the end-to-end one in almost all evaluation metrics at all time points, with an AUROC increase of up to 7.1%, 3.7%, and 5.5% at different time points, suggesting denoising the recurrence label will make the model behave better in prognostic tasks.

Table 3 Comparison of our method and end-to-end strategy

The PPS-based recurrence risk outperforms clinical risk indicators

We compared the discrimination performance of the PPS-based prognostic model with traditional clinical risk indicators in terms of AUROC and C-index for prognosis prediction. Our PPS-based recurrence risk at 1 year outperformed all clinical indicators, including HBsAg, ALT, AFP, tumor size, BCLC stage, and vascular invasion, even the comprehensive logistic regression model, which incorporated all clinical risk indicators and the integrated model (Fig. 6A, Additional file 1: Fig. S10). The result above suggested that PPS contained powerful prognostic signatures beyond existing clinical characteristics. On the TCGA set, 1-year and 2-year recurrence risks based on PPS also had the highest AUROC among other clinical indicators, showing good generalizability (Fig. 6A).

Fig. 6

The comparison between the risk score of the PPS-based model and clinical risk factors. A The AUROC values in clinical univariable and multivariate scores in the internal test set and TCGA set. The clinical risk was computed by the logistic regression (LR) model using all clinical indicators, and the integrated risk was also derived from the LR model by clinical indicators and the PPS-based prognostic model risk score. All comparisons of the left panel were conducted based on a fivefold cross-validation strategy. The lines within the boxes represent the medium AUROC value, the bounds of the boxes represent the interquartile range (IQR), and the whiskers represent the 95% confidence intervals. B The survival difference of the vascular invasion (VI) and tumor size subgroups in the internal and external test sets. C The HR values of the risk score of the PPS-based prognostic model and clinical indicators in the internal and external test sets

Previous studies had demonstrated that the vascular invasion [32, 33] and tumor size-related signatures [34] may contribute to the prognostic outcomes in HCC patients, which was also observed in our study. The vascular invasion and tumor size had the highest predictive powers among all clinical indicators in both the internal test set and TCGA set. We then attempted to perform the survival analysis within these two variable subgroups, and all the P values were significant, demonstrating recurrence risk based on PPS could further divide patients into two groups (Fig. 6B). Cox univariate analysis PPS-based recurrence risks had higher HR than clinical indicators, reaching 7.00 (95% CI 2.07–23.70) and 4.51 (95% CI 1.61–12.62) in the internal and external set at 1 year and 4.74 (95% CI 1.61–13.93) in the external set at 2 years (Fig. 6C). In conclusion, PPS-based recurrence risk was the independent predictive indicator in prognostic tasks and could become the powerful tool in estimating patients’ survival.

The PPS-based prognostic model predicts response to sorafenib treatment

Sorafenib is recognized as the first-line target drug in HCC patients, but it is still challenging to forecast its effectiveness [35]. We then investigated the predictive ability of the PPS-based prognostic model for the sorafenib treatment response in HCC. To examine whether there were differences in the clinical backgrounds of patients in the sorafenib group, we performed significance testing between the recurrence and non-recurrence groups. The results indicated that the P values for all clinical features were greater than 0.05, suggesting no clinical indicator significantly influences the patients’ treatment response (Table 4). The significance test of clinical features between the predicted recurrence and non-recurrence in the testing set was also conducted to ensure alignment of the clinical backgrounds between the two groups (Additional file 2: Table S2). In the test set, sorafenib-treated patients were divided into predicted recurrence and non-recurrence groups by comparing their recurrence risk scores to the medium value of the training set at the 1-year time point. Patients in the 1-year recurrence subgroup were significantly associated with poor PFS, P = 0.003 (Fig. 7A), suggesting the PPS-based prognostic model could also distinguish high recurrence risk HCC patients after being treated with sorafenib.

Table 4 The baseline characteristic comparison of clinical indicators between recurrence and non-recurrence patients in the sorafenib group

Fig. 7

Performance of the PPS-based prognostic model in predicting the survival and benefits in patients treated with sorafenib. A The survival curve for sorafenib-treated patients in the test set. B The correlation test between PFS and DRS. A series of significant tests between PFS and therapy strategies in all (C), predicted recurrence (D), and predicted non-recurrence (E) patients in the test set, respectively

Aside from the prognostic model, we also investigated the predictive ability of CNN-SASM, which was trained by DRL. CNN-SASM’s output was defined as the denoised recurrence score (DRS) representing the theoretical recurrence risk. Then, we computed the correlation coefficient between the PFS of sorafenib-treated patients and the DRS and found that there was a significant negative correlation (r = − 0.66, P = 0.0058). Patients exhibited worse PFS showed the significant increasing of DRS (Fig. 7B). These results indicated that PPS-based prognostic model’s output could not only predict outcomes of HCC patients but also strongly associated with the outcomes of sorafenib-treated HCC patients, offering more specific treatment management strategy in clinical.

Additionally, we then attempted to predict patients’ survival benefits from sorafenib with PPS-based prognostic model. Patients with specific physical conditions would be treated, and such clinical practice would bring bias, so the baseline physical situations of non-treated and sorafenib-treated patients varied hugely, P = 0.017 (Fig. 7C). However, for patients predicted with recurrence at the 1-year time point, the PFS between non-treated and sorafenib-treated patients had no significant difference (Fig. 7D), which meant patients in predicted recurrence subgroup would decrease the PFS gap between two treatments because of the benefit from sorafenib, while the PFS of patients predicted with non-recurrence remained the significant difference between two treatments (Fig. 7E), demonstrating patients under this restrictions got few benefits from sorafenib. In this way, we can screen patients who tend to respond well to sorafenib after surgery and improve the medication efficacy in clinical.

Discussion

Accumulating studies have demonstrated that digital pathological images could contribute to predicting patient prognosis based on deep learning [36, 37]. In this work, we developed a pathology-based deep learning model that can adaptively extract as well as decode the prognostic-related features and facilitate the prediction of HCC patients’ recurrence condition. The predictive ability of PPS-based prognostic model was superior to all conventional clinicopathological factors and could stratify patients with significantly different prognoses in HCC. Additionally, we explored the PPS-based prognostic model’s sorafenib response predictive ability and demonstrated that it could identify patients who were more likely to benefit from sorafenib, which would offer help in clinical medication management.

It is challenging to extract new prognostic features from pathological images and train a deep neural network in image analysis. The conventional attention layers in CNN have channel and spatial location dimensions [38]; they can directly compute weights from sequences and labels by gradient renewal but fail to consider connections between sequences. The self-attention mechanism [39] can address this issue by computing attention scores of sequences. Herein, we successfully applied the architecture of CNN-SASM incorporated the integration of multi-head self-attention mechanisms across two distinct dimensions: channel and spatial location. This design contemplates the interconnections between sequences within two dimensions and can offer the flexibility to be independently integrated into any framework. This design could obtain substantial and stable classification performance even though we employed the simple architecture ResNet18. In our study, the softmax layer in CNN-SASM functioned as a conventional attention layer, aiming to bestow more weights on particular channels or spatial locations without extra parameters. As the sample size grew and noise increased, the softmax struggled to discern vital channels and spatial locations alone. Thus, CNN-SASM can improve the classification performance with limited parameter scale, making it suitable as a foundational framework for small dataset.

Recently, some novel pathological features have been reported in HCC, such as tumor necrosis [40], inflammatory score [41], and immune-related biomarkers [42]. Deep learning models based on pathological features face challenges in predicting real-world recurrence due to complicated influencing factors. For the first time, we proposed utilizing DRL under different thresholds to train pathological models, and it can extract biologically interpretable pathological signatures maximumly, PPS, by ignoring irrelevant noised labels, which can be visualized on heatmaps. Compared with the end-to-end strategy, training the deep learning model by predictable labels would be more effective than the noising labels (Table 3). It may provide a new paradigm for complicated tasks involved in intricate influencing factors.

Considering the comprehensive performance of the pathological model, we traded off the classification ability and generalizability in the multiple subsets training process to determine the best-performing CNN-SASM. Despite displaying good generalization on the 3/4 subset, CNN-SASM was in a fluctuating phase. We attributed the fluctuation phase to the quality variability of added data and the rare sample size of the 14 TCGA cases used for the generalization test. Ultimately, we selected the 1/2 subset CNN-SASM to guarantee the robustness of the model. The risk area visualization on WSI shows that CNN-SASM effectively discriminated against DR and DRF. The high-risk areas were primarily located in the tumor’s peripheral tissues, indicating the potential relevance of the surrounding tissue to the patient’s prognosis. This observation aligns with Zhu et al.’s findings [43].

A growing number of studies has substantiated that the morphological characteristics observable in pathological images are indicative of the prognostic outcomes for patients in various cancers, including breast [44], gastric [45], cervical cancers [46], and HCC. However, either the methodologies and models employed are more complicated, or their efficacy in prognostic classification necessitates further enhancement [24, 47]. In our study, the approach of denoising recurrence labels facilitated the employment of less complex network architectures for training. Moreover, our AUROC for prognosis classification achieved a value exceeding 0.8 and further exhibited robust generalization using TCGA HCC data. Notably, the profound signature markers, identified as PPS in our study, exhibited superior performance relative to clinical risk indicators across all evaluation metrics, encompassing C-index, AUROC, and HR. The proposed model based on PPS was even better than the integrated multimodal model which combined PPS and clinical risk factors, suggesting PPS might already incorporate clinical characteristics from pathological images. The prognostic model based on PPS can greatly help doctors focus on the postoperative adverse reaction population so that preventive measures can be taken in advance to improve patients’ prognostic survival.

Although substantial progress has been made in the imaging-based deep learning model for cancer detection and diagnosis, there is limited success in the prediction of the response to specific treatments of cancer patients. Among the limited studies, the predominant focus has been on utilizing more accessible data modalities, including CT or MRI [48], or in other carcinomas, such as lung [49] and ovarian [50] cancers. To our knowledge, this study is the initial application to predict patient response to targeted therapy in HCC based on histopathological images. According to our results, PPS-based prognostic model could further predict sorafenib treatment response, manifested in further dividing sorafenib-treated patients into high and low risks associated with postoperative survival situations. In addition, DRS, the output of the CNN-SASM, was also significantly related to sorafenib-treated patients’ PFS. Our results indicate a significant predictive effect of the PPS-based prognostic model for prognosis and the benefit of target therapy in HCC. Patients who predicted recurrence by the PPS-based prognostic model were more likely to benefit from sorafenib than non-recurrence patients. In this way, it would be useful to provide supplemental information for better selection of therapeutic strategies and avoid unnecessary medication.

This study has several limitations. First, we only employed ResNet18 architecture to develop CNN-SASM although CNN-SASM yielded improved results and showed better generalization. In addition, the results would be more convincing if prospective multicenter and large-sample research is conducted.

Conclusions

Overall, our study proposed the CNN-SASM, a pathological model integrating CNN with channel and spatial self-attention mechanisms to predict recurrence condition in HCC patients and sorafenib survival benefits. Furthermore, the proposed concept that uses DRL as training labels rather than real-world recurrence labels to develop pathological models is broadly applicable to prognostic tasks.