A novel recommender system using light graph convolutional network and personalized knowledge-aware attention sub-network (2025)

In this section an overview of the LGKAT and its stages are presented.

Model overview

As can be seen in Figure 1. the LGKAT retains a hierarchical neural network including two sub-networks for processing user-item and knowledge graphs and a prediction layer for integrating the results of these two previous sub-networks. The user-item graph serves as input for the Light Graph Convolution sub-network, while the knowledge graph feeds into the Personalized Knowledge Graph Attention sub-network.

Overview of the proposed LGKAT is summarized in the algorithm 1. The LightGCN’s incorporation aims to enrich the overall performance of the recommender system by enhancing the collaborative signal modulation. By leveraging the power of LightGCN, the proposed method aspires to provide users with more precise, personalized recommendations, thus enhancing user experience and system performance. While the attention sub-network incorporation aims to increase the impact of related entities and relationships to provide better recommendations.

The integration of LightGCN and personalized knowledge-aware attention in LGKAT can be theoretically justified through the lens of representation learning and the principle of multi-view learning.

In graph-based recommender systems, the objective is to learn user and item embeddings that preserve structural and semantic relationships in a shared latent space. LightGCN excels at modeling collaborative signals from the user-item interaction graph. However, it primarily captures topological proximities and lacks semantic interpretability, especially in sparse or cold-start scenarios. The knowledge-aware attention mechanism complements this by incorporating domain-specific semantics through weighted aggregation of neighboring entities in the knowledge graph, tailored to each user’s profile.

Theoretically, this design leverages complementary information sources:

• Structural signals from LightGCN capture explicit behavioral patterns.

• Semantic signals from KG attention encode personalized contextual preferences.

By combining these views, the model forms richer and more robust embeddings that bridge both behavior and semantics. According to representation learning theory, such a fusion improves generalization and representation quality, especially when one view (e.g., interaction data) is sparse or noisy.

Hence, the performance gains observed in LGKAT are not only empirical but also supported by well-established theoretical foundations in multi-source representation learning.

Data preprocessing

The data preprocessing consists of three main steps:

1. 1.

   Mapping items to entities: First, the file mapping item IDs to entity IDs is read, and dictionaries are created to map old item indices and entities to new indices.

2. 2.

   Converting ratings: The user rating file is read, and items are categorized into positive and negative ratings. Then, the data is converted into a final format suitable for recommender systems.

3. 3.

   Converting the knowledge graph: The knowledge graph data, consisting of entities and relations, is read and transformed into a format that can be used in graph-based models.

These steps are designed to prepare raw data and transform it into formats suitable for further processing.

Embedding layer

Like previous studies, we employ an embedding lookup layer to convert one-hot representations of users, items, entities and relations into compact, low-dimensional vectors^6,30.

Personalized knowledge graph attention sub-network

The central element of the proposed method is the personalized knowledge-aware attention sub-network.

Let’s consider a quartet consisting of a user (u), a target entity (\({e}_{i}\)), a relation (\({r}_{ij}\),) and a neighboring entity (\({e}_{j}\)). The model utilizes the embeddings of this quartet (u, \({e}_{i}\), \({r}_{ij}\), \({e}_{j}\)) as inputs to acquire and master the attention score.

In this context, f (·) represents the attention function and \({x}_{u}\), \({x}_{{e}_{i}}\), \({x}_{{r}_{ij}}\) and \({x}_{{e}_{j}}\) denote the initial embeddings of the user, target entity, relation and neighboring entity, respectively depicted in Fig.2. We investigate various approaches to implement the attention function f (·) to learn the attention score depicted. Details regarding these methods can be found in "Personalized knowledge graph attention sub-network". These approaches expand upon the attention model initially introduced in KGCN¹⁶.

Attention scores play a pivotal role. Once the attention scores for any of the entities are acquired, aggregating embeddings is performed depicted in Fig.3. When we talk about the first aggregation layer, the entity embedding, represented as \({e}_{i}\), in relation to user u, is calculated using the Eq.(4).

Here, to ensure the entity’s embedding is considered, \({({a}_{u})}_{ii}\) is given a value of 1. The activation function, represented as σ (.), is chosen to be LeakyReLU and W is the weight vector.

To delve into more complex connectivity patterns within the knowledge graph (KG), we stack another layer, \({L}_{2}\), of embedding aggregations. For a given user u, the entity embeddings in the first layer are derived from Eq.(5).

In this context, \({A}_{u}\) is a weighted adjacency matrix that’s learned via proposed attention-focused model that’s tailored to an individual’s knowledge. The term \({({a}_{u})}_{ij}\) represents an entry in \({A}_{u}\) at the position (i,j). The final entity representations concerning user u are denoted as \({H}_{u}={H}_{u}^{({l}_{2})}\)

For efficiency’s sake, the neighborhood size for every entity is maintained at T from the initial neighbor set; uniform sampling is done for embedding aggregation. This follows the methods previously proposed by^16,17.

An important advantage of the personalized knowledge-aware attention sub-network in LGKAT is its capacity to provide interpretability for recommendations. Unlike traditional black-box approaches, our attention mechanism assigns explicit weights to entities and relations in the knowledge graph, reflecting their importance in generating the final recommendation.

For instance, if a user frequently interacts with science fiction movies, and entities such as “Sci-Fi” or “Christopher Nolan” receive higher attention scores, this indicates their strong influence on the recommended items. These attention weights can be extracted and visualized to explain why a particular item was recommended, offering a transparent reasoning path.

This interpretability is particularly valuable in real-world applications such as e-commerce or educational platforms, where users or stakeholders often require justifiable explanations for personalized suggestions.

Searching for the best attention score function

In this paper we propose four attention score functions which are Bi_Interaction attention, Product attention, Bi_Perceptron attention and Concat attention. The obtained results are presented in "Model training and loss evaluation".

Bi_Interaction attention:

It introduces a new component to explore feature interactions among the tetrad (\({x}_{u},{x}_{{r}_{ij}},{x}_{{e}_{i}},{x}_{{e}_{j}}\)) as Eq.(6)

This proposed attention score function uses the following two operator of embedding combinations:

Summation: This operator allows the model to easily aggregate diverse information.

Element-wise multiplication: This operator emphasizes the interactions between embeddings and can generate new features based on various combinations.

These techniques enable the model to understand complex relationships between users and items. By focusing on significant and relevant features, they can enhance the accuracy of predictions. Furthermore, these methods help the model perform effectively across different scenarios and with new data, facilitating better interaction with complex datasets.

Product attention: features of the target and relation entities element-wise product then they concatenate with user and neighbor entity as Eq.(7).

This proposed attention score function is employed to integrate item embeddings, neighbor embeddings, and user relation embeddings. This combination assists the model in aggregating diverse and relevant information into a unified representation, enabling it to comprehend complex relationships between users and items. Furthermore, by gathering information from various sources and focusing on significant features, the accuracy of predictions is significantly enhanced.

Bi_ Perceptron attention: The embeddings of user and relation entities are concatenated and passed through a single-layer perceptron. Similarly, the embeddings of target and neighbor entities are concatenated and passed through another single-layer perceptron. To compute the attention score, sum these two as Eq.(8).

The proposed method leverages an attention mechanism to dynamically assign weights to user-relation and target-neighbor interactions, allowing the model to focus on the most relevant parts of the graph. By combining these interactions, it captures complex relationships within the data, enhancing the recommendation quality. Additionally, the use of a multi-layer perceptron helps the model learn non-linear patterns, further improving its generalization and predictive performance in recommendation tasks.

Concat attention: it concatenates the embeddings of the tetrad (\({x}_{u},{x}_{{r}_{ij}},{x}_{{e}_{i}},{x}_{{e}_{j}}\)) as Eq.(9).

where \({W}_{1}\), \({W}_{2}\) and \({W}_{3}\) represent the weight vectors, while \({b}_{1}\) and \({b}_{2}\) are the biase and ʘ is the element-wise product. This proposed attention score function captures a richer representation of the interaction space by concatenating the embeddings of users, relations, targets, and neighbors. This approach allows the system to integrate multiple types of information in a unified manner, enhancing the model’s ability to discern relevant patterns in recommendation tasks.

Lastly, a softmax function is used to normalize these attention scores. The use of softmax normalization ensures that attention scores are appropriately distributed, allowing the model to focus on the most influential neighbors and interactions.

Predicting layer

The LGKAT obtains refined user embeddings from the user-item interaction matrix (UI). Specifically, the embedding of user \({u}_{n}\) is denoted as \({e}_{{u}_{n}}= {h}_{n}\),\({h}_{n}^{\mathsf{T}}\) is the nth row of matrix H.

The refined item embeddings are a fusion those from both the UI and the Knowledge Graph (KG). Let’s denote the refined embedding of item \({i}_{m}\) concerning user \({u}_{n}\) as \({e}_{{i}_{m}}^{{u}_{n}}\). It is defined as Eq.(10).

Here, α is a parameter that helps in strike a balance between the UI and KG information and it’s set to 0.5 in our tests. The term \({\mathbf{h}}_{N+m}\) represents the (N + m)th row of H, while \({({\mathbf{h}}_{{u}_{n}})}_{m}\) is the mth row of \({H}_{{u}_{n}}\). Consequently, this item embedding effectively merges collaborative insights from the UI and attribute-centric data from the KG. Given the optimized user and item embeddings, the likelihood of an interaction between user \({u}_{n}\) and item \({i}_{m}\) is derived as Eq.(11).

In this formula, the function g (\({e}_{1},{e}_{2})=\upsigma ({\mathbf{e}}_{1}^{\mathsf{T}}{\mathbf{e}}_{2})\) refers to the sigmoid function. It’s worth noting that for examining more intricate interactions between the embeddings, we can harness neural network-based models.

The loss function is presented as Eq.(12) that uses cross-entropy.

Here, \({p}_{k}(i)\) denotes a distribution used for negative sampling. The symbol 'K' indicates the quantity of negative items that are sampled corresponding to each observed interaction between a user and an item. Notably, in our experimental evaluations, we have chosen 'K' as 1."

Experiments

In this section, datasets and evaluation metrics are introduced. Finally, the performance of the proposed LGKAT is assessed and compared with several prominent methods in the recommender systems domain. To maintain fairness and consistency in our evaluation, we adopted the same set of metrics across all techniques focusing on key performance indicators like recall and F1-score.

Datasets and hyperparameters

Four well-known benchmark datasets include Book-Crossing (Book), MovieLens-20M (Movie), Last.FM (Music) and Dianping-Food (Restaurant), the first one from¹³ and the last three from⁷ are used for evaluations. The descriptive statistics of these datasets are presented in Table 1. The hyperparameters in the LGKAT framework was chosen by a combination of empirical testing and domain-specific considerations. The model includes several key components, such as attention mechanisms, neighborhood size (T), learning rate(η), regularization weight (λ), and the number of LightGCN layers. To analyze the model’s sensitivity to these parameters, we conducted controlled experiments in which each hyperparameter was varied individually while keeping the others fixed.

Results indicated that LGKAT is particularly sensitive to the choice of attention mechanism and neighborhood size. For example, Bi-Perceptron and Product attention consistently outperformed other variants across all datasets. Increasing the neighborhood size T generally led to performance improvements up to a certain point (e.g., T = 8 for Music), beyond which performance degraded due to over-smoothing.

The learning rate η = 0.005 achieved a good balance between convergence speed and training stability across most datasets. For datasets with smaller interaction sizes, such as Book, a lower learning rate (5×10⁻⁵) ensured stable training. The regularization weight λ required careful tuning larger values caused underfitting, while overly small values led to overfitting. In the Music dataset, λ = 10⁻⁴ provided the best performance.

The number of users, items, interactions, entities, and relationships across the four datasets (Movie, Book, Music, Restaurant) reflects the diverse characteristics and complexities inherent in each dataset. The embedding size (d) was set to 32 for Movie and Music datasets and 64 for the Book dataset, to stablish a trade-off between model complexity and generalization ability. The learning rate (η), regularization weight (λ), and neighborhood size (T) were meticulously selected to enhance the model’s performance and convergence behavior. A lower learning rate for the Book dataset (5 × \({10}^{-5}\)) was chosen to ensure stability during training, given its smaller interaction size. Regularization weights were tuned to mitigate overfitting, particularly important in datasets with fewer interactions, such as Music, where λ was set to \(({10}^{-4}\)). The neighborhood size (T) was adjusted according to the dataset’s complexity, with higher values for the Book and Music datasets to capture more intricate relationships. Collectively, these parameter choices are pivotal in maximizing the model’s predictive accuracy and generalization across different contexts, thereby demonstrating their significance in practical applications.

For the Restaurant dataset, the embedding size (d) was set to 8, reflecting the relatively small item space and interaction structure. The learning rate (η) was configured to (2 × \({10}^{-2}\)), consistent with the Movie dataset, to ensure effective learning while accommodating the large user count and high interaction density. A regularization weight (λ) of \(({10}^{-7}\)) was chosen to balance overfitting risks while maintaining generalization, given the dataset’s extensive interaction volume. The neighborhood size (T) was set to 4, similar to the Movie dataset, to capture local interaction patterns while maintaining computational efficiency. These parameter selections ensure the model’s robustness and scalability to the unique characteristics of the Restaurant dataset.

Evaluation metrics

Two metrics are used for evaluations which are Top-K recommendation and F1-Score. The reported results are obtained by averaging three different runs.

Top-K recommendation

The objective of top-K recommendation is to discover K items that will pique the interest of a specific user¹⁴.

In top-K recommendation, the chosen evaluation metric to assess recommendation performance is Recall@K. For a given user, the calculation of Recall@K is as Eq.(13).

For a specific user, #TP@K and #FN@K denote the number of true-positive and false-negative results, respectively, among the top-K items with the highest prediction scores. The overall Recall@K score is determined by averaging the Recall@K scores of all users.

F1-score

Evaluation of the accuracy of the proposed method done with the F1-score which is derived as equations (14).

Model training and loss evaluation

The training performance of the proposed model across four benchmark datasets Movie, Book, Music and Restaurant shows consistent improvement in loss over the epochs, indicating the effectiveness of the model in learning from diverse data sources depicted in Fig.4. On the Movie dataset, the training loss begins at 0.2339 in the first epoch and steadily decreases to 0.1467 by the 39th epoch. This demonstrates a consistent learning process, with the most significant reduction occurring in the initial epochs, where the loss drops from 0.2339 to 0.1598 by epoch 3. The model continues to refine its predictions, showing a steady improvement throughout the training phase. Similarly, on the Book dataset, the loss starts at 0.6931 and gradually decreases to 0.3937 by epoch 39. The model exhibits a rapid decrease in the early epochs, from 0.6931 to 0.6750 by epoch 4, and further improvement until it stabilizes at 0.3937 by the final epoch. This indicates that the model effectively adapts to the data, showing consistent progress throughout the training period.

In the Music dataset, the loss starts at 0.5786 and decreases to 0.2677 by the final epoch. There is a noticeable drop in loss during the early stages, from 0.5786 to 0.4622 by epoch 1, followed by a steady decline to 0.3311 by epoch 16. Similar to the other datasets, the loss reduction continues in the later epochs, stabilizing at 0.2677, reflecting the model’s effective learning on musical data. On the Restaurant dataset, the training loss starts at 0.4384 in the first epoch and steadily decreases to 0.2174 by the 39th epoch. The most significant reduction is observed in the early epochs, where the loss drops sharply from 0.4384 to 0.2868 by epoch 2 and further to 0.2459 by epoch 5.

As training progresses, the loss continues to decrease at a slower but consistent rate, stabilizing at 0.2174 by the final epoch. This trend highlights the model’s ability to learn and generalize effectively on the Restaurant dataset, even as the data becomes increasingly refined during training. In conclusion, the results across all datasets demonstrate the model’s robust learning ability and its ability to generalize effectively, showing steady improvement in performance as training progresses.

Comparison of the proposed LGKAT with the state-of-the-art Methods

In this section the proposed LGKAT is compared with the state-of-the-art methods and the effect of using different attention sub-networks in the proposed LGKAT is investigated. One of the evaluation metrics for recommendation performance is Recall. The results of Recall@K in Top-K recommendation for the top 2, 10, 50 and 100 can be found in Table 2. The metrics of Top-K recommendation are calculated for the four proposed attention score functions which are Bi_Interaction Attention, Concat Attention, BI_Perceptron Attention and Product Attention. The proposed model is better than advanced baselines in all comparisons, including path-based method PER, hybrid method RippleNet, Contrastive learning method XSimGCL, and GNN-based methods GCMC, NGCF, KGAT, KGCN and KGNN-LS, in the all Movie, Book, Music and Restaurant benchmark test datasets. Additionally, recent methods such as LightGCN and MGDCF, which are based on the graph neural network (GNN) paradigm, were included in our comparison for a more comprehensive evaluation. As shown in Table 2, although LightGCN and MGDCF demonstrate competitive performance across the datasets. The methodologies of existing recommendation systems exhibit both strengths and limitations. Traditional methods like GCMC and PER focus on modeling user-item interactions, yet they struggle to scale with large datasets and fail to capture complex, multi-relational knowledge. RippleNet improves upon this by leveraging knowledge graphs (KGs), though it remains sensitive to the quality of input data and prone to over-propagation. NGCF and KGAT incorporate graph neural networks (GNNs) and attention mechanisms, enhancing the representation of user-item relationships, but their scalability and computational demands pose challenges. Similarly, KGCN and KGNN-LS excel in local feature extraction but are limited in exploring deeper, multi-layered interactions. LightGCN and MGDCF simplify computations and emphasize multi-graph diffusion, yet they may overlook non-linear interactions and require careful tuning. XSimgcl integrates contrastive learning with graph representations to capture fine-grained similarities between entities. While it is effective in enhancing representation learning, it may suffer from suboptimal performance in sparsely connected graphs. In contrast, the proposed LGKAT model leverages advanced attention mechanisms (e.g., Product, BI_Perceptron) to effectively model multi-layer interactions within KGs, yielding superior performance across datasets. However, it demands substantial computational resources and careful hyperparameter optimization for optimal results.

The experimental results show that the proposed LGKAT with Product Attention and BI_Perceptron Attention still outperforms these newly baselines across the Movie, Book, Restaurant and Music datasets further validating LGKAT’s robustness and superior recommendation performance. Finally, we chose the proposed Product Attention for being involved in the final version of the proposed LGKAT.

In Table 3,-F1-score is calculated and compared to some well-known recommender systems. As shown by the experiment, the proposed LGKAT can significantly improve the recommendation performance in the Book and Music benchmark test datasets and competitive in Movie dataset. Moreover, it demonstrates comparable results in the Restaurant dataset compared to advanced baselines.

The proposed LGKAT shows better results than the state-of-the-art baselines including path-based method PER, hybrid method RippleNet and GNN-based methods GCMC, NGCF, KGAT, KGCN and KGNN-LS in almost comparisons. For example, the proposed model (with Product Attention) improved on average 4% in F1-scores compared to NGCF on the movie, book and music datasets. Additionally, LGKAT shows slightly better performance than the advanced baselines like LightGCN and MGDCF in the Movie and Music datasets. Specifically, LightGCN and MGDCF exhibit close results to LGKAT, but the proposed model surpasses them in Movie and Music, while maintaining competitive performance in Book. In the Restaurant dataset, XSimGCL achieves slightly better results compared to LGKAT, demonstrating the effectiveness of contrastive learning in specific domains. However, LGKAT still outperforms MGDCF and KGNN-LS in this dataset, indicating its robustness. These results suggest that the LGKAT with Product Attention provides superior recommendation performance, making it a promising alternative to other advanced models in the field.

In conclusion, the proposed LGKAT model, especially with Product Attention, not only matches but often exceeds the performance of the latest GNN-based baselines such as LightGCN and MGDCF.

This demonstrates the robustness and flexibility of the LGKAT framework in capturing complex dependencies in various datasets, making it a promising method for future recommender systems. The core strength of LGKAT lies in its novel integration of collaborative signals from the user-item graph and rich semantic features from the knowledge graph through a personalized attention sub-network. In scenarios involving complex relational semantics or rich domain-specific knowledge, this hybrid design enables LGKAT to capture deeper and more personalized patterns, leading to improved recommendations. Although this design introduces additional computational overhead, the experimental results confirm that the added complexity translates into more expressive user and item representations. For real-world applications requiring high-quality personalization (e.g., e-commerce, healthcare), even marginal performance improvements can significantly enhance user satisfaction and engagement, justifying the architectural sophistication.

By leveraging structural and semantic information from both User-Item and Knowledge Graphs, LGKAT excels across diverse datasets, including Movie, Book, Music, and even Restaurant benchmarks. The LGKAT method showcases the design advantages by utilizing structural and semantic information from the UI and KG.

One of the prominent advantages of incorporating LightGCN into the LGKAT framework is its inherent computational efficiency and scalability. LightGCN simplifies traditional GCN operations by removing non-linearities and feature transformations, allowing it to scale seamlessly to large datasets. In our implementation, we observed that even with over 23 million user-item interactions in the Restaurant dataset and more than 2 million users, the model was able to converge efficiently within a reasonable number of epochs (39), as illustrated in "Model training and loss evaluation".

Furthermore, the layer-wise propagation scheme in LightGCN avoids recursive message passing, significantly reducing the computational burden. The sparse nature of the user-item interaction matrix is also preserved during training, which reduces memory usage and enables batch processing, making LGKAT well-suited for large-scale and real-time recommendation scenarios.

To validate scalability, we conducted experiments on four datasets with diverse characteristics ranging from small-scale (Music) to highly dense and large-scale (Restaurant). The stable training behavior and consistent performance across all datasets confirm the model’s robustness and practical applicability to real-world systems where fast inference and scalability are critical.