Effective graph representation learning for ranking-based recommendation

Liu, Siwei (2023) Effective graph representation learning for ranking-based recommendation. PhD thesis, University of Glasgow.

Abstract

Ranking-based recommender systems are designed to generate a personalised ranking list of items for a given user to address the information overload problem. An effective and efficient ranking-based recommender system can benefit users by providing them with items of interest as well as service providers by increasing their exposure and profits. Since more and more users and providers of items have been increasingly interacting with online platforms, the underlying recommendation algorithms are facing more challenges. For example, traditional collaborative filtering-based recommender systems cannot generate effective recommendations to cold-start users due to the lack of sufficient interactions. In addition, although recommender systems can leverage deep learning-based techniques to enhance their effectiveness, they are not robust enough against variances in the models’ initialisations, which can degrade the users’ satisfaction. Furthermore, when incorporating these complex deep models, the training phases of recommender systems become less efficient, which might slower the online platforms from quickly capturing the users’ interests.

Graph representation learning includes techniques that can leverage graph-structured data and generate latent representations for the nodes, graphs/sub-graphs and edges between nodes. Since the user-item interaction matrix is in fact a bipartite graph, we can use these graph-based techniques to leverage the interaction matrix and generate more effective node representations for the users and items. Therefore, this thesis aims to enhance the ranking-based recommendations by proposing novel recommender systems based on graph representation learning. In particular, this thesis uses heterogeneous graph representation learning, graph pre-training and graph contrastive learning to improve the effectiveness of ranking-based recommendations while alleviating the aforementioned cold-start problem as well as the low-robustness and low training-efficiency issues.

To enhance the effectiveness of ranking-based recommendations and alleviate the cold-start problem, we propose to use the heterogeneous graph representation learning technique to encode the typical side information of the users and items, which are usually defined as the attributes of users and the descriptions of items. For example, a user-item interaction matrix, social relations are one of the most naturally available relations that can be used to enrich such an interaction matrix. Therefore, we choose the social relations among different types of side information to build the heterogeneous graph. We propose a novel recommender system, the Social-aware Gaussian Pre-trained model (SGP), which encodes the user social relations and interaction data using the heterogeneous graph representation learning technique. Next, in the subsequent fine-tuning stage, our SGP model adopts a Gaussian Mixture Model (GMM) to factorise these pre-trained embeddings for further training. Our extensive experiments on three public datasets show that SGP can alleviate the cold-start problem while also ensuring effective recommendations for regular users.

To alleviate the low-robustness issue and enhance the recommendation effectiveness, we propose to leverage multiple types of side information using the graph pre-training technique. In particular, we aim to generalise the pre-training technique used by SGP for multiple types of side information associated with both users and items. Specifically, we propose two novel pre-training schemes, namely Single-P and Multi-P, to leverage side information such as the ages and occupations of users and the textual reviews and categories of items. Instead of jointly training with two objectives, our pre-training schemes first pre-train a representation model under the users and items’ multi/single relational graphs constructed by their side information and then fine-tune their embeddings under an existing general representation-based recommendation model. Extensive experiments on three public datasets show that the graph pre-training technique can effectively enhance the effectiveness of ranking-based recommender systems and alleviates the cold-start problem. In addition, our pre-training schemes can provide more ef-fective initialisations for both the users and items; hence the robustness of fine-tuning models namely MF, NCF, NGCF and LightGCN, can be improved.

Finally, to enhance the training efficiency of graph-based recommenders while ensuring their effectiveness, we propose to use the graph contrastive learning technique to improve the traditional random negative sampling approach. In particular, we propose a dynamic negative sampling (DNS) approach that leverages the graph contrastive learning technique to replace the randomly sampled negative items with more informative negative items. Our experiments show that DNS can improve the recommendation effectiveness of four competitive recommenders. Next, we further propose a novel graph-based model, i.e. MLP-CGRec, that leverages a multiple sampling approach to enhance the training efficiency of the graph-based recommender system. In particular, MLP-CGRec uses DNS to sample contrastive negative items and an efficient graph-based sampling method to select pseudo-positive samples. Experimental results on three public datasets show that MLP-CGRec can maintain competitive effectiveness and achieve the best efficiency compared with state-of-the-art recommender systems.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• Colleges/Schools: College of Science and Engineering > School of Computing Science
• Supervisor's Name: Ounis, Professor Iadh and Macdonald, Professor Craig
• Date of Award: 2023
• Depositing User: Theses Team
• Unique ID: glathesis:2023-83382
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 27 Jan 2023 11:57
• Last Modified: 30 Jan 2023 09:49
• Thesis DOI: 10.5525/gla.thesis.83382
• URI: https://theses.gla.ac.uk/id/eprint/83382