Deep learning-based implicit feedback recommendation

Xin, Xin (2021) Deep learning-based implicit feedback recommendation. PhD thesis, University of Glasgow.

Abstract

Recommender systems are of vital importance, in the era of the Web, to address the problem of information overload. It can benefit both users by recommending personalized interesting items and service providers by increasing their site traffic. Plenty of use cases have emerged as applied recommender systems, including but not limited to multimedia recommendation (e.g., news, movies, music, and videos) and e-commerce recommendation. A recommendation agent can be trained from user-item interaction data which can be categorized as explicit feedback and implicit feedback. Compared with explicit ratings which depict the user preference explicitly, implicit feedback data like clicks, purchases, and dwell time is more prevalent in the real-world scenario. On the other hand, deep learning has achieved great success recently due to the high model expressiveness and fidelity. In this thesis, we investigate deep learning techniques for recommendation from implicit feedback data. We focus on two learning perspectives: deep supervised learning and deep reinforcement learning.

Supervised learning tries to infer knowledge from implicit historical interactions. From this perspective, two models namely Convolutional Factorization Machines (CFM) and Relational Collaborative Filtering (RCF) are proposed. CFM tackles the implicit user-item interactions with side information as feature vectors and utilizes convolutional neural networks to learn high-order interaction signals. RCF considers multiple item relations into the recommendation model and tackles the implicit feedback as relation-enriched data. The two models investigate deep learning techniques for recommendation by tackling the data as two different structures: feature vectors and relations. Experimental results demonstrate that the proposed deep learning models are effective to improve the recommendation accuracy. Besides, RCF also helps to provide explainable recommendation and get a better comprehension of user behaviors.

Reinforcement learning is reward-driven and focuses on long-term optimization in a whole interaction session, which conforms more with the objective of recommender systems. From this perspective, we first formulate the next-item recommendation task from implicit feedback data as a Markov Decision Process (MDP). Then we analyzed that directly utilizing reinforcement learning algorithms for recommendation is infeasible due to the challenge of pure off-policy setting and lack of negative reward signals. To address the problems, we proposed Self-Supervised Q-learning (SQN) and Self-Supervised Actor-Critic (SAC). The key insight is to combine reinforcement learning with supervised learning and perform knowledge transfer between the two components. Based on SQN and SAC, we further proposed Self-Supervised Negative Q-learning (SNQN) and Self-Supervised Advantage Actor-Critic (SA2C) to introduce the negative sampling strategy to enhance the learning of the reinforcement component. Experimental results demonstrate that the proposed learning frameworks are effective when integrated with different existing base models. Moreover, we show that combining supervised learning and reinforcement learning is a promising direction for future recommender systems. In that case, reinforcement learning introduces reward-driven objectives and long-term optimization perspectives into supervised learning while supervised learning helps to improve the data efficiency for reinforcement learning.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Keywords: Recommender system, implicit feedback, deep learning, reinforcement learning, recommendation.
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• Colleges/Schools: College of Science and Engineering > School of Computing Science
• Supervisor's Name: Joemon, Prof. Jose
• Date of Award: 2021
• Depositing User: Dr. Xin Xin
• Unique ID: glathesis:2021-82109
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 07 Apr 2021 08:38
• Last Modified: 07 Apr 2021 10:03
• Thesis DOI: 10.5525/gla.thesis.82109
• URI: https://theses.gla.ac.uk/id/eprint/82109