Recommender Systems
COMPSCI 753 Kaiqi Zhao
Roadmap
§ Recap collaborative filtering and latent factor model § Context-aware recommendation
§ Session-based recommendation
48
Session-based recommendations
§ Many of the modern recommender systems are session-based.
§ A Session starts from logging to the system and ends with signing out.
Example actions of a user who added two shoes to the cart
49
Session-based recommendations
§ Many of the modern recommender systems are session-based.
§ A Session starts from logging to the system and ends with signing out.
§ Rather than the whole rating matrix, the sequence of user actions in the current session are considered (e.g., cart, browse, etc.)
Waterfall flow RS
50
Session-based recommendations
§ Next action prediction – what will be the next item the user browse or purchase?
§ If past sessions are used, it is also called session-aware recommendations Current session
Session
?
User 1
User 2
…
Example
Historical data
51
Approaches
§ Item-based approaches
§ Simple pairwise item co-occurrence
§ E.g., “users who bought this also bought” § Frequent pattern mining
§ E.g., trousers, shirts and socks are often purchased together
§ Sequence modeling
§ Factorized Personalized Markov Chains § Recurrent neural networks
52
Factorization-based methods
§ Factorized Personalized Markov Chains (Rendle, 2009)
§ Markov Chain – Transitions from sets to sets § 𝑝(𝐵8|𝐵892) — how many possible states of a set? § Model transitions between items instead of sets:
§ 𝑝(𝑙 ∈ 𝐵F|𝑖 ∈ 𝐵FG!) 𝐼 ×|𝐼| matrix! § Maximum likelihood estimation
§𝑝𝑙∈𝐵F𝑖∈𝐵FG! =##→( # {#→∗}
2M 2M×2M matrix!
53
Factorization-based methods
§ Recommendation with Markov Chain
𝑝𝑎𝑐,𝑒 =0.15
𝑝𝑏𝑐,𝑒 =0.35
𝑝𝑑𝑐,𝑒 =0.00
§ Problems
§ User independent — 𝑑 should be better considering User 3 and 4 share similar
interests.
§ Sparse transition matrices – transitions from each item (rows) is estimated independently.
54
Factorization-based methods
§ Personalized Markov Chains
§ Input as a tensor of 𝑈×𝐼×𝐼, each user has an item “transition matrix” § The transition is asymmetric
Item
Any tensor decomposition methods can be used
In the original paper, they use Canonical Decomposition for prediction
𝑟̂ = 𝑣NM,𝑣MN + 𝑣M+,𝑣+M + 𝑣N+,𝑣+N #,$,( # $ $ ( # (
User vectors: 𝑣#NM , 𝑣#N+ Current item vectors: 𝑣$MN , 𝑣$M+ Next item vectors: 𝑣(+N , 𝑣(+M
Item
55
User
Recurrent neural networks
§ RNN is a natural way to model sequence
hF=𝑔(𝑈𝑋F+ hFG!)
Share across time steps!
(Source: https://commons.wikimedia.org/wiki/File:Recurrent_neural_network_unfold.svg)
Hidden state h
The core idea behind RNN is to memorizes the current sequence information and maintains the hidden states for prediction
Intention shoes dress up dress up
Memory
56
𝑉
Gated recurrent units
§ Gated Recurrent Unit (GRU)
§ Reset gate: 𝑟â = 𝜎 𝑈M𝑋â + 𝑉Mhâä”
§ Update gate: 𝑧â = 𝜎 𝑈ã𝑋â + 𝑉ãhâä”
§ Candidate state: h`â = tanh(𝑈𝑋â + 𝑉 𝑟â ∘ hâä” ) § h â = 1 − 𝑧 â ∘ h â ä ” + 𝑧 â ∘ h` â
hF = 𝑔(𝑈𝑋F + 𝑉hFG!)
𝑟F 𝑧F
𝜎(𝑥) – sigmoid function, range [0,1]
tanh 𝑥 – hyperbolic tangent, range[-1,1]
57
RNN for recommendations
§ GRU4REC (Hidasi et al. 2016)
§ Use gated recurrent unit (GRU) as hidden unit § Use session parallel mini-batch for training
Motivation:
§ The size of the session can be very different.
§ Need to capture how the session evolve over time, i.e., should not break the sequence into segments
(Image from Hidasi https://arxiv.org/abs/1511.06939)
58
Summary
§ Context-aware recommendation § Context modeling with CF – USG
§ Tensor decomposition
§ Factorization machine
§ Session-based recommendation § Factorization-based method
§ Recurrent neural net (GRU4REC)
59
Resources
§ Python package:
§ Surprise https://surprise.readthedocs.io/en/stable/getting_started.html#basic-usage
§ Datasets:
§ RecSys Challenge 2015: https://www.kaggle.com/chadgostopp/recsys-challenge-
2015
§ Yelp Dataset Challenge: https://www.yelp.com/dataset/challenge
§ MovieLens: https://www.kaggle.com/prajitdatta/movielens-100k-dataset
§ Readings:
§ Collaborative filtering: https://lazyprogrammer.me/tutorial-on-collaborative-filtering-
§ Context-aware recommendations http://www.inf.unibz.it/~ricci/papers/08- Adomavicius.pdf
§ Session-based recommendations evaluation https://arxiv.org/pdf/1803.09587.pdf
§ GRU4REC https://arxiv.org/pdf/1511.06939.pdf
and-matrix-factorization-in-python/
60