RECOMMENDER
APPLIED ANALYTICS: FRAMEWORKS AND METHODS
SYSTEMS 2
OUTLINE
• Discuss need for recommendation systems
• Explain how recommender systems work
• Compare and contrast types of recommendation systems
• Discuss applications of recommender systems
• Utilize recommender systems to make product recommendations
2
NEED
• Consumers are drawn to stores that offer them variety but when faced with a large number of options, they get overwhelmed and postpone their purchase.
• Recommender systems solve the problem by offering relevant suggestions that makes choosing easier.
3
WHO USES THEM
• Websites (e.g., Amazon, Walmart)
• Movie and Music systems (e.g., Netflix, Spotify, Pandora) • Social Networks (e.g., Facebook, LinkedIn, Twitter)
• Consider the following
• Two-thirds of movies watched by Netflix customers are recommended movies
• 38% of click-through rates on Google News are recommended links
• 35% of sales at Amazon arise from recommended products
• ChoiceStream claims that 28% of people would like to buy more music, if they find what they like
4
HOW DOES IT WORK
5
ILLUSTRATION
COLLABORATIVE FILTERING RECOMMENDER
6
COLLABORATIVE FILTERING
7
COLLABORATIVE FILTERING PROCESS
• Load Data: Create user-item matrix. Standardize ratings.
• Calculate similarity matrix: Similarity may be measured using Euclidean distance,
cosine distance, Pearson coefficient, Jaccard distance
• Predict unknown ratings: For an unknown item, obtain ratings of n nearest neighbors who have rated it. Computed a score weighted by similarity to target user.
• Recommend: Suggest top k items.
8
LOAD DATA
9
SIMILARITY MATRIX
10
IDENTIFY MISSING RATINGS FOR TOBY
11
RATINGS BY OTHERS FOR MOVIES TOBY HAS NOT WATCHED
12
APPEND SIMILARITY SCORES
13
COMBINE RATING WITH SIMILARITY
14
PREDICTION IS THE SIMILARITY WEIGHTED AVERAGE
15
TYPES OF RECOMMENDATION SYSTEMS
• Collaborative Filtering • Content-based
• Hybrid
• Context-aware
16
COLLABORATIVE FILTERING
• Basic idea is to identify users similar to the target user and then make product recommendations for unseen products.
• User-based
• Recommendations are generated by considering the preferences in the user’s neighborhood.
1. Identify similar users based on similar user preferences
2. Recommend new items to an active user based on the rating given by similar users on the items not rated by the active user.
• Item-based
• Recommendations are generated using the neighborhood of items. Unlike user-based collaborative filtering, we first find similarities between items and then recommend non-rated items which are similar to the items the active user has rated in past.
1. Calculate the item similarity based on the item preferences
2. Find the top non-rated items similar to the rated items by active user and recommend them
17
COLLABORATIVE FILTERING
• These methods are easy to implement and very accurate. However, they suffer from the following problems
• Sparsity problem: Users have not rated/used most items
• Cold Start Problem: An extreme case of sparsity problem.
• New users do not have any ratings
• New products do not have any ratings
18
CONTENT
• Uses item properties and user preferences to the item properties while building content-based recommendation engines.
• A content-based recommender system uses the content information of the items for building the recommendation model. Contains
• a user-profile-generation step,
• item-profile-generation step- and
• model-building step to generate recommendations for an active user.
–
BASED FILTERING
19
HYBRID RECOMMENDER SYSTEM
• Works by combining various recommender systems to build a more robust system • Addresses shortcomings of individual methods
20
CONTEXT AWARE RECOMMENDATION
• A context-aware recommender system takes the context into account before computing or serving recommendations
• User preferences may differ with the context, such as
• time of day,
• season,
• mood,
• place,
• location,
• options offered by the system, and so on.
• A person at a different location at a different time with different people may need different things.
21
ADVANCED RECOMMENDATION SYSTEMS
• Scalable Recommenders:
• Mahout, a machine-learning library built on the Hadoop platform provides infrastructure to build, evaluate, and tune the different types of recommendation-engine algorithms. Since Hadoop is designed for offline batch processing, we can build offline recommender systems, which are scalable.
• Scalable real-time Recommenders
• “You may also like feature” requires generating personalized
• Apache Spark Streaming leverages scalability of big data and generates recommendations in real time, and processes data in- memory.
recommendations in real-time
• Graph-based recommender systems
• graph databases allow us to store user and product information in
graphs as nodes and edges (relations). • Neo4j, a NoSQL database
22
SUMMARY
• In this module we
• discussed need for recommendation systems and how recommender systems work
• compared and contrasted types of recommendation systems
• discussed applications of recommender systems
• utilize recommender systems to make product recommendations
23