The intention with this blog post is to dig a little bit into what strategies recommendation engines use, without going too deep into the math behind it. By reading this post, you will gain insights on a selection of the strategies that can be used and is used on both free and commercial recommendation engines. The hope is that this gives a better background and will help you in your effort to choose the right strategy. Most importantly, it should help you consider what dimensions that are special to your business and that can give your recommendations a little edge for your business.
BACKGROUND AND BUSINESS VALUE
As a user I expect to get relevant recommendations. Sounds easy, but not always so. To create relevance is the key here, and when it comes to recommendation engines there has been a huge evolvement since Amazon set the standard for product recommendations in 1998. Various sources state that up to 35% of the sales on amazon is generated by their recommendations [1].
To the business, recommendation engines should absolutely generate better conversions and be visible on the bottom line, and if not – something is seriously wrong.
THE BASICS
Recommendation engines are roughly speaking based on two types of data:
- Basic information: Meta data, product information, product categories, tags, keywords, prices etc.
- Interactions: Orders, ratings, add to cart, product views etc.
A very basic technique is to look at what categories, products etc. other users look at:
- Create statistics of all product details page views, category views, orders per user.
- Register the relationship between users, orders and products.
- The binding important element is, of course, the user – so we can interlink all things one customer does with others.
With the above we can answer the following questions, based on statistics, for example:
When user A is viewing a product X:
- Give me the top 3 products purchased or viewed by other users that also purchased or viewed product X.
- Give me top 3 products that user A previously purchased.
MORE STATISTICS
More statistics and dimensions can be added, no limits really, based on all we know about the users and the orders – to mention some more examples:
- Location – zip code, geo coordinates
- User's gender
- User's age
- Other user meta data and preferences, use of devices etc.
- Time of day of the order
- Order size
- Promotions and discounts on the order
- Product Categories or the product and order.
Simple statistics like the above might be effective – and can be extended on a wide range of parameters to filter and segment. The above strategy has one drawback though – with a lot of products and a lot of users the computations can be heavy.
Note: Besides using all this data for recommendations, it’s a good source for insight to your customer base and candidates for segmentation and persona inspiration and validation.
COLLABORATIVE FILTERING
The filtering-oriented strategies are normally based on user-item interactions. The fundamental assumption is that if User 1 likes Product A, and User 2 likes Product A and Product B, then User 1 likes Product B.
When Amazon started recommendations, they used item-based collaborative filtering, but they turned that around moving it from being user-based (looking at what other users also purchased) and toward an item-based collaborative filtering looking at the user's similarities and the product's similarities instead.
This award winning (and business winning) strategy is based on defining a measurement for “similarity” on the products and users. Using a similarity measure we end up have a simple way to identify and show “similar products”. This demands less computation and is a fast strategy that works better with a large product catalog. Remember that the similarity measure can be used in conjunction with other recommendation strategies.
PREDICTIONS AND MACHINE LEARNING
YouTube, Netflix etc. – they all use collaborative filtering for their recommendations. The challenge with the collaborative filtering techniques is basically that they require some history and some data. You could call it a cold start if you introduce new products and base your recommendations on reviews that doesn’t exist yet – the same for new users that you do not have any interactions with yet.
Some obvious mitigation techniques are to try to get at least some information from a first-time user like: where do you live, gender, age, interests etc. – just something! So, we can avoid the cold start risking that the user gets bored with the questions and leave.
WHAT ABOUT MACHINE LEARNING?
Collaborative filtering is obvious for machine learning, and any machine learning guy will have their favorite algorithm for that. This can also be done without machine learning though, but the ML approach adds the predictive aspect to it. I am not here to say what works the best – the users must decide that. They are the real judges.
WHAT'S THE BEST STRATEGY?
Measure and evaluate! As with everything else, one can do a lot of assumptions, but the only things that count are the voice and actions of your users. So, let’s ask them! Use the statistics, questionnaires, A/B testing and what you have in the tool box. Technology is interesting, but if it doesn’t solve the problem – it’s useless.
SUMMARY
A simple and brief overview of selected and simple recommendation engine strategies have been presented. The hope is that it inspires you and helps you move your business forward in that regards.
Hopefully this also helps you keeping things simple, focus on the business and not technology and let your users/customers have the final vote!
NEXT STEPS
In a series of more technical posts we will dive in to implementation of recommendations using different strategies and show how to measure and evaluate them. Stay tuned.
If you have any questions on recommendations engines in specific, or anything else, please reach out to us.
ROLAND VILLEMOES
CTO
