Explaining Recommendations: Fidelity versus Interpretability Derek Bridge Insight Centre for Data Analytics University College Cork, Ireland
Overview • Recommender Systems • Explaining Recommendations • Case Studies • Concluding Remarks
RECOMMENDER SYSTEMS
What is a Recommender System? • Software that helps • Recommendations must users discover typically be – new music and other – relevant to the user media (‘ personalized ’) and the context-of-use – cultural artefacts such as (‘ contextualized ’) works of art and – diverse architecture – products and services – serendipitous – travel experiences – … – … Photo by Nickolai Kashirin (CC by 2.0)
A Scenario A hungry academic …. …receives a recommendation for a place-to-eat but …not within walking distance …a fusion-style cuisine with which the academic is unfamiliar. …Her confidence in the recommendation might be improved by an explanation…
A Scenario A hungry academic …. …receives a recommendation for a place-to-eat but …not within walking distance high cost …a fusion-style cuisine with which the academic is high uncertainty unfamiliar. …Her confidence in the recommendation might be improved by an explanation…
Types of Recommender System • Content-based • Collaborative – User-based nearest-neighbours – Item-based nearest-neighbours – Matrix factorization Build Model Training set
Types of Recommender System • Content-based • Collaborative – User-based nearest-neighbours – Item-based nearest-neighbours User & Context-of-use – Matrix factorization Build Model Training set
Types of Recommender System • Content-based • Collaborative – User-based nearest-neighbours – Item-based nearest-neighbours User & Context-of-use – Matrix factorization Build Model Training set Recommendation
Content-Based Crime, drama Action, sci-fi Adventure, drama, fantasy Comedy, drama, romance Western
User-Based Nearest-Neighbours 5 4 4 3 4 5 2 5 3 3 2 5 4 1 5
User-Based Nearest-Neighbours 5 4 4 3 4 5 2 5 3 User-user similarity 3 2 5 4 1 5
Item-Based Nearest-Neighbours 5 4 4 3 4 5 2 5 3 3 2 5 4 1 5
Item-Based Nearest-Neighbours 5 4 4 3 4 5 2 5 3 3 2 5 4 1 5 Item-item similarity
Matrix Factorization 𝑜 movies 𝑛 users
Matrix Factorization 𝑜 movies 𝑜 movies 𝑔 latent factors × 𝑔 latent factors ≈ 𝑛 users 𝑛 users
Matrix Factorization 𝑜 movies 𝑜 movies 𝑔 latent factors × 𝑔 latent factors ≈ 𝑛 users 𝑛 users
Ever More Complex Models Hybrids and Ensembles Multi- Objective Systems Latent Deep Feature Models Spaces
Interpretable Models • Intelligible global • Challenges descriptions of systems – preserving accuracy – E.g. decision trees – intelligibility, e.g. when there are many features or No highly-engineered features High Yes Humidity – protecting Intellectual Normal Sunny Property Outlook Yes Overcast • Interpretable deep Rain Strong Wind No models Weak Yes – learn to associate semantic feature with nodes in – E.g. linear models (esp. hidden layers sparse linear models)
DARPA’s XAI Initiative https://www.darpa.mil/program/explainable-artificial-intelligence
EXPLAINING RECOMMENDATIONS
Explanations are Relational
Explanations are Relational • Recommendation only – “You might like Never Let Me Go ”
Explanations are Relational • Recommendation only – “You might like Never Let Me Go ” • Recommendation plus description – “You might like Never Let Me Go , a 2010 dystopian drama based on the 2005 novel of the same name…”
Explanations are Relational • Recommendation only – “You might like Never Let Me Go ” • Recommendation plus description – “You might like Never Let Me Go , a 2010 dystopian drama based on the 2005 novel of the same name…” • Recommendation plus explanation – “You liked Atonement , so you might also like Never Let Me Go ” User & Context-of-use
Intermediaries in Explanations Items User Recommendation Users Features [Vig et al., 2009]
Intermediaries in Explanations likes Items User Recommendation Users Features [Vig et al., 2009]
Intermediaries in Explanations likes are similar to Items User Recommendation Users Features [Vig et al., 2009]
Intermediaries in Explanations likes are similar to Items is similar to User Recommendation Users Features [Vig et al., 2009]
Intermediaries in Explanations likes are similar to Items is similar to who like User Recommendation Users Features [Vig et al., 2009]
Intermediaries in Explanations likes are similar to Items is similar to who like User Recommendation Users Features likes [Vig et al., 2009]
Intermediaries in Explanations likes are similar to Items is similar to who like User Recommendation Users Features likes are present in [Vig et al., 2009]
Explanation Dimensions Interpretable Ethical Actionable Explanations Sound and Cheap-to- Complete compute (Fidelity)
Fidelity Soundness Completeness How truthful each The extent to element in an which an Fidelity explanation is with explanation respect to the describes all of the underlying system underlying system [Kulesza et al., 2013]
Fidelity Soundness Completeness How truthful each The extent to element in an which an Fidelity explanation is with explanation respect to the describes all of the underlying system underlying system increasing trust, fewer requests for clarification, better understanding [Kulesza et al., 2013]
White-Box Explanations Build Model Training set
White-Box Explanations User & Context-of-use Build Model Training set
White-Box Explanations User & Context-of-use Build Model Training set Recommendation + “trace” data
White-Box Explanations User & Context-of-use Build Model Training set Recommendation + “trace” data Explanation Generation Recommendation + Explanation
White-Box Explanations Sound explanations User & Context-of-use Build Model Training set Recommendation + “trace” data Explanation Generation Recommendation + Explanation
Black-Box Explanations Build Model Training set
Black-Box Explanations User & Context-of-use Build Model Training set
Black-Box Explanations User & Context-of-use Build Model Training set Recommendation
Black-Box Explanations User & Context-of-use Build Model Training set Recommendation Explanation Generation Recommendation + Explanation
Black-Box Explanations User & Context-of-use Build Model Training set Recommendation Explanation Generation Recommendation + Explanation
Black-Box Explanations User & Context-of-use Build Model Training set Recommendation Explanation Other data Generation Recommendation + Explanation
Black-Box Explanations User & Context-of-use Build Model Training set Queries Recommendation Explanation Other data Generation Recommendation + Explanation
Black-Box Explanations Model-agnostic, User & probably not sound Context-of-use Build Model Training set Queries Recommendation Explanation Other data Generation Recommendation + Explanation
Why Explain? Scrutability Trust Decision- Persuasion support
CASE STUDIES
CASE STUDY A White-Box Explanations of User-Based Nearest-Neighbours Recommendations
Explaining User-Based Nearest Neighbours Recommendations is similar to who like User Recommendation Users • Difficulties – Often 50+ neighbours – Userids are meaningless: strangers! – Profiles are large and private
Explaining User-Based Nearest Neighbours Recommendations is similar to who like User Recommendation Users • Difficulties – Often 50+ neighbours – Userids are meaningless: strangers! – Profiles are large and private – Good at persuading [Herlocker et al, 2000] – Less good for trust and decision-support [Bilgic & Mooney, 2005]
Explaining User-Based Nearest Neighbours Recommendations User & Context-of-use Build Model Training set Recommendation + neighbours Explanation Generation Recommendation + Explanation
CASE STUDY B Item-Based Explanations for User-Based Nearest-Neighbours Recommendations
Item-Based Explanations likes are similar to User Recommendation Items • Good for trust and decision-support [Bilgic & Mooney, 2005] • Familiar, e.g. Amazon: – “Customers who bought Atonement also bought Never Let Me Go ”
Item-Based Explanations for User-Based Recommendations is similar to who like User Recommendation Users
Item-Based Explanations for User-Based Recommendations Explanation • The user’s partners neighbours • The movies the Candidate user has in common with items her partners • Rules that link Association candidates to the rules recommended item [Bridge & Dunleavy, 2014; Kaminskas, Durão & Bridge, 2017]
Recommend
More recommend
