Cognitive IoT Recipe Maven Cognitive IoT Recipe Maven Digital - - PowerPoint PPT Presentation

Miguel Jiménez, Prashanti Angara, Harshit Jain, Kirti Agarwal, Roshni Jain, Hausi Müller, Ulrike Stege, Joanna Ng IBM Canada LabWeek - CASTLE 2017 University of Victoria Miguel Jiménez, Prashanti Angara, Harshit Jain, Kirti Agarwal, Roshni Jain, Hausi Müller, Ulrike Stege, Joanna Ng IBM Canada LabWeek - CASTLE 2017 University of Victoria

Cognitive IoT Recipe Maven

Digital Expertise in the Kitchen

Cognitive IoT Recipe Maven

Digital Expertise in the Kitchen

Picture from http://foodindustryexecutive.com/2016/04/the-internet-of-things-and-the-future-of-food/

Motivation

• Food wastage is currently 40% in North America [1]
• Obesity rate is increasing globally [2]
• Limited context integration in existing applications

○ Nutrient information and dietary goals ○ Effective use of expiry dates ○ Knowledge of users’ taste, preferences, allergies and diseases

• Limited knowledge and time to cook healthy
• Sparse and duplicate data across multiple devices and applications

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3228640/ [2] https://www.hsph.harvard.edu/obesity-prevention-source/obesity-trends/obesity-rates-worldwide/

2

Outline

1. Overview of the IoT Recipe Maven 2. Key stakeholders 3. Maven components

a. CAPRecipes: context-aware personalized recipes recommender b. Foodie Fooderson: a conversational agent for the smart kitchen c. Smidge: Smart Refrigerator and Grocery d. SmartGrocer: a profit-aware store path optimizer

4. Challenges 5. Conclusion

3

Key Stakeholders

Grocery stores Fridge Makers Users

❖ Smidge ❖ SmartGrocer ❖ CAPRecipes ❖ Foodie

4

Cognitive IoT Recipe Maven

Context Sphere Web app Foodie Mobile app SmartGrocer CAPRecipes Spoonacular Grocery Store Smidge

Kitchen data Personal data Heart monitor diabetes monitor Weighing scale Fridge camera Smarter mats Food data

5

[1] https://spoonacular.com/food-api

CAPRecipes: Context-Aware Recipe Recommendation

6

Collaborative filtering Content-based filtering Recommender Personalized Recipes

Personal health information: allergies and diseases Cuisine and ingredient preferences Preferences of family and friends Browser search history and social interactions Time of day, location (general context) Expiry date, quantity and missing ingredients

Context

Recipe preferences and ratings

Ratings

+Reasoning

CAPRecipes: Hybrid interface

Mobile Application Web UI

7

Foodie: A Cognitive Kitchen Assistant

A cognitive conversational assistant connected to our recipe recommendation engine that: ○ Assists people in finding a suitable recipe ○ Describes step-by-step cooking instructions ○ Provides nutrition information ○ Helps people in creating and tracking dietary goals

I’m hungry Would you like to make Lemon Garlic Chicken? I don’t like garlic How about Grilled Cheese Pizza? Sounds good! Cuisine Ingredients 8

Sugar intake Body Weight Foodie, I’d like to reduce my intake of sugar [...] how much fat is in that recipe?

Under the Hood

• #Intents

○ Determine the purpose of arbitrary user input ○ Example: “I’m feeling hungry” is classified under the intent #start_cooking

• @Entities

○ Keyword identification ○ Example: “I want to eat a french breakfast” ■ @cuisine = french ■ @mealType = breakfast

• Dialog

○ Possible flows of a conversation via nodes ○ Nodes are triggered by conditions

• Context

○ Mechanism for passing information between the dialog and the application

Foodie: A Cognitive Kitchen Assistant (2)

9

Smidge: Smart Refrigerator and Grocery

10

Smarter Mats FridgeCam

G r

• c

e r y S t

• r

e

• Update ingredient inventory

through grocery receipts

• Personalize recommendations

for coupons to increase savings

• FridgeCam: Analyze contents
• f the fridge
• Smarter Mats: Track weights

for contents in the fridge

Cashier

Smart Grocer: Profit-Aware Store Path Optimization

Promote products by incentivizing customers:

• Guide users through the

store to purchase a list of items

• Indicate additional items in
• r near the customer’s path

that the store wants to promote or offer discounts

• n.

11 List item Promoted item Optimal Path Suggested Detour

Entry

Smart Grocer: Profit-Aware Store Path Optimization

12

Grocery List User’s preferences Context Sphere

Kitchen data Personal data

Smidge Smart Grocer

Optimal Path Optimal Path + Suggested Detours Optimal Path + Extra Detours

Challenges

Discovering agents in a shared environment to accomplish tasks collaboratively. Effective context management Foodie

Building a richer conversational agent is difficult. Platforms are not sophisticated enough yet. Accurate voice recognition services for people with accents and environments with noise

Smidge

Accurately identifying products inside the fridge

13

Key Takeaways

Cognitive IOT Recipe Maven integrates food-related applications and uses context to collaboratively enhance the user experience. It includes CAPRecipes, a context-aware personalized recipes recommender, Foodie: a conversational agent for the smart kitchen, Smidge: An IoT Enabled Fridge and SmartGrocer: a profit-aware store path optimizer There is great potential for building systems which cross context barriers and enhance user experiences.

14

Thanks

15

Miguel Jiménez, miguel@uvic.ca Prashanti Angara, pangara@uvic.ca Hausi Müller, hausi@uvic.ca Joanna Ng, jwng@ca.ibm.com