Intelligence in FINN DND Software 2020 Nicolai Hge 2008 Nicolai - PowerPoint PPT Presentation

Use of Artificial Intelligence in FINN DND Software 2020 Nicolai Høge

2008 Nicolai Høge 2001 - 2006 1999 - 2001 2006 - 2008

Hva skal jeg snakke om? • Hva er kunstig intelligens og maskinlæring? • To konkrete eksempler fra FINN: • Annonsekontroll • Anbefalinger • Hva har vi lært etter 5+ år ?

What is Artificial Intelligence? ● To get machines to solve tasks, that humans need intelligence to do ● Most of the work around (AI) is open and available to everyone. That means that knowledge is not the key, but rather the access to data and competent people ● Many sub categories ○ NLP, ML, Image Recognition etc.

Why is everybody talking about Machine Learning? The computer learns how to categorize or group input Without ML Supervised Unsupervised Input Input Output Model Input Model Algorithm Output

Ad Control

Machine Learning for Ad Control Store the results Ad 500 000 changes pr / week 1 . Control 2. 3.

Human rules, manual weighting

ML weighting of rules From manual weighting to ML weighting of rules

ML generated rules From manual rules to ML generated rules

Train model with more data Train the ML model with more data

Keep tuning until “good enough”

Recommendations

One visit, many signals 100 different signals pageviews recommendation inscreen contact_action search ... Typically 100 signals pr. visit

Many visits, A LOT of signals 200-300 million signals pr. day Data lake 9 TB

From signals to recommendations AB testing Data lake 9 TB Machine Learning 30 - 40 models ● Experimentation ● Always 3-4 different ● algorithms running against each other Validation of models in ● production

Recommendations based on users or objects or Input Recommendation Output Service

One engine, many products Personalized search BLINK Recommendations Use signals to improve Recommendation recommendations Service

Results CTR 30% 1000 clicks/ min

What is under the hood?

Combining more than one model ● Collaborative Filtering works with a lot of data ● Confused with little data ● Combine text, image and collaborative filtering KDD 2018: Eide, Zhou, Øygard: Five lessons from building a deep neural network recommender

More relevant results

When things go wrong...

...and challenges Our success factors... ● Difficult to recruit the right ● Data Scientists are part of the people product development teams ● Hard to gather data ● Tons of experiments ● The models are only as good ● Our ability to create products as the data they learn from ● Off the shelves models may be good enough, but you probably need use them in inventive ways