S7504: Improving Consumer Compliance Through Better Product - - PowerPoint PPT Presentation

S7504: Improving Consumer Compliance Through Better Product Recommendation New Skin Advisor Tool Powered by AI

Jun Xu PhD, Faiz Sherman PhD, Matthew Barker PhD* Frauke Neuser PhD, Shannon Weitz

The Procter & Gamble Company

THE SCIENCE BEHIND OLAY SKIN ADVISOR skinadvisor.olay.com

5

• f the top10

Non-Food Product Launches

P&G scientists around the world work together to develop products that improve more lives in more meaningful ways – now and for generations to come.

BRAND INNOVATION BUSINESS INNOVATION SOCIAL INNOVATION

Years of innovation history

50 100 150

179

40,000

Active patents worldwide

More than

1,000

Ph.Ds

In 120 scientific disciplines

$2 billion+

Annual R&D investment

7,500

R&D employees

Procter & Gamble (P&G) NYSE: PG Global Research & Development

2016 IRI New Product Pacesetters

P&G 25+ Years of Industry Leading Skin Imaging

Background

• Consumers struggle to find the right cosmetic skin care products suited to their

personal needs and preferences. The ability to make an informed product selection decision can drive product compliance and delight.

• A new skin advisor tool has been developed to deliver a personalized beauty

consultation tailored for consumers’ unique skin needs right at her fingertips.

• This tool combines deep learning with consumer preferences related to visible skin

concerns, cosmetic product use, and skin feel for the optimal product recommendation.

Development Overview

Deep Learning Algorithm Visible skin age prediction with aging area identification. Visible Skin Age Validation Predictions compared to expert. Aging Area Insights Facial Mapping Study informs how appearance of aging areas change with chronological age. Compliance Verification Proving skin advisor with deep learning algorithm, visible aging insights and consumer preferences drives compliance.

Facial Features & Aging

nasolabial folds glabella marionette lines

Deep Neural Network application

• The skin advisor uses convolutional neural networks trained using NVIDIA

graphics processors to perform trillions of calculations per second. The model was trained on 50,000 images with chronological age data tags.

• When an image of a user is received, the model is used to determine the

visible skin age based on the pixels in the image, further a two- dimensional heat map is generated that identifies a region of the image that contributes to the visible skin age.

Predicted Age Raw Image Pixels

Data Setup

• Face detection & alignment performed using dlib: rotated, scaled &

cropped to a standard size.

• Spatial augmentation was applied: random horizontal flipping, rotation,

scaling, zoom cropping causing slight translation.

• HSV Color augmentation: random changes to saturation & exposure.
• Oval Mask, global contrast normalization GCN, reapply Oval Mask.

CNN using Torch

• Regime of learning rate as epochs increase
• Small batch size was utilized, using less memory, explorer more places to

find parameters estimates corresponding minimum

• 20+ layers: convolution, max pooling, leaky ReLU, decreasing spatial

dimension while increasing depth dimension. Dropout was also used in later layers … consistent with Alexnet architecture with adaption

• Multi-threading to aid speed of decompressing JPG and send data to the

GPU, otherwise the GPU is starved

• RMSProp was used to optimize gradient descent
• Model training took roughly 8 hours on NVIDIA Titan X GPU

Gradient Heat Map for Visualization

• After training, with fixed model parameters. A gradient heat map was

created in order to localize pixel differences of a subject’s image relative to younger than their predicted age.

• An input image was forward propagated through the model to obtain a

predicted age. Then a target of predicted age minus 10 years was set and the gradients were propagated back through the network to the input

• image. A heat map was created by summing absolute values of the RGB

gradients for each pixel and rescaling from 0 to 1 for display purposes.

• The gradient heat map was then blended with the original image to

visualize areas that were different from their younger predicted age.

Visible Skin Age Validation

Evaluate robustness of the visible skin age algorithm by comparing output to a gold standard dermatologist assessment.

• 1. A validation set of 630 selfie images representing the general US female

population were obtained.

• 2. These images were presented to 615 dermatologists, who represent the

gold standard in visible skin evaluation, in a randomized order in sets of 8

• images. Each dermatologist evaluated images.
• 3. The dermatologists were asked to input the perceived age of each image.

Validation Results

The mean difference of the predicted visible skin age versus the chronological age using the skin advisor deep learning algorithm was comparable to the mean difference

• f the perceived age versus the chronological age by dermatologists.

Mean Age Difference Deep Learning Algorithm Dermatologists

• To build a fundamental understanding of the underlying mechanisms of facial aging

across different facial sites, a clinical Facial Mapping Study enrolling over 150 subjects

• Study assessed facial skin genomics, image analysis parameters, lifestyle factors,

and skin measurements in two groups of female subjects: a younger ages (20-29 years) and an older ages (55-75 years). Study did not assess applying cosmetics.

• Facial locations analyzed included the forehead, crow’s feet area, under eye,

nasolabial fold, cheek, glabella, marionette lines, above mouth, and nose regions.

Lifestyle Molecular Physical Optical Visual

Facial Area Insights – Mapping Study

Facial Mapping Study - Results

GB = Glabella FH = Forehead UE = Under Eye CF = Crow’s Feet CK = Cheek NL = Nasolabial Fold MN = Marionette LP = Above Lip NS = Nose

• The Skin Advisor Tool shares the best aging area and the area that needs

improvement based on the deep learning algorithm. Key educational information about how those areas age is also given.

• Insights from the facial mapping study were used to inform how visible aging

areas change with chronological age.

• Quantitative assessment of wrinkles revealed distinct visible topography feature

presentation across facial zones and with aging.

Ages 55-75 Ages 20-29

Compliance Verification

• 100 US women, age 25-65, facial moisturizer users, were

enrolled in a 4-week online consumer test.

• Group 1 (n=50) received a product regimen based on the skin

advisor deep learning algorithm and preferences and Group 2 (n=50) self-selected a product regimen.

• Self-assessment questions were completed pre-use and post-

4 weeks product use.

Compliance Results

Figure 5 Post 4 weeks product use indicates satisfaction with the skin advisor product recommendation and improved consumer compliance with longer product use. Figure 4 Pre-product use indicates satisfaction with the skin advisor product recommendation.

Demo

Olay Skin Advisor – Website Results

• Over 1.4 million visits to the site
• High engagement rates
• Half the bounce ratio of a typical beautybrand.com website
• Twice the time spent vs. a typical beautybrand.com website
• Huge opportunity for real time consumer learnings
• 2.0 upgrade launched 6 weeks ago

Conclusion

Creating a tool that leverages a deep learning algorithm to predict visible skin age and aging areas creates motivation to comply to a cosmetic skin care regimen. Visible skin age and aging area analysis is further backed by dermatologist validation and clinical data to support a robust product recommendation. The new skin advisor tool combines this technical information with consumer preferences to recommended cosmetic products that provide delight required for skin care regimen compliance.

Thank you! Questions? Barker.ML@pg.com