How reliable are the consumers? Comparison of sensory profiles from - - PowerPoint PPT Presentation

How reliable are the consumers? Comparison of sensory profiles from consumers and experts from consumers and experts

WORCH Thierry(1) LÊ Sébastien(2) LÊ Sébastien( ) PUNTER Pieter(1)

(1) OPP Product Research (2) A

C O t

mailto: thierry@opp.nl

(2) AgroCampus Ouest

Project 8013 July 2008 Senior project manager Pieter Punter Project manager Thierry Worch

introduction

• in the sensory theory:
• experts panels are used for the products’ description

p p p p

• consumers should only be used for the hedonic task
• they lack two essentials qualities for profiling (consensus

and reproducibility)

• there are strong halo effects (Earthy, MacFie & Hedderley,

there are strong halo effects (Earthy, MacFie & Hedderley, 1997)

• in the sensory practice:
• consumers are sometimes used for both tasks
• it has been proven that consumers’ description show the
• it has been proven that consumers description show the

required qualities (consensus and reproducibility) (Husson, Le Dien, Pagès, 2001)

8013

3

problematic

How reliable are the consumers?

8013

4

presentation of the studies

• products:
• twelve luxurious women perfumes

twelve luxurious women perfumes (Gazano, Ballay, Eladan & Sieffermann, 2005)

A l L’I t t Angel (Eau de Parfum) L’Instant (Eau de Parfum) Cinéma J’Adore (Eau de Parfum) (Eau de Toilette) Pleasures (Eau de Parfum) J’Adore (Eau de Parfum) (Eau de Parfum) (Eau de Parfum) Aromatics Elixir (Eau de Parfum) Pure Poison (Eau de Parfum) Lolita Lempicka (Eau de Parfum) Shalimar (Eau de Toilette) Chanel N°5 Coco Mademoiselle

8013

5

Chanel N 5 (Eau de Parfum) Coco Mademoiselle (Eau de Parfum)

presentation of the studies

• expert panel (Agrocampus Rennes)
• twelve persons (11 students and 1 teacher) from the Chantal Le

Cozic school (esthetics and cosmetic school)

• focus group per group of six, with two animators
• generation of a list of twelve attributes
• generation of a list of twelve attributes
• “Vanille”, “Notes Florales”, “Agrume”, “Boisé”, “Vert”, “Epicé”, “Capiteux”,

“Fruité”, “Fraîcheur Marine”, “Gourmand”, “Oriental”, “Enveloppant”

• training session for the most difficult ones
• the twelve products were tested two times in two one-hour

sessions

8013

6

presentation of the studies

• consumer panel (OP&P Product Research, Utrecht)
• 103 naïve Dutch consumers living in the Utrecht area
• the same twelve perfumes were rated on 21 attributes
• the same twelve perfumes were rated on 21 attributes
• “odour intensity”, “freshness”, “jasmine”, “rose”, “camomile”, “fresh lemon”,

“vanilla”, “mandarin/orange”, “anis”, “sweet fruit/melon”, “honey”, “caramel”, “spicy” “woody” “leather” “nutty/almond” “musk” “animal” “earthy” “incense” spicy , woody , leather , nutty/almond , musk , animal , earthy , incense , “green”

• two products (Shalimar and Pure Poison) were duplicated
• the fourteen (12+2) products were tasted in two one-hour sessions (seven

products in each session, presentation order was balanced)

8013

7

presentation route map

• the consumer and expert data are compared in three different ways

1.Univariate analysis

• analyses of variance
• correlations

2 Multivariate comparison 2.Multivariate comparison

• construction of the two products’ spaces (PCA)
• comparison of the products’ spaces through GPA and MFA

comparison of the products spaces through GPA and MFA 3.Confidence ellipses

• graphical confidence intervals around the products averaged
• ver the two panels
• graphical confidence intervals around the products defined

8013

8

graphical confidence intervals around the products defined by the different panels

Performance of the two panels (univariate analysis)

8013

9

performance of the panels

• usually, the expert panels should have many qualities:
• discrimination: panelists should be able to detect and describe

the differences existing between the products

• reproducibility: panelists should describe the products in the

same way, when they are repeated same way, when they are repeated

• agreement: panelists should give the same description of the

f products as the rest of the panel

• it can be measured with the correlations (usually, one

panelist is compared to the mean over the rest of the pa e s s co pa ed o e ea

• e

e es o e panel)

8013

10

expert panel

• panel performance
• discriminate on 11 out of 12 attributes (“Agrume”, pvalue=0.08)
• reproducible for 11 out of 12 attributes (“Notes Florales”)

panellist performance (discrimination reproducibility)

• panellist performance (discrimination, reproducibility)
• panellists 1, 3 and 12 are very good

panellists 1, 3 and 12 are very good

• panellists 8, 9 and 10 are not good in discrimination

(discriminate the products on less than 6 out of 12 attributes) lli 9 i l d i d ibili ( d ibl

• panellist 9 is also not good in reproducibility (reproducible on
• nly 3 out of 12 attributes. “Notes Florales”, “Agrume” and

“Enveloppant”)

8013

11

expert panel (correlations)

• distribution of the correlations (correlation between expert i and the

mean over the (n-1) others) ( ) )

8013

12

consumer panel

• discrimination (on the twelve original products)
• the consumers discriminate the products on all attributes except

“camomile” (pvalue = 0.62)

• NB: the consumers discriminate on “Citrus” (pvalue < 0.001)
• reproducibility (on the two duplicated products only)
• consumers are reproducible on all attributes except one

(“woody”)

8013

13

consumer panel (reproducibility)

Shalimar Shalimar 2

8013

14

consumer panel (correlations)

• distribution of the correlations (correlation between a consumer i and

the mean over the (n-1) others) ( ) )

8013

15

conclusions on the panel performance

• expert panel
• discriminates between the products

discriminates between the products

• are reproducible
• high correlations

consumer panel

• consumer panel
• discriminates between the products
• shows reproducibility’s qualities

shows reproducibility s qualities

• lower but still positive correlations (consumers are untrained)

Both panels show the same qualities

8013

16

Both panels show the same qualities

Products’ spaces (multivariate analysis)

8013

17

methodology

• products’ spaces
• the products profiles (averaged over the panellists or consumers)

the products profiles (averaged over the panellists or consumers) are computed.

• Principal Components Analysis is then run on these product x

attribute matrices attribute matrices

• comparison of the two products’ spaces (expert and consumer) is a

comparison of the two products spaces (expert and consumer) is a “multi-table problem”

• comparison through the Procrustean analysis

( )

• comparison through Multiple Factor Analysis (MFA)
• comparison through the confidence ellipses technique

8013

18

expert panel

g p

2 5 3,0 3,5

AromaticsElixir

1,0 1,5 2,0 2,5

%)

AromaticsElixir Chaneln5 JAdore EP Pleasures Shalimar

1

• 1,0
• 0,5

0,0 0,5

nsion 2 (21.87 %

CocoMelle _ JAdore_ET LInstant PurePoison Epicé Boisé

• 2,5
• 2,0
• 1,5

Dimen

Angel Cinema

2 (21.87 %)

Capiteux Vert Notes.florales Agrume Fraicheur.marine Oriental

• 4,5
• 4,0
• 3,5
• 3,0

LolitaLempicka

Dimension

Fruité Enveloppant

• 4
• 3
• 2
• 1

1 2 3 4

Dimension 1 (64.22 %)

Vanille

8013

19

• 1

1

Dimension 1 (64.22 %)

• 1

Gourmand

consumer panel

4,5 5,0 5,5 6,0 1 5 2,0 2,5 3,0 3,5 4,0

%)

Angel Cinema LInstant LolitaLempicka

g

1

vanilla

• 1,0
• 0,5

0,0 0,5 1,0 1,5

nsion 2 (17.97 %

CocoMelle JAdore_EP JAdore_ET Pleasures PurePoison camomille citrus anis honey caramel

• 4,0
• 3,5
• 3,0
• 2,5
• 2,0
• 1,5

Dimen

Chaneln5 Shalimar

n 2 (17.97 %)

freshness jasmin sweet_fruit nutty animal green 7 0

• 6,5
• 6,0
• 5,5
• 5,0
• 4,5

AromaticsElixir

Dimension

i t it jasmin rose fresh_lemon i woody leather musk animal earthy incense

• 5
• 2.5

2.5 5 7.5

Dimension 1 (68.29 %)

• 7,0

1

intensity spicy

8013

20

• 1

1

Dimension 1 (68.29 %)

• 1

Multivariate comparison of the two panels (GPA and MFA)

8013

21

expert vs consumer: Procrustes analysis

GPA consensus space (coefficient of similarity: 0.93)

0.4

(coefficient of similarity: 0.93)

0.2 0.3 A l LolitaLempicka 0.1 m 2 Angel Cinema LInstant 0.0 Dim CocoMelle JAdore_EP JAdore_ET Pleasures PurePoison

• 0.2
• 0.1

AromaticsElixir Chaneln5 Shalimar 0 2 0 0 0 2 0 4

• 0.3
• 8013

22

• 0.2

0.0 0.2 0.4 Dim 1

expert vs consumer: Multiple Factor Analysis

experts consommateurs

MFA partial points’ representation

2 Angel LolitaLempicka

(RV coefficient: 0.87)

1 9.35 %) Angel Cinema LInstant Dim 2 (1 Chaneln5 CocoMelle JAdore_EP JAdore_ET Pleasures PurePoison Shalimar

• 1

AromaticsElixir

• 2
• 1

1 2 3

• 2

8013

23

• 2
• 1

1 2 3 Dim 1 (64.06 %) I di id l f t

expert vs consumer: Multiple Factor Analysis

g

1

Vanille Gourmand vanilla honey

MFA variables’ representation

expert consumer

camomille anis caramel

(RV coefficient: 0.87)

(19.35 %)

Fruité Fraicheur marine Enveloppant freshness citrus sweet_fruit nutty

Dimension 2

Capiteux Vert Notes.florales Agrume Fraicheur.marine freshness jasmin rose f h l woody leather musk animal earthy incense green

D

Epicé Boisé Oriental intensity fresh_lemon spicy

• 1

8013

24

• 1

1

Dimension 1 (64.05 %)

Comparison through the fid lli t h i confidence ellipses technique

(Husson, Lê & Pagès, 2005) ( , g , ) (Lê, Pagès & Husson, 2008)

8013

25

confidence ellipses

methodology

1.Compute the product profiles (averaged by product over the judges) 2.Create the products’ space 3 R l b b t t i l 3.Re-sample by bootstraping new panels 4.For each new panel, compute new products’ profiles 5.Project as illustrative the products on the original product space 6.Steps 3 to 5 are repeated many times (i.e. 500 times) 7.Confidence ellipses around the products containing 95% of the data are constructed constructed

principle

• if ellipses are superimposed, the products are not significantly different
• the size of the ellipses is related to the variability existing around the

products

8013

26

p

confidence ellipses

2

Confidence ellipses around the products

2

LolitaLempicka

1 .39%)

Angel Cinema LInstant

Dim 2 (19.

Chaneln5 CocoMelle JAdore_EP JAdore_ET Pleasures PurePoison

• 1

AromaticsElixir Chaneln5 Shalimar

• 3
• 2
• 1

1 2 3 4

8013

27

Dim 1 (64.02%)

confidence ellipses

• as we have two different panels, we can apply this methodology to

both

• creation of confidence ellipses around each product seen by

each panel (24 ellipses are created here) each panel (24 ellipses are created here)

• comparison of a given product through the two panels (same

comparison of a given product through the two panels (same colour) f ff (

• comparison of the different products within a panel (same type
• f line)

8013

28

confidence ellipses

Confidence ellipses for the partial points

2 1 9.39%)

Angel Cinema LInstant LolitaLempicka

Dim 2 (19

Chaneln5 CocoMelle JAdore_EP JAdore_ET Pleasures PurePoison Shalimar

• 2
• 1

AromaticsElixir cons. expert

• 4
• 2

2 4

• Dim 1 (64 02%)

expert

8013

29

Dim 1 (64.02%)

confidence ellipses

• partial points
• within a product, the ellipses related to the two panels are

always superimposed (no differences between the panels)

• the sizes of the ellipses are equal
• the sizes of the ellipses are equal
• the higher amount of consumers compensate the higher

variability due to the lack of training for consumers

8013

30

conclusions

• although consumers don’t have the habit to describe perfumes

although consumers don t have the habit to describe perfumes (difficult task), they give the same information as the expert panel (and it’s identical to the standard description of the perfumes)

• they also have the same qualities (discrimination and reproducibility)
• a difference between consumers and experts panel exists in the

variability of the results (more variability for consumers), but this is compensated by the larger size of the panel (here 103 vs 12) compensated by the larger size of the panel (here 103 vs 12)

• with consumers, not only intensity, but also ideal and hedonic

co su e s,

• o y

e s y, bu a so dea a d edo c questions can be asked in the same time

8013

31

references

• Earthy P., MacFie H & Hedderlay D. (1997). Effect of question order on sensory

perception and preference in central locations. Journal of Sensory Studies, vol.12, p215- 237 237

• Gazano G., Ballay S., Eladan N. & Sieffermann J.M. (2005). Flash Profile and flagrance

research: using the words of the naïve consumers to better grasp the perfume’s universe research: using the words of the naïve consumers to better grasp the perfume s universe. In: ESOMAR Fragrance Research Conference, 15-17 May 2005, New York, NY.

• Husson F

Le Dien S & Pagès J (2001) Which value can be granted to sensory Husson F., Le Dien S. & Pagès J. (2001). Which value can be granted to sensory profiles give by consumers? Methodology and results. Food Quality and Preference, vol.16, p291-296

• Husson F., Lê S.& Pagès J. (2005). Confidence ellipses for the sensory profile obtained

by principal component analysis. Food Quality and Preference, vol.16, p245-250

• Lê S., Pagès J. & Husson F. (2008). Methodology for the comparison of sensory

profiles provided by several panels: Application to a cross cultural study. Food Quality and Preference, vol.19, p179-184

8013

32

thank you

• special thanks to

special thanks to

• Melanie COUSIN
• Maëlle PENVEN
• Mathilde PHILIPPE
• Marie TOULARHOAT

students from AgroCampus-Rennes, who took care of the whole expert panel data.

8013

33

Thank you for your attention! Thank you for your attention!