Incorporating Engineering Knowledge Max Yi Ren, Panos Y. Papalambros - PowerPoint PPT Presentation

Adaptive Choice-Based Conjoint Analysis Incorporating Engineering Knowledge Max Yi Ren, Panos Y. Papalambros University of Michigan, Ann Arbor IDETC’14 Buffalo, NY Aug 19 th , 2014

Motivation (1/2) partworth Engineering Survey or & cost model Preference model market data Optimal design: Conditional 1 Design w/ the highest prob. 3 probabilities to be to be the most profitable. 2 the most profitable Objective from marketing perspective: Improve the hit-rate of the preference model to help design optimization. 2

Motivation (2/2) Consider optimizing the profit of a USB drive w.r.t. memory capacity $200 Cost $40 $16 $6 $5 $5 (Example modified from E. Feit Dissertation) 8G 16G 32G 64G 128G 256G  “8G” is never optimal if larger memory is more preferred.  No need to measure the partworth on the level “8G”.  Probability of “256G” to be the optimal is low due to high cost. 3

Outline  Overview : Optimization by asking good questions  Intuition : Preference modeling vs. design optimization  Algorithm : Group Generalized Binary Search  Case study  Conclusion 4

Overview Partworth distribution Deterministic Queries engineering & (pairwise Preference model cost model choices) Optimal design: Conditional 1 One w/ the highest prob. 3 probabilities to be to be the most profitable, 2 the most profitable within a finite set . Objective : To find the optimal design w/ the least number of queries 5

Intuition User feedback to a query is a cut in the version space (the space of feasible partworth). The difference b/w preference modeling and design optimization: 6

Group Generalized Binary Search (1/3) Originally proposed in: Bellala et al., 2012. “ Group based active query selection for rapid diagnosis in time critical situations ”. IEEE Transactions on Information Theory, 58(1), pp. 459 – 478. Internal node : Contains the Leaf node : Contains the final constrained version space, and constrained version space the design most probable to be reaching the node, the current the most profitable. query and response.  A sequence of queries forms a path . Ideally, it terminates when the optimal design is derived.  A query strategy determines which query to make under user feedback to previous queries. 7

Group Generalized Binary Search (2/3)  The partworths of the user is a random vector following an unknown distribution. A prior distribution is assumed .  Each query strategy leads to an expected path length , which could be minimized by the best strategy. 8

Group Generalized Binary Search (3/3) The GGBS algorithm finds the best question to ask, based on current and previous users’ responses The algorithm minimizes at each node : Probability mass from Most uncertain D1 the same design into the question same child node D2 MCMC used for calculating conditional probabilities. 9

Case Study: Dial-readout Scale Design 2455 feasible designs on 6 attributes w/ 5 levels. Data and model from Michalek, Feinberg, Papalambros, 2005. 10

Case Study: Settings & Assumptions  The true partworths are used to simulate responses.  Responses have no random error.  Engineering models are deterministic.  The best query is picked from four candidates w/ the highest conditional probabilities to be the optimal.  GGBS is compared w/ uncertainty sampling (utility balance).  20 independent simulations, each with 50 queries. 11

Case Study: Convergence Result Average #query to find the correct best two designs derived from full knowledge 27 34 Proposed GGBS Variance from MCMC Utility balance Sampling size = 1e5 Most probable vs. current chosen 12

Case Study: Correlation Correlation btw. Estimated partworths and the truth: Proposed GGBS Utility balance Most probable vs. current chosen 13

Future Work  Update of conditional probabilities Queries Prior distribution Posterior distribution HB Model  Query w/ more than two designs  GGBS with noisy user choices 14

Conclusions Key points : The query strategy for finding the optimal design can be enhanced by engineering knowledge. Contribution : Adaptive query is achieve by greedily minimizing the expected path length through GGBS. Major limitation : Real-time interaction is limited to ~1000 candidate designs due to high computation time of MCMC. 15

Acknowledgement  NSF CMMI-1266184  Dr. Fred Feinberg, B.School, Univ. of Michigan  Dr. Clayton Scott, EE, Univ. of Michigan  All my reviewers 16

Thank you What questions do you have? 17