Machine Learning-based Anomaly Detection for Post-silicon Bug - - PowerPoint PPT Presentation
Machine Learning-based Anomaly Detection for Post-silicon Bug Diagnosis Andrew DeOrio , Qingkun Li, Matthew Burgess and Valeria Bertacco University of Michigan University of Illinois Verification trends Wilson Research Group and Mentor
University of Michigan University of Illinois
Wilson Research Group and Mentor Graphics 2010 Functional Verification Study
Andrew DeOrio / University of Michigan 20-Mar-2013 2
Bob Barton, Intel. Invited talk at GSRC.
20-Mar-2013 Andrew DeOrio / University of Michigan 3
Design and pre-silicon verification effort Post-silicon validation effort
Pre-silicon Post-silicon Product
20-Mar-2013 Andrew DeOrio / University of Michigan 4
20-Mar-2013 Andrew DeOrio / University of Michigan 5
pushl %epb movl %epb
same test
many different results
difficult to locate bug!
same card
many different transactions
difficult to locate fraud!
20-Mar-2013 Andrew DeOrio / University of Michigan 6
pushl %epb movl %epb
same test same card
new transaction
anomaly?
compare
anomalous time and location
failing test compare
20-Mar-2013 Andrew DeOrio / University of Michigan 7
pushl %epb movl %epb
same test
unknown example training data: positive examples clustering algorithm anomalous time and location signal A signal B
time@1=2 time@1=1
time@1=1 time@1=2
signal A feature value signal B feature value
20-Mar-2013 Andrew DeOrio / University of Michigan 8
clustering algorithm
clusters
time@1
feature values of passing examples One test, 1st time window signal A signal B 1st time window
signal A feature value signal B feature value
20-Mar-2013 Andrew DeOrio / University of Michigan 9
clustering algorithm
One test, 1st time window signal A signal B 1st time window feature values of unknown examples
signal A feature value signal B feature value
20-Mar-2013 Andrew DeOrio / University of Michigan 10
clustering algorithm
One test, 2nd time window signal A signal B 2nd time window
# anomalies > threshold
signal A feature value signal B feature value
20-Mar-2013 Andrew DeOrio / University of Michigan 11
clustering algorithm
– Circular clusters become hyper-spheres – High dimensionality is a challenge
– Cap #signals in one clustering set (500) – Group signals by module(s) (100-500 signals) – Apply clustering to each group
10 testcases 100 random seeds: variable memory delay, crossbar random traffic 10 bugs: e.g., functional bug in PCX, electrical error in Xbar monitored 41,743 top level control signals 1000 buggy runs 1000 passing runs
HW
20-Mar-2013 Andrew DeOrio / University of Michigan 12
10 seeds
Bug Description PCX_gnt SA Stuck-at in PCX grant Xbar elect Electrical error in crossbar BR fxn Functional bug in branch logic MMU fxn Functional bug in memory controller PCX_atm SA Stuck-at in PCX atomic grant PCX fxn Functional bug in PCX XBar combo Combined electrical errors in Xbar/PCX MCU combo Combined electrical errors in mem/PCX MMU combo Combined functional bugs in MMU/PCX EXU elect Electrical error in execute unit
20-Mar-2013 Andrew DeOrio / University of Michigan 13
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Percentage of Testcases
exact signal detected
signals detected no bug effect false negative false positive
20-Mar-2013 Andrew DeOrio / University of Michigan 14
9/10 bugs caught
20-Mar-2013 Andrew DeOrio / University of Michigan 15
20-Mar-2013 Andrew DeOrio / University of Michigan 16