Approximate property checking of mixed-signal circuits Parijat - - PowerPoint PPT Presentation

Approximate property checking of mixed-signal circuits

Parijat Mukherjee, Texas A&M University Chirayu Amin, Intel Corporation Peng Li, Texas A&M University

TAU 2014

Thursday, March 6th 2014

Outline

• Mixed-signal design
• Property checking
• Why approximate?
• Interpreting “failure probability”
• Approximate property checking
• Implementation details
• Conclusion : Challenges and future work

2 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Mixed-signal design

• Significant analog components

– Inherently complex : Continuous variables – Extremely large state space – Simulation computationally expensive

• Why talk of it now?

– Circuit complexity – Process variability – Frequency band of interest – Effect of Jitter / Noise etc

3

INCREASING !

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

May not always capture the intended function (specification) Violations are observed here

Does a circuit “always” perform its intended function?

4 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

May not always capture the intended function (specification) Violations are observed here

Does a circuit “always” perform its intended function?

5 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Detection and Diagnosis

Property checking

Input Specs PVT Circuit Output Specs

• Outputs

– Probability of failure – Debug information

• Counterexamples
• Important parameters
• Failure patterns

6

Does circuit meet specifications over its entire range of operation?

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Why doesn’t it meet target specifications?

Statistical property checking

• What is the statistical quantity varied?

– Any parameter that can change circuit behavior

• What is the statistical quantity checked?

– Output signals vs. specifications

7 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Addressing circuit complexity

8

Circuit Size Failure Probability Can be simulated via SPICE Cannot be simulated via SPICE

Circuit level techniques System level techniques Hierarchical decomposition via models

Warning : Figure may not be drawn to scale

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Addressing circuit complexity

9

Circuit Size Failure Probability Can be simulated via SPICE Cannot be simulated via SPICE

Circuit level techniques System level techniques Hierarchical decomposition via models

Warning : Figure may not be drawn to scale

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Existing work - Detection

10

Circuit Size Failure Probability

Standard Monte Carlo System level techniques Yield Estimation

Warning : Figure may not be drawn to scale

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Existing work - Detection

11

Circuit Size Failure Probability

Standard Monte Carlo System level techniques Yield Estimation Proposed

Warning : Figure may not be drawn to scale

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

• P. Mukherjee, C. Amin and P. Li,

"Approximate property checking of mixed-signal circuits", DAC 2014

• R. Kanj, R. Joshi, and S. Nassif, “Mixture

importance sampling and its application to the analysis of sram designs in the presence of rare failure events” DAC ’06

• S. Sun, Y. Feng, C. Dong, and X. Li,

“Efficient sram failure rate prediction via gibbs sampling,” TCAD ’12

• M. H. Kalos and P. A. Whitlock, “Monte

carlo methods”, Wiley-VCH, 2008.

Outline

• Mixed-signal design
• Property checking
• Why approximate?
• Interpreting “failure probability”
• Approximate property checking
• Implementation details
• Conclusion : Challenges and future work

12 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Unknown Circuit

An example

13

Input Parameter Time domain property distribution Process Parameter

Uniform Gaussian

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Parameter and property spaces

Parameter space Property space

14

There exists a mapping between Parameters and Properties !

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Parameter and property spaces

Parameter space Property space

15

Failure can be observed directly in the property space !

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

𝑄 𝐺 𝑍1 = 1 𝑄 𝐺 𝑍1 = 0

Parameter and property spaces

Parameter space Property space

16

Interactions between parameters give rise to failures !

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

𝑄 𝐺 𝑍1 = 1 𝑄 𝐺 𝑍1 = 0 𝑄 𝐺 𝑌1 ∈ 𝑠𝑓𝑒, 𝑌2 ∈ 𝑠𝑓𝑒 = 1

The real picture

Parameter space Property space

17

Multiple violations over multiple parameter combinations

Property 1 Property 2

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Approximate failure probability

Failure probability estimation

18

Statistical framework

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Requirements

• A statistical framework that :

– Requires low number of SPICE simulations – Can bound the failure probability uncertainty – Can deal with varying input distributions – Does not need to assume an output distribution – Does not assume an input-output relationship – Can exit early with an approximate bounded estimate

• By assuming that :

– Failure regions are more or less contiguous (Few probable failure regions vs. many unlikely ones)

19 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

What is Failure probability?

20

𝑄 𝑔𝑏𝑗𝑚𝑣𝑠𝑓 = 𝐵𝑠𝑓𝑏 𝑠𝑓𝑒 𝐵𝑠𝑓𝑏 𝑠𝑓𝑒 + 𝐵𝑠𝑓𝑏 𝑕𝑠𝑓𝑓𝑜

P (Y1) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Where does uncertainty arise? Part 1

21

Have we followed the output distribution exactly?

Have the relative areas been estimated correctly?

P (Y1) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Adaptive sampling

22

SPICE is Expensive !

Failure Estimate Few Strategic Samples (Arbitrary distribution)

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

TRAINING SET TRAINING SET

Model driven sampling

Accuracy ∝ How well model has captured failure boundaries

23

SPICE is Expensive !

Extensive Resampling (Parameter distribution) Failure Estimate

Inexpensive !

Few Strategic Samples (Arbitrary distribution)

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Where does uncertainty arise? Part 2

24

Have we covered all the failure regions?

Do we have at least one sample in the “likely” regions?

P (Y1) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

The N initial samples

25

Following parameter distribution can cause redundant effort

Warning : Artificial test case designed to excite complex interactions

Target parameter space Initial samples

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

The N initial samples

26

Spread out samples using prior information (only if it exists)

Warning : Artificial test case designed to excite complex interactions

Target parameter space Initial samples

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Where does uncertainty arise? Part 3

27

How accurate are our boundaries?

Are inaccurate boundaries for unlikely events really a problem?

P ( misclassification(Y1) ) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Reducing uncertainty

28

Observation: Highest uncertainty around failure boundaries

( Warning : Chosen model may introduce additional spurs )

P ( misclassification(Y1) ) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Reducing uncertainty

29

Can be reduced by Active learning !!!

( Run SPICE simulations for low confidence regions )

P ( misclassification(Y1) ) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Adaptive Sampling

30

Model accuracy ∝ How well failure boundaries have been captured

( Discussed previously )

Target parameter space Training samples

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Adaptive Sampling

31

Model accuracy ∝ How well failure boundaries have been captured

Adaptive sampling produces new samples close to boundary

Target parameter space Training samples

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Bounding the failure probability

32

𝑄 𝑔𝑏𝑗𝑚𝑣𝑠𝑓 ≤ P 𝑔𝑏𝑗𝑚𝑣𝑠𝑓 + Area 𝑕𝑠𝑓𝑓𝑜 𝑄 𝑔𝑏𝑗𝑚𝑣𝑠𝑓 ≥ P 𝑔𝑏𝑗𝑚𝑣𝑠𝑓 – Area 𝑠𝑓𝑒

P ( misclassification(Y1) ) vs Y1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

High level overview

• Introducing the “interval learner”
• A bias compensation stage wrapped around it

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14 33

The interval learner

• Instead of a mapping 𝑌 → 𝑍, the model returns,

∀𝑦, 𝑄 𝐺 𝑦 = 𝑄 𝐺 𝑧 𝑞 𝑧 𝑦 𝑒𝑧

𝑧

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14 34

∀𝑦, 𝑄 𝐺 𝑦 = 𝑄 𝐺 𝑧 𝑞 𝑧 𝑦 𝑒𝑧

𝑧

The interval learner

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14 35

• Purpose of model is to make a prediction. Thus, ∀𝑦 :

– The prediction 𝑗 𝑦 = 𝑄 𝐺 𝐹 𝑧 𝑦 – Probability of misprediction 𝑄 𝜗 𝑦 = 𝑄 𝐺 𝑦 𝑗 𝑦 = 0 1 − 𝑄 𝐺 𝑦 𝑗 𝑦 = 1

The interval learner

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14 36

• Purpose of model is to make a prediction. Thus, ∀𝑦 :

– The prediction 𝑗 𝑦 = 𝑄 𝐺 𝐹 𝑧 𝑦 – Probability of misprediction 𝑄 𝜗 𝑦 = 𝑄 𝐺 𝑦 𝑗 𝑦 = 0 1 − 𝑄 𝐺 𝑦 𝑗 𝑦 = 1

P (F) = 𝑗 𝑦 𝑞 𝑦 𝑒𝑦

𝑦

P (F) − 𝑄 𝜗|𝑦 𝑞 𝑦 𝑒𝑦

𝑦|𝑗 𝑦 =1

, P (F) + 𝑄 𝜗|𝑦 𝑞 𝑦 𝑒𝑦

𝑦|𝑗 𝑦 =0

𝑛𝑗𝑜𝑄, 𝑛𝑏𝑦𝑄 =

Bias reduction : Ensemble learning

• K-fold cross validation on original training data

37

Original Training Data Randomly sampled buckets of size -m

Delete -m

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Bias reduction : Ensemble learning

• At each iteration, delete 𝑛 = 𝑂/𝐿 samples

38

Original Training Data

Delete -m

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Model 1

Bias reduction : Ensemble learning

• At each iteration, delete 𝑛 = 𝑂/𝐿 samples

39

Original Training Data

Delete -m

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Model 2

Bias reduction : Ensemble learning

• At each iteration, delete 𝑛 = 𝑂/𝐿 samples

40

Original Training Data

Delete -m

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Model 5

Bias reduction : Data and Model

• Consolidate estimates from last 𝐿 models

41

Model 1 Model 2 Model 3 Model 4 Model 5 Model 1 Model 2 Model 3 Model 4 Model 5 Overall P(failure) 1 P(failure) 2 P(failure) 3 P(failure) 4 P(failure) 5 ← 𝑏𝑤𝑓𝑠𝑏𝑕𝑓 Lower bound 1 Lower bound 2 Lower bound 3 Lower bound 4 Lower bound 5 ← 𝑛𝑗𝑜 Upper bound 1 Upper bound 2 Upper bound 3 Upper bound 4 Upper bound 5 ← 𝑛𝑏𝑦

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Bias reduction : Training Data

• Adaptive sampling points added at each step

– Slowly reduces bias due to original training set

42

Model 1

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Bias reduction : Training Data

• Adaptive sampling points added at each step

– Slowly reduces bias due to original training set

43

Model 2

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Bias reduction : Training Data

• Nature of ensemble changes very little

– K-fold subsampling only on original seed data

44

Model 1 Model 2 Model 3 Model 4 Model 5

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

0.3 0.4 0.5 100 200 300 400 500

When to stop?

45

Bounds Target parameter space Resampled Model

𝐿 = 5

P (𝑔𝑏𝑗𝑚𝑣𝑠𝑓) →

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

# 𝑇𝑄𝐽𝐷𝐹 𝑇𝑗𝑛𝑣𝑚𝑏𝑢𝑗𝑝𝑜𝑡 →

0.3 0.4 0.5 100 200 300 400 500 Original Bounds

When to stop?

46

Bounds Target parameter space Resampled Model

𝐿 = 5

P (𝑔𝑏𝑗𝑚𝑣𝑠𝑓) →

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

# 𝑇𝑗𝑛𝑣𝑚𝑏𝑢𝑗𝑝𝑜𝑡 →

Summary

𝑂 Initial samples Simulate (SPICE) K-fold subsampling Build failure model Resample failure model Check failure probability and bounds Pick 𝑟 low confidence points

47

Run loop at least 𝐿 times Exit if met !

Tournament selection Average / bound

• ver last 𝐿 runs

Bayesian Additive Regression Trees Latin Hypercube sampling Latin Hypercube sampling

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Experimental setup

Integer-N Phase-locked loop (PLL)

• 9 Input parameters and 31 design uncertainties (40 total)
• 8 output properties
• SPICE Simulation time : 0.5 hrs
• Two versions with varying error rates

Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14 48

Results : Adaptive sampling

• Cost of SPICE simulation dominant
• Cost of model building and evaluation negligible

Evolution of P(F) estimate with number of simulations for PLL2

49 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Results : Bias compensated interval learner

• Interval grows smaller or less biased over time
• Better off using bootstrap if sampling using MC + LHS
• Works better for lower failure probabilities (Previous slide)

50 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Outline

• Mixed-signal design
• Property checking
• Why approximate?
• Interpreting “failure probability”
• Approximate property checking
• Implementation details
• Conclusion : Challenges and future work

51 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Summary of preliminary work

• Failure probability estimation

– Accuracy comes at a (high) cost – Simulation budgets are limited – Focus on doing the best job within the limited budget – Allow user to exercise accuracy vs. turn-around-time trade-offs

52 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Future work

Short Term

• Optimal choice of model?
• Compensate for model bias?

Long Term

• Diagnosis using discovered failures [DAC ’14]
• Extend to equivalence checking
• Failure model applied to system level analysis?

53 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14

Questions

Lots of questions answered BUT Most are yet to be answered Your questions !!!

54 Approximate property checking of mixed-signal circuits - Parijat Mukherjee, TAU '14