ESTIMATION AS UNCERTAINTY REDUCTION What is this estimation thing, - - PowerPoint PPT Presentation

▶
estimation as uncertainty reduction
SMART_READER_LITE
LIVE PREVIEW

ESTIMATION AS UNCERTAINTY REDUCTION What is this estimation thing, - - PowerPoint PPT Presentation

Sep 19, 2023 137 likes •588 views

ESTIMATION AS UNCERTAINTY REDUCTION What is this estimation thing, anyway? Michael Godeck DrupalCamp Atlanta 2018 Sorting out time and materials Estimation is like of a problem of topography, where were mapping time and materials to a

slide-1
SLIDE 1

Michael Godeck DrupalCamp Atlanta 2018

What is this estimation thing, anyway?

ESTIMATION AS UNCERTAINTY REDUCTION

slide-2
SLIDE 2

Michael Godeck DrupalCamp Atlanta 2018

Sorting out time and materials…

Estimation is like of a problem of topography, where we’re mapping time and materials to a schedule to reveal the landscape where the project will play out.

slide-3
SLIDE 3

Michael Godeck DrupalCamp Atlanta 2018

One Bad Bid

Michael Godeck DrupalCamp Atlanta 2018 …what he missed was just how many times the roof had been resurfaced over the years with layers of tar and gravel, all hidden below the surface.

slide-4
SLIDE 4

Michael Godeck DrupalCamp Atlanta 2018

It’s not what you don’t know that’ll hurt you, it’s what you know that ain’t so.

Leveraging Uncertainty

– Mark Twain

Expressing certainty about something you have no rights to be certain about.

slide-5
SLIDE 5

Michael Godeck DrupalCamp Atlanta 2018

What is this estimation thing?

…making commitments against hazy business

  • bjectives and obviously changing scope.

Words get written Words get twisted Old meanings move in the drift of time

– Jethro Tull

slide-6
SLIDE 6

Michael Godeck DrupalCamp Atlanta 2018

“It all ends in tears anyway.”

– Jack Kerouac

Expert Opinion

…what starts out with the best of intentions ends up being a target, often unrealistic expectations, as assumptions crumble in the face of implementation.

slide-7
SLIDE 7

Michael Godeck DrupalCamp Atlanta 2018

Assumptions in discrete value estimates

…we substitute uncertainty with assumptions

  • n what they might turn out to be.

DrupalCamp Atlanta 2018 Michael Godeck

slide-8
SLIDE 8

Michael Godeck DrupalCamp Atlanta 2018

Range estimates

…a range estimate allows you to quantify what you’re uncertain about.

Allow you to quantify your uncertainty.

slide-9
SLIDE 9

Michael Godeck DrupalCamp Atlanta 2018

A better way to express uncertainty.

…make your uncertainty evident, leaving it exposed for ongoing, even iterative reevaluation.

You don’t have to commit to anything that you’re uncertain about.

slide-10
SLIDE 10

Michael Godeck DrupalCamp Atlanta 2018

A concept of measurement:

An iterative process of reducing your uncertainty about something that matters.

– Douglas Hubbard

Having a clear conception of what estimation actually is, that’s half the battle, right there.

slide-11
SLIDE 11

Michael Godeck DrupalCamp Atlanta 2018

A concept of measurement:

– Douglas Hubbard

…Uncertainty is reduced by obtaining new information by means of observations. When observations are quantitatively expressed, that constitutes a measurement.

A quantitatively expressed reduction

  • f uncertainty based on observations.
slide-12
SLIDE 12

Michael Godeck DrupalCamp Atlanta 2018

A Mathematical Theory of Communication

…Shannon postulated that information is the amount of uncertainty reduction in a signal.

slide-13
SLIDE 13

Michael Godeck DrupalCamp Atlanta 2018

…the amount of uncertainty reduction in a signal…

…uncertain values become first class citizens, where we can more easily take into consideration intangibles which we once dismissed as not subject to precise measurement.

slide-14
SLIDE 14

Michael Godeck DrupalCamp Atlanta 2018

Information Entropy

…2 coins, four possible outcomes, 2 bits of entropy.

slide-15
SLIDE 15

Michael Godeck DrupalCamp Atlanta 2018

Information Entropy

What is that average time spent in meetings among all IT department personnel during the course of an normal work week?

slide-16
SLIDE 16

Michael Godeck DrupalCamp Atlanta 2018

Information Entropy

A measurement doesn’t have to vanquish uncertainty to be valuable.

slide-17
SLIDE 17

Michael Godeck DrupalCamp Atlanta 2018

Information Entropy

A measurement doesn’t have to vanquish uncertainty to be valuable.

slide-18
SLIDE 18

Michael Godeck DrupalCamp Atlanta 2018

Confidence Interval

…to be useful, your range has to within striking distance

  • f reality, but something less than 100% certainty.

Something less than perfect certainty.

slide-19
SLIDE 19

Michael Godeck DrupalCamp Atlanta 2018

Tool Calibration

…when we think of calibration, usually we think about tools.

slide-20
SLIDE 20

Michael Godeck DrupalCamp Atlanta 2018

Tool Calibration

…Calibration is a procedure where you measure against a known quantity, to that when applied against an unknown, you can have confidence in the observation.

slide-21
SLIDE 21

Michael Godeck DrupalCamp Atlanta 2018

Eliminating Bias

…Bookies and insurance actuarials learn to reign in

  • verconfidence.

Bias influences expert

  • pinion resulting in

inconsistent estimates

slide-22
SLIDE 22

Michael Godeck DrupalCamp Atlanta 2018

Measuring against a known quantity

…Estimation calibration relies on arbitrary questions as known quantities to identify over and under confidence.

slide-23
SLIDE 23

Michael Godeck DrupalCamp Atlanta 2018

Measurement Methods

…There a many ways to measure things besides time and money. Knowing how different scales work, and how to use them is important to effective estimation.

slide-24
SLIDE 24

Michael Godeck DrupalCamp Atlanta 2018

Stanley Smith Stevens

  • 1. Nominal
  • 2. Ordinal
  • 3. Rational
  • 4. Interval

A taxonomy of measurement scales

slide-25
SLIDE 25

Michael Godeck DrupalCamp Atlanta 2018

Rational Scale Rational Scales: Time and Money

slide-26
SLIDE 26

Michael Godeck DrupalCamp Atlanta 2018

Rational Scale Interval Scales: Story Points

An arbitrary baseline is established each time the scale is used, so comparisons become meaningless and velocity calculations can be easily gamed.

Interval scale’s arbitrary baseline is the key to understanding scrum story points.

slide-27
SLIDE 27

Michael Godeck DrupalCamp Atlanta 2018

Ordinal Scales - Mohs Scale of Mineral Hardness

slide-28
SLIDE 28

Michael Godeck DrupalCamp Atlanta 2018

Ordinal Scales - 5 Start Rating

A five one star movies as good as one five star movie?

slide-29
SLIDE 29

Michael Godeck DrupalCamp Atlanta 2018

Conflating ordinal and rational scales

Mixing up measurement scales increases ambiguity, when it’s supposed to be reducing it.

slide-30
SLIDE 30

Michael Godeck DrupalCamp Atlanta 2018

Ordinal Scales: The Cockburn Scale

A brontosaurs is any story that’s too big to go into work. It doesn’t matter how much bigger on brontosaurs is from another, because both need decomposition before technical planning can be meaningful.

slide-31
SLIDE 31

Michael Godeck DrupalCamp Atlanta 2018

Ordinal Scales: OWASP Risk Scale

For sizing security risks, it definitely matters how much bigger one Tyrannosaurus Rex is from another; thus, ordinal is a poor choice of scale for this purpose.

slide-32
SLIDE 32

Michael Godeck DrupalCamp Atlanta 2018

Nominal Scales

…take into account the qualities of things, which sometimes offer more insight than quantitative measurements. DrupalCamp Atlanta 2018 Michael Godeck

slide-33
SLIDE 33

Michael Godeck DrupalCamp Atlanta 2018

Quantify your uncertainty

…quantifying our uncertainty about qualitative questions helps us better understand just what we’re uncertain about DrupalCamp Atlanta 2018 Michael Godeck

slide-34
SLIDE 34

Michael Godeck DrupalCamp Atlanta 2018

Range Validation: The Absurdity Test

The absurdity test gives you a starting point, when you have no clue.

slide-35
SLIDE 35

Michael Godeck DrupalCamp Atlanta 2018

Range Validation: Imagine you are wrong.

Imagine you’re wrong exercise helps you better understand the work.

slide-36
SLIDE 36

Michael Godeck DrupalCamp Atlanta 2018

Range Validation: Mid-point Anchoring

Mid-point anchoring exercise helps you eliminate bias.

slide-37
SLIDE 37

Michael Godeck DrupalCamp Atlanta 2018

Range Validation: Equivalent Bet

Equivalent bet exercise helps you recognize over and under confidence.

slide-38
SLIDE 38

Michael Godeck DrupalCamp Atlanta 2018

Range Validation: Arbitrage

The arbitrage exercise helps you better understand 90% confidence interval.

slide-39
SLIDE 39

Michael Godeck DrupalCamp Atlanta 2018

– Claud Shannon

  • Range estimates help us quantity uncertainty.
  • Range and probability replace the need for assumptions.
  • Decomposition is a form of estimation.
  • Understanding measurement scales is important to estimation.
  • Range validation exercises help pressure test estimates.
  • Estimation calibration helps up understand how to reflect our uncertainty.
  • 90% confidence interval keeps up focused on meaningful estimates.
  • The greater uncertainty, the easier to reduce it.
  • The only estimates that have value are those the affect decisions we face.

“Information is the resolution of uncertainty”

slide-40
SLIDE 40

Michael Godeck DrupalCamp Atlanta 2018

Diving Deeper…

This session developed the concept, object and methods of estimation, with a focus on range estimation and confidence intervals. Companion sessions that build on this foundation are:

  • “Estimation for Release Planning:

understanding the impact of queue time on when work will really be done.”

  • “Monte Carlo Simulations for project

estimation: range estimating techniques for complex scenarios such as project planning.”

  • “Story Decomposition: right-size your stories

and the world is your oyster.”

slide-41
SLIDE 41

Michael Godeck DrupalCamp Atlanta 2018

Monte Carlo Simulations

“Predictability builds and holds trust, a core Agile value, better than does delivering more with less reliability.”

Release Planning and Queue Time

Predicting when work will be done depends as much on understanding queue time in your process, as it does on developer effort-estimation. Understanding queue time depends on reasoning about different problems then effort estimation.

slide-42
SLIDE 42

Michael Godeck DrupalCamp Atlanta 2018

Monte Carlo Simulations Monte Carlo Simulations for Project Estimation

Monte Carlo simulations are the mathematics

  • f uncertainty, ideal for estimation when

complexity exceeds our mental capacity. DrupalCamp Atlanta 2018 Michael Godeck

slide-43
SLIDE 43

Michael Godeck DrupalCamp Atlanta 2018

Monte Carlo Simulations

Right sizing stories is half the battle.

Story Decomposition

It’s become popular to say that a good team doesn’t need estimation if they get story size right. That misses the point the right-sizing stories is estimation, and it’s as easy to get wrong as any

  • ther kind of estimation.
slide-44
SLIDE 44

Michael Godeck DrupalCamp Atlanta 2018

Bibliography