[PPT] - Formal Methods for Interactive Systems Part 10 Reverse PowerPoint Presentation

SLIDE 1

Formal Methods for Interactive Systems

Part 10 — Reverse Engineering with Matrix Algebra Antonio Cerone United Nations University International Institute for Software Technology Macau SAR China email: antonio@iist.unu.edu web: www.iist.unu.edu

A. Cerone, UNU-IIST – p.1/26

SLIDE 2

Reverse Engineering

Define a model of an existing device

A. Cerone, UNU-IIST – p.2/26

SLIDE 3

Reverse Engineering

Define a model of an existing device Danger: model accuracy relies on accuracy of the reverse engineering

A. Cerone, UNU-IIST – p.2/26

SLIDE 4

Reverse Engineering

Define a model of an existing device Danger: model accuracy relies on accuracy of the reverse engineering = ⇒ manifacturers do not provide formal specifications

A. Cerone, UNU-IIST – p.2/26

SLIDE 5

Reverse Engineering

Define a model of an existing device Danger: model accuracy relies on accuracy of the reverse engineering = ⇒ manifacturers do not provide formal specifications = ⇒ formal specifications must be reconstructed

A. Cerone, UNU-IIST – p.2/26

SLIDE 6

Reverse Engineering Process

Reconstruct formal specification from

checking interaction results against the real

device

A. Cerone, UNU-IIST – p.3/26

SLIDE 7

Reverse Engineering Process

Reconstruct formal specification from

checking interaction results against the real

device

observing user-machine interaction in the real
perating environemt
A. Cerone, UNU-IIST – p.3/26

SLIDE 8

Reverse Engineering Process

Reconstruct formal specification from

checking interaction results against the real

device

observing user-machine interaction in the real
perating environemt
interviewing real users and manifacturer
A. Cerone, UNU-IIST – p.3/26

SLIDE 9

Reverse Engineering Process

Reconstruct formal specification from

checking interaction results against the real

device

observing user-machine interaction in the real
perating environemt
interviewing real users and manifacturer
checking documentation (e.g., user manual)
A. Cerone, UNU-IIST – p.3/26

SLIDE 10

RE Advantages

model of a real system =

⇒ no need to specify it in a paper)

A. Cerone, UNU-IIST – p.4/26

SLIDE 11

RE Advantages

model of a real system =

⇒ no need to specify it in a paper)

results can be verified against the device by

the reader

A. Cerone, UNU-IIST – p.4/26

SLIDE 12

RE Advantages

model of a real system =

⇒ no need to specify it in a paper)

results can be verified against the device by

the reader

case studies are more credible ⇐

= not tailored for the used modelling / verification approach

A. Cerone, UNU-IIST – p.4/26

SLIDE 13

Matrix Algebra

to describe interaction human-machine

A. Cerone, UNU-IIST – p.5/26

SLIDE 14

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history

A. Cerone, UNU-IIST – p.5/26

SLIDE 15

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history = ⇒ no need to introduce yet another notation

are easy to calculate
A. Cerone, UNU-IIST – p.5/26

SLIDE 16

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history = ⇒ no need to introduce yet another notation

are easy to calculate
semantics reflects human way of thinking
A. Cerone, UNU-IIST – p.5/26

SLIDE 17

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history = ⇒ no need to introduce yet another notation

are easy to calculate
semantics reflects human way of thinking
have
A. Cerone, UNU-IIST – p.5/26

SLIDE 18

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history = ⇒ no need to introduce yet another notation

are easy to calculate
semantics reflects human way of thinking
have
structure =

⇒ easy modelling

A. Cerone, UNU-IIST – p.5/26

SLIDE 19

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history = ⇒ no need to introduce yet another notation

are easy to calculate
semantics reflects human way of thinking
have
structure =

⇒ easy modelling

properties =

⇒ verification tool

A. Cerone, UNU-IIST – p.5/26

SLIDE 20

Matrix Algebra

to describe interaction human-machine Matrices

are standard mathematical objects

with a long history = ⇒ no need to introduce yet another notation

are easy to calculate
semantics reflects human way of thinking
have
structure =

⇒ easy modelling

properties =

⇒ verification tool [Thimbleby 04]

A. Cerone, UNU-IIST – p.5/26

SLIDE 21

Example: Pushbutton Switch

A ligh bulb controlled by a pushbutton switch

A. Cerone, UNU-IIST – p.6/26

SLIDE 22

Example: Pushbutton Switch

A ligh bulb controlled by a pushbutton switch Pressing the switch alternatively turns the light on or off.

A. Cerone, UNU-IIST – p.6/26

SLIDE 23

Example: Pushbutton Switch

A ligh bulb controlled by a pushbutton switch Pressing the switch alternatively turns the light on or off.

2 states: ON and OFF
1 operation: push
A. Cerone, UNU-IIST – p.6/26

SLIDE 24

Example: Pushbutton Switch

A ligh bulb controlled by a pushbutton switch Pressing the switch alternatively turns the light on or off.

2 states: ON and OFF
1 operation: push

[Thimbleby 04]

A. Cerone, UNU-IIST – p.6/26

SLIDE 25

Model of Pushbutton Switch

✓ ✒ ✏ ✑

OFF

✬ ✩ ❄

push

✓ ✒ ✏ ✑

ON

✫ ✪ ✻

push

A. Cerone, UNU-IIST – p.7/26

SLIDE 26

Model of Pushbutton Switch

✓ ✒ ✏ ✑

OFF

✬ ✩ ❄

push

✓ ✒ ✏ ✑

ON

✫ ✪ ✻

push OFF = 0 , 1 ON = 1 , 0

A. Cerone, UNU-IIST – p.7/26

SLIDE 27

Model of Pushbutton Switch

✓ ✒ ✏ ✑

OFF

✬ ✩ ❄

push

✓ ✒ ✏ ✑

ON

✫ ✪ ✻

push OFF = 0 , 1 ON = 1 , 0 push = 1 1

A. Cerone, UNU-IIST – p.7/26

SLIDE 28

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1

A. Cerone, UNU-IIST – p.8/26

SLIDE 29

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1 OFF

A. Cerone, UNU-IIST – p.8/26

SLIDE 30

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1 OFF push

A. Cerone, UNU-IIST – p.8/26

SLIDE 31

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1 OFF push = 0 , 1 1 1

A. Cerone, UNU-IIST – p.8/26

SLIDE 32

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1 OFF push = 0 , 1 1 1 = 1 ,

☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ❦

A. Cerone, UNU-IIST – p.8/26

SLIDE 33

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1 OFF push = 0 , 1 1 1 = 1 , 0

☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ❦

A. Cerone, UNU-IIST – p.8/26

SLIDE 34

Pushbutton Switch Transition

OFF = 0 , 1 ON = 1 , 0 push = 1 1 OFF push = 0 , 1 1 1 = 1 , 0 = ON

A. Cerone, UNU-IIST – p.8/26

SLIDE 35

Switch Properties

OFF push = ON

A. Cerone, UNU-IIST – p.9/26

SLIDE 36

Switch Properties

OFF push = ON ON push = OFF

A. Cerone, UNU-IIST – p.9/26

SLIDE 37

Switch Properties

OFF push = ON ON push = OFF push push

A. Cerone, UNU-IIST – p.9/26

SLIDE 38

Switch Properties

OFF push = ON ON push = OFF push push = 1 1 1 1

A. Cerone, UNU-IIST – p.9/26

SLIDE 39

Switch Properties

OFF push = ON ON push = OFF push push = 1 1 1 1 = 1 1

A. Cerone, UNU-IIST – p.9/26

SLIDE 40

Switch Properties

OFF push = ON ON push = OFF push push = 1 1 1 1 = 1 1 = I

A. Cerone, UNU-IIST – p.9/26

SLIDE 41

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF

A. Cerone, UNU-IIST – p.10/26

SLIDE 42

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF ON x = 1 , 0 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF

A. Cerone, UNU-IIST – p.10/26

SLIDE 43

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF ON x = 1 , 0 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF x2,1 = 0 x2,2 = 1

A. Cerone, UNU-IIST – p.10/26

SLIDE 44

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF ON x = 1 , 0 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF x2,1 = 0 x2,2 = 1 x1,1 = 0 x1,2 = 1

A. Cerone, UNU-IIST – p.10/26

SLIDE 45

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF ON x = 1 , 0 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF x2,1 = 0 x2,2 = 1 x1,1 = 0 x1,2 = 1 = ⇒ x = 1 1

A. Cerone, UNU-IIST – p.10/26

SLIDE 46

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF ON x = 1 , 0 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF x2,1 = 0 x2,2 = 1 x1,1 = 0 x1,2 = 1 = ⇒ x = 1 1 ∃ n such that push

n

= x ?

A. Cerone, UNU-IIST – p.10/26

SLIDE 47

Switch Safety

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF ON x = 1 , 0 x1,1 x2,1 x1,2 x2,2 = 0 , 1 = OFF x2,1 = 0 x2,2 = 1 x1,1 = 0 x1,2 = 1 = ⇒ x = 1 1 ∃ n such that push

n

= x ? No because push

2

= I

A. Cerone, UNU-IIST – p.10/26

SLIDE 48

Safer System

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 1 , 0 = ON ON y = 1 , 0 y1,1 y2,1 y1,2 y2,2 = 0 , 1 = OFF

A. Cerone, UNU-IIST – p.11/26

SLIDE 49

Safer System

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 1 , 0 = ON ON y = 1 , 0 y1,1 y2,1 y1,2 y2,2 = 0 , 1 = OFF x2,1 = 1 x2,2 = 0 = ⇒ x = 1 0

-
A. Cerone, UNU-IIST – p.11/26

SLIDE 50

Safer System

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 1 , 0 = ON ON y = 1 , 0 y1,1 y2,1 y1,2 y2,2 = 0 , 1 = OFF x2,1 = 1 x2,2 = 0 = ⇒ x = 1 0 y1,1 = 0 y1,2 = 1 = ⇒ y = 0 1

-
-
A. Cerone, UNU-IIST – p.11/26

SLIDE 51

Safer System

OFF x = 0 , 1 x1,1 x2,1 x1,2 x2,2 = 1 , 0 = ON ON y = 1 , 0 y1,1 y2,1 y1,2 y2,2 = 0 , 1 = OFF x2,1 = 1 x2,2 = 0 = ⇒ x = 1 0 y1,1 = 0 y1,2 = 1 = ⇒ y = 0 1 1 0 =

n

0 1 =

ff
A. Cerone, UNU-IIST – p.11/26

SLIDE 52

Two-position Switch

A ligh bulb controlled by a two-position switch

A. Cerone, UNU-IIST – p.12/26

SLIDE 53

Two-position Switch

A ligh bulb controlled by a two-position switch One switch position (on) turns the light on; the other (off) turns the light off.

A. Cerone, UNU-IIST – p.12/26

SLIDE 54

Two-position Switch

A ligh bulb controlled by a two-position switch One switch position (on) turns the light on; the other (off) turns the light off. Properties

The system is closed
A. Cerone, UNU-IIST – p.12/26

SLIDE 55

Two-position Switch

A ligh bulb controlled by a two-position switch One switch position (on) turns the light on; the other (off) turns the light off. Properties

The system is closed
any sequence of actions is equivalent to the

last actions the user does

A. Cerone, UNU-IIST – p.12/26

SLIDE 56

Remarks

matrix algebra can be used to find interesting

properties

A. Cerone, UNU-IIST – p.13/26

SLIDE 57

Remarks

matrix algebra can be used to find interesting

properties through

direct manipulation of matrices (operations)

rather than vectors (states)

A. Cerone, UNU-IIST – p.13/26

SLIDE 58

Remarks

matrix algebra can be used to find interesting

properties through

direct manipulation of matrices (operations)

rather than vectors (states) ⇐ = number of operations much smaller than number of states

A. Cerone, UNU-IIST – p.13/26

SLIDE 59

Remarks

matrix algebra can be used to find interesting

properties through

direct manipulation of matrices (operations)

rather than vectors (states) ⇐ = number of operations much smaller than number of states

the methodology is
scalable
A. Cerone, UNU-IIST – p.13/26

SLIDE 60

Remarks

matrix algebra can be used to find interesting

properties through

direct manipulation of matrices (operations)

rather than vectors (states) ⇐ = number of operations much smaller than number of states

the methodology is
scalable
mechanisable [Gow and Thimbleby 04]
A. Cerone, UNU-IIST – p.13/26

SLIDE 61

Calculator

Model: CASIO HL-820LC
A. Cerone, UNU-IIST – p.14/26

SLIDE 62

Calculator

Model: CASIO HL-820LC
State
A. Cerone, UNU-IIST – p.14/26

SLIDE 63

Calculator

Model: CASIO HL-820LC
State
d = display contents
A. Cerone, UNU-IIST – p.14/26

SLIDE 64

Calculator

Model: CASIO HL-820LC
State
d = display contents
m = memory contents
A. Cerone, UNU-IIST – p.14/26

SLIDE 65

Calculator

Model: CASIO HL-820LC
State
d = display contents
m = memory contents
state representation: vector s = [d,m]
A. Cerone, UNU-IIST – p.14/26

SLIDE 66

Calculator

Model: CASIO HL-820LC
State
d = display contents
m = memory contents
state representation: vector s = [d,m]
Operations: binary 2×2 matrices
A. Cerone, UNU-IIST – p.14/26

SLIDE 67

Calculator Functions

M+ add display to memory
A. Cerone, UNU-IIST – p.15/26

SLIDE 68

Calculator Functions

M+ add display to memory
M− subtract display from memory
A. Cerone, UNU-IIST – p.15/26

SLIDE 69

Calculator Functions

M+ add display to memory
M− subtract display from memory
AC clear display
A. Cerone, UNU-IIST – p.15/26

SLIDE 70

Calculator Functions

M+ add display to memory
M− subtract display from memory
AC clear display
MRC recall memory
A. Cerone, UNU-IIST – p.15/26

SLIDE 71

Calculator Functions

M+ add display to memory
M− subtract display from memory
AC clear display
MRC recall memory
MRC MRC recall and clear memory
A. Cerone, UNU-IIST – p.15/26

SLIDE 72

MRC

What is the precise semantics of MRC?

pressed once
A. Cerone, UNU-IIST – p.16/26

SLIDE 73

MRC

What is the precise semantics of MRC?

pressed once
if m = 0 then no effect
A. Cerone, UNU-IIST – p.16/26

SLIDE 74

MRC

What is the precise semantics of MRC?

pressed once
if m = 0 then no effect
if m = 0 then d := m
A. Cerone, UNU-IIST – p.16/26

SLIDE 75

MRC

What is the precise semantics of MRC?

pressed once
if m = 0 then no effect
if m = 0 then d := m
pressed twice
A. Cerone, UNU-IIST – p.16/26

SLIDE 76

MRC

What is the precise semantics of MRC?

pressed once
if m = 0 then no effect
if m = 0 then d := m
pressed twice
m := 0
A. Cerone, UNU-IIST – p.16/26

SLIDE 77

Remarks on MRC

MRC MRC is not the same as the product

MRC · MRC

A. Cerone, UNU-IIST – p.17/26

SLIDE 78

Remarks on MRC

MRC MRC is not the same as the product

MRC · MRC

Semantics of pressed once depends on

previous content of memory

A. Cerone, UNU-IIST – p.17/26

SLIDE 79

Remarks on MRC

MRC MRC is not the same as the product

MRC · MRC

Semantics of pressed once depends on

previous content of memory

Is MRC ( MRC MRC )

= ( MRC MRC ) MRC ?

A. Cerone, UNU-IIST – p.17/26

SLIDE 80

Remarks on MRC

MRC MRC is not the same as the product

MRC · MRC

Semantics of pressed once depends on

previous content of memory

Is MRC ( MRC MRC )

= ( MRC MRC ) MRC ?

Calculator:

MRC ( MRC MRC ) = MRC MRC

A. Cerone, UNU-IIST – p.17/26

SLIDE 81

Bad Design?

Do we really need to give such a special

function to MRC MRC ?

A. Cerone, UNU-IIST – p.18/26

SLIDE 82

Bad Design?

Do we really need to give such a special

function to MRC MRC ?

No!

MRC MRC = MRC M−

A. Cerone, UNU-IIST – p.18/26

SLIDE 83

Bad Design?

Do we really need to give such a special

function to MRC MRC ?

No!

MRC MRC = MRC M−

MRC MRC is a bad design choice!
A. Cerone, UNU-IIST – p.18/26

SLIDE 84

Bad Design?

Do we really need to give such a special

function to MRC MRC ?

No!

MRC MRC = MRC M−

MRC MRC is a bad design choice!

[Thimbleby 00]

A. Cerone, UNU-IIST – p.18/26

SLIDE 85

Safety-critical

Is a calculator safety-critical?
A. Cerone, UNU-IIST – p.19/26

SLIDE 86

Safety-critical

Is a calculator safety-critical?
May a calculator-like interface be

safety-critical?

A. Cerone, UNU-IIST – p.19/26

SLIDE 87

Safety-critical

Is a calculator safety-critical?
May a calculator-like interface be

safety-critical?

Yes!

Syringe pump have calculator-like interfaces

A. Cerone, UNU-IIST – p.19/26

SLIDE 88

Examinations

A. Cerone, UNU-IIST – p.20/26

SLIDE 89

Seminar 6 — Usable Calculator

Topic: Calculator Interfaces Towards a truly usable calculator

Harold Thimbleby

Computer Algebra for User Interface Design, 2004

Harold Thimbleby

A Novel Pen-based Calculator and its Evaluation, 2004

Will Thimbleby and Harold Thimbleby

A Novel Gesture-based Calculator and its Design Principles, 2005

Websites on
Computer Algebra: http://www.cs.swan.ac.uk/∼csharold/CA/
Calculator: http://www.cs.swan.ac.uk/calculators/
A. Cerone, UNU-IIST – p.21/26

SLIDE 90

References

A. Cerone, UNU-IIST – p.22/26

SLIDE 91

Reverse Engineering

[Thimbleby 04]: Reverse Engineering and

Matrix Algebra

[Gow and Thimbleby 2004]: Matrix Algebra

and Tools

[Thimbleby 00]: Reverse Engineering
A. Cerone, UNU-IIST – p.23/26

SLIDE 92

[Thimbleby 04]

Harold Thimbleby. User interface design with matrix algebra. ACM Transactions on Computer-Human Interaction, Vol. 11, No. 2, 2004, pages 181–236. About:

Reverse Engineering
Matrix Algebra
A. Cerone, UNU-IIST – p.24/26

SLIDE 93

[Gow and Thimbleby 04]

Jeremy Gow and Harold Thimbleby. MAUI: An Interface Design Tool Based on Matrix Algebra. In Proceedings of CADUI 2004, Springer, 2004, pages 81–94. About:

Tools
A. Cerone, UNU-IIST – p.25/26

SLIDE 94

[Thimbleby 00]

Harold Thimbleby. Calculators are needlessly bad. International Journal of Computer-Human Studies, Vol. 52, No. 6, 2000, pages 1031–1069. About:

Reverse Engineering of Calculatos
A. Cerone, UNU-IIST – p.26/26