Chapter 10.1-10.3, Markov Chains I Review of Matrix Multiplication - - PowerPoint PPT Presentation

Chapter 10.1-10.3, Markov Chains I

Review of Matrix Multiplication on page 484 Transition Matrix Markov Chain Probability Vector Sample Problems: page 487 1, 10, 15, 28 Regular Chains Equilibrium Vector Summary on page 495 Sample Problem page 497, #17 Long Term Behaviour

Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 1 / 7

Chapter 10.1-10.3, Markov Chains II

Example (Problem 17, page 497)

  0.9 0.05 0.05 0.15 0.75 0.10 1  

Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 2 / 7

Chapter 10.1-10.3, Markov Chains III

Answer.

The matrix is not regular; it has an absorbing state. It is absorbing, because any state has positive probability to reach the absorbing state. The vector, [0, 0, 1], is an equilibrium vector. To compute all equilibrium vectors, solve the system v1 + v2 + v3 = 1 together with

• v1

v2 v3

• · P =
• v1

v2 v3

• :

           v1 + v2 + v3 = 1 −0.1v1 + 0.15v2 = 0 0.05v1 − 0.25v2 = 0 0.05v1 + 0.1v2 = 0 The only solution is [0, 0, 1].

Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 3 / 7

Absorbing Markov Chains

Definition on page 501 Summary on page 506 Fundamental Matrix Reviews on pages 505, 506 Sample Problem #23 Problems from Chapter Review.

1 Read the Summary. 2 Concept Check, # 1-14, 31-34. 3 Problems 16, 18, 20, 24, 27-30, 36, 38, 40, 42, 43-44, 49 Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 4 / 7

Long Range Trend for Problem 17, page 497

In standard form the matrix is   1 0.05 0.9 0.05 0.10 0.15 0.75   So R = 0.05 0.10

• and Q =

0.90 0.05 0.15 0.75

• . The fundamental matrix and the

long term trend are F = (I − Q)−1 = 100/7 20/7 60/7 40/7

• ,

FR = 1 1

• .

Since there is only one absorbing state, each state will end up in the absorbing state with probability 1; obvious conclusion. The matrix Q tells you how many times (on the average) the chain will spend in states 1 and 2 before being absorbed.

Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 5 / 7

Chapter 11.1, 11.2, Game Theory I

Payoff Matrix Dominated Strategies Strictly Determined Games Saddle Points (two ways of computing) Sample Problem page 525 #33 Mixed Strategies, Expected Value, Value of the Game Optimum Strategy, summary on page 532 Sample Problems, page 532 # 2, 16 Problems from the Review Section.

1 Read the Summary pertaining to 11.1-11.2. 2 Concept Check. # 1-12 3 Problems # 13-18, 20, 22, 24, 26, 29-36, 47-49, 60-62. Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 6 / 7

Problem 33, page 525

When A, B are in the same city, Player A get 65%. The payoff to A is 65 − 50 = 15. When A is in city 1, and B in city 2, the market shares are 1 2 3 A 0.8 × 0.3 0.2 × 0.45 0.6 × 0.25 B 0.2 × 0.3 0.8 × 0.45 0.4 × 0.25 The boxed numbers come from the text. The other ones complete the market in the given city. So the market share of A is 0.24 + 0.09 + 0.15 = 48. The payoff is 48 − 50 = −2.

Exercise

Complete the payoff matrix and check it equals   15 −2 6 7 15 9 3 −3 15  

Dan Barbasch Math 1105 Chapter 10-11, Review Week of November 27 7 / 7