On Domination and Control in Strategic Ability Micha l Knapik - - PowerPoint PPT Presentation

On Domination and Control in Strategic Ability

Micha l Knapik

Institute of Computer Science Polish Academy of Sciences (joint work with Wojtek Jamroga and Damian Kurpiewski)

TDCS Seminar, 30 May 2019

Outline

Enforceability in Concurrent Epistemic Game Structures Comparing Partial Strategies: Strategic Domination Applications: Model Checking Conclusions

2 / 17

Enforceability in CEGS

• Models: Concurrent Epistemic Game Structures
• Verif. properties: Alternating-time Temporal Logic (ATL) - enforceability
• A

Fgoal: coalition A has a collective strategy to enforce goal eventually. I.e., ∃σAσAFgoal.

• Difficult: NP-complete and no fixed-point algorithms!
• Our contributions:
• new method of comparing partial strategies
• application: alleviation of brute-force winning strategy synthesis
• application: strategy optimisation
• a step towards distributed and parallel synthesis

3 / 17

Concurrent Epistemic Game Structures

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

CEGS over Agt = {1, . . . , k} and Act:

• automaton with states St
• protocol di(q) ⊆ Act for agent i
• transition function o(q, α1, . . . , αk)

agent i selects action αi in q. . .

• labeling V (p) ⊆ St of states with

propositions

• ∼i indistinguishability relation for i

4 / 17

Concurrent Epistemic Game Structures - Strategies

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Uniform Strategy for agent i:

• σi : St → Act

s.t. q ∼i q′ = ⇒ σi(q) = σi(q′)

5 / 17

Concurrent Epistemic Game Structures - Strategies

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Uniform Strategy for agent i:

• σi : St → Act

s.t. q ∼i q′ = ⇒ σi(q) = σi(q′) Uniform Strategy for coalition A:

• set σA of uniform strat. for all i ∈ A

5 / 17

Concurrent Epistemic Game Structures - Strategies

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Uniform Strategy for agent i:

• σi : St → Act

s.t. q ∼i q′ = ⇒ σi(q) = σi(q′) Uniform Strategy for coalition A:

• set σA of uniform strat. for all i ∈ A

Outcome out(q, σA) of σA from q:

• all paths resulting from A following

σA and Agt \ A unrestricted

5 / 17

Concurrent Epistemic Game Structures - Strategies

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Uniform Strategy for agent i:

• σi : St → Act

s.t. q ∼i q′ = ⇒ σi(q) = σi(q′) Uniform Strategy for coalition A:

• set σA of uniform strat. for all i ∈ A

Outcome out(q, σA) of σA from q:

• all paths resulting from A following

σA and Agt \ A unrestricted

Example:

• let σ1(qinit) = σ1(q1) = σ1(q2) = A
• ∀π ∈ out(qinit, σ1) ∃iπi is red

so: qinit | = σ1Fred

5 / 17

Some Problems and Previous Results

• Checking

A Fgoal under no memory and imperfect knowledge is NP-complete (and ∆P

2 -complete for whole ATLir).

• Standard fixed-point ATL equivalences don’t work . . . and ATLir cannot

be embedded in AEµC = ⇒ no fixed-point procedures? [1,2]

• (Symbolic) algorithms and methods mostly based on brute-force. [3,4,5]
• Exception: in [6] an approximate fixed-point verification put forward (but

sometimes inconclusive).

• 1. Bulling, Jamroga: Alternating Epistemic Mu-Calculus, 2011.
• 2. Dima, Maubert, Pinchinat: Relating Paths in Transition Systems. . . , 2015.
• 3. Lomuscio, Raimondi: Automatic Verification of Multi-agent Systems by Model Checking. . . , 2007.
• 4. Busard et. al: Reasoning About Memoryless Strategies. . . , 2015.
• 5. Pilecki et. al: SMC: Synthesis of Uniform Strategies. . . , 2017.
• 6. Jamroga et. al: Approximate Verification of Strategic Abilities. . . , 2018.

6 / 17

Partial Strategies

• Partial Strategy for agent i: partial function that can be extended to

strategy for i. (Accord. σA for coalition A.)

7 / 17

Partial Strategies

• Partial Strategy for agent i: partial function that can be extended to

strategy for i. (Accord. σA for coalition A.)

• Partial strategies σA and σ′A are conflictless iff dom(σA) ∩ dom(σ′A) = ∅

and σA ∪ σ′A is a partial strategy.

7 / 17

Partial Strategies

• Partial Strategy for agent i: partial function that can be extended to

strategy for i. (Accord. σA for coalition A.)

• Partial strategies σA and σ′A are conflictless iff dom(σA) ∩ dom(σ′A) = ∅

and σA ∪ σ′A is a partial strategy.

• Fusion of conflictless partial strategies:

σA ∪ · σ′A = σA ∪ σ′A ∪ {assignments induced for all agents from A by σA ∪ σ′A}.

7 / 17

Partial Strategies

• Partial Strategy for agent i: partial function that can be extended to

strategy for i. (Accord. σA for coalition A.)

• Partial strategies σA and σ′A are conflictless iff dom(σA) ∩ dom(σ′A) = ∅

and σA ∪ σ′A is a partial strategy.

• Fusion of conflictless partial strategies:

σA ∪ · σ′A = σA ∪ σ′A ∪ {assignments induced for all agents from A by σA ∪ σ′A}.

• Notions of outcome, etc. adapted to partial strategies.

7 / 17

Partial Strategies, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example (part. strat. for 1):

• dom(σ1) = {qinit} with σ1(qinit) = A
• dom(σ′1) = {q2} with σ′1(q2) = A
• σ1 ∪

· σ′1 with dom(σ1 ∪ · σ′1) = {qinit, q1, q2}

8 / 17

Partial Strategies, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example (part. strat. for 1):

⋆ dom(σ1) = {qinit} with σ1(qinit) = A

• dom(σ′1) = {q2} with σ′1(q2) = A
• σ1 ∪

· σ′1 with dom(σ1 ∪ · σ′1) = {qinit, q1, q2}

8 / 17

Partial Strategies, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example (part. strat. for 1):

• dom(σ1) = {qinit} with σ1(qinit) = A

⋆ dom(σ′1) = {q2} with σ′1(q2) = A

• σ1 ∪

· σ′1 with dom(σ1 ∪ · σ′1) = {qinit, q1, q2}

8 / 17

Partial Strategies, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example (part. strat. for 1):

• dom(σ1) = {qinit} with σ1(qinit) = A
• dom(σ′1) = {q2} with σ′1(q2) = A

⋆ σ1 ∪ · σ′1 with dom(σ1 ∪ · σ′1) = {qinit, q1, q2}

8 / 17

Comparing Strategies: Strategic Domination

• Let σC

A and σA be conflictless. Call σC A context.

9 / 17

Comparing Strategies: Strategic Domination

• Let σC

A and σA be conflictless. Call σC A context.

• Call all states in dom(σA ∪

· σA) found along

q∈dom(σC

A ) out(q, σC

A)

inputs of σC

A into σA: I(σC A, σA).

(For technical reasons also qinit is an input if qinit ∈ dom(σA)).

9 / 17

Comparing Strategies: Strategic Domination

• Let σC

A and σA be conflictless. Call σC A context.

• Call all states in dom(σA ∪

· σA) found along

q∈dom(σC

A ) out(q, σC

A)

inputs of σC

A into σA: I(σC A, σA).

(For technical reasons also qinit is an input if qinit ∈ dom(σA)).

• Let q ∈ I(σC

A, σA). Outputs of σA (w.r.t. σC A) in q are states along

• ut(q, σA ∪

· σA) but not in dom(σA): IO(σC

A, σA)(q).

9 / 17

Comparing Strategies: Strategic Domination

• Let σC

A and σA be conflictless. Call σC A context.

• Call all states in dom(σA ∪

· σA) found along

q∈dom(σC

A ) out(q, σC

A)

inputs of σC

A into σA: I(σC A, σA).

(For technical reasons also qinit is an input if qinit ∈ dom(σA)).

• Let q ∈ I(σC

A, σA). Outputs of σA (w.r.t. σC A) in q are states along

• ut(q, σA ∪

· σA) but not in dom(σA): IO(σC

A, σA)(q).

Strategic domination

Let σC

A, σA and σC A, σ′A be conflictless. If I(σC A, σA) = I(σC A, σ′A) and for

each q ∈ I(σC

A, σA) we have IO(σC A, σ′A)(q) ⊆ IO(σC A, σA)(q) then σ′A

dominates σA w.r.t. σC

A:

σA σC

A σ′

A.

9 / 17

Comparing Strategies: Strategic Domination, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example:

• dom(σC

1 ) = {qinit} with

σC

1 (qinit) = A

• dom(σ1) = {q2} with σ1(q2) = A,

I(σC

1 , σ1) = {q1, q2},

IO(σC

1 , σ1)(q2) = {q3, q4},

IO(σC

1 , σ1)(q1) = {q4, q5}.

• dom(σ′1) = {q2} with σ′1(q2) = B,

I(σC

1 , σ′1) = {q1, q2},

IO(σC

1 , σ′1)(q2) = {q4},

IO(σC

1 , σ′1)(q1) = {q4}.

σA σC

A σ′A

10 / 17

Comparing Strategies: Strategic Domination, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example:

⋆ dom(σC

1 ) = {qinit} with

σC

1 (qinit) = A

• dom(σ1) = {q2} with σ1(q2) = A,

I(σC

1 , σ1) = {q1, q2},

IO(σC

1 , σ1)(q2) = {q3, q4},

IO(σC

1 , σ1)(q1) = {q4, q5}.

• dom(σ′1) = {q2} with σ′1(q2) = B,

I(σC

1 , σ′1) = {q1, q2},

IO(σC

1 , σ′1)(q2) = {q4},

IO(σC

1 , σ′1)(q1) = {q4}.

σA σC

A σ′A

10 / 17

Comparing Strategies: Strategic Domination, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example:

• dom(σC

1 ) = {qinit} with

σC

1 (qinit) = A

⋆ dom(σ1) = {q2} with σ1(q2) = A, I(σC

1 , σ1) = {q1, q2},

IO(σC

1 , σ1)(q2) = {q3, q4},

IO(σC

1 , σ1)(q1) = {q4, q5}.

• dom(σ′1) = {q2} with σ′1(q2) = B,

I(σC

1 , σ′1) = {q1, q2},

IO(σC

1 , σ′1)(q2) = {q4},

IO(σC

1 , σ′1)(q1) = {q4}.

σA σC

A σ′A

10 / 17

Comparing Strategies: Strategic Domination, ct’d

qinit start q2 q1 q4 q3 q5 (A, U) (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

Example:

• dom(σC

1 ) = {qinit} with

σC

1 (qinit) = A

• dom(σ1) = {q2} with σ1(q2) = A,

I(σC

1 , σ1) = {q1, q2},

IO(σC

1 , σ1)(q2) = {q3, q4},

IO(σC

1 , σ1)(q1) = {q4, q5}.

⋆ dom(σ′1) = {q2} with σ′1(q2) = B, I(σC

1 , σ′1) = {q1, q2},

IO(σC

1 , σ′1)(q2) = {q4},

IO(σC

1 , σ′1)(q1) = {q4}.

σA σC

A σ′A

10 / 17

Strategic Domination: Preserving Enforceability

Theorem

If:

• σC

A, σA and σC A, σ′A conflictless

• qinit ∈ dom(σC

A ∪

· σA) and qinit ∈ dom(σC

A ∪

· σ′A)

• σA σC

A σ′A

• no qinit–loops

then: qinit | = σC

A ∪

· σAFgoal implies qinit | = σC

A ∪

· σ′

AFgoal.

Deciding optimality of σA w.r.t. σC

A is co-NP-complete.

11 / 17

Switch Contexts for ’Global Optimisation’

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

12 / 17

Switch Contexts for ’Global Optimisation’

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

12 / 17

Switch Contexts for ’Global Optimisation’

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

12 / 17

Switch Contexts for ’Global Optimisation’

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

12 / 17

Switch Contexts for ’Global Optimisation’

qinit start q2 q1 q4 q3 q5 (A, U) (B, ⋆) (A, V ) (A, V ) (A, U) (B, ⋆) (B, ⋆) (A, U) (A, V ) 1

12 / 17

Model Checking via Simple(r) Strategy Synthesis

DominoDFS(σA)

Goal: find strategy σ′A extending σA s.t. qinit | = σ′AFgoal.

• 1. IF σAFgoal THEN EXIT(σA)
• 2. CURR := a state reachable via σA s.t. CURR |

= goal

• 3. CANDS := all non-dominated w.r.t. σA one-step strategies in CURR
• 4. FOREACH CAND in CANDS:
• 5. IF σA, CAND are conflictless AND σA ∪

· CAND is loopless THEN CALL DominoDFS(σA ∪ · CAND)

13 / 17

Model Checking: (Simplified) Bridge Endings

Can N-S win?

K DominoDFS MCMAS Approx.

• Approx. opt.

1 0.0006 0.12 0.0008 < 0.0001 2 0.01 8712 0.01 < 0.0001 3 0.8 timeout 0.8 0.06 4 160 timeout 384 5.5 5 1373 timeout 8951 39 6 memout timeout memout 4524 K cards / hand, avg. of 50 random hand deals / experiment.

14 / 17

Model Checking: Castles

Can castles 1 and 2 defeat 3?

(c1, c2, c3) DominoDFS MCMAS SMC (1, 1, 1) 0.3 65 63 (2, 1, 1) 1.5 12898 184 (3, 1, 1) 25 timeout 6731 (2, 2, 1) 25 timeout 4923 (2, 2, 2) 160 timeout timeout (3, 2, 2) 2688 timeout timeout (3, 3, 2) timeout timeout timeout Three castles, ci workers / castle i; each worker can defend or attack; need to pause a round after defense. (Approximate verifier can’t handle it: inconclusive results.)

15 / 17

Conclusions

• Conceptual framework for pruning space of strategies.
• Applications in model checking (and strategy optimisation paper).
• When verifying enforceability under imperfect knowledge first try

fixed-point approximations and then strategy pruning?

• Future work: full ATLir?
• Future work: parallelisation?

16 / 17

THANK YOU

17 / 17