CHAPTER 15: BARGAINING Multiagent Systems - - PowerPoint PPT Presentation

CHAPTER 15: BARGAINING Multiagent Systems http://www.csc.liv.ac.uk/˜mjw/pubs/imas/

Chapter 15 An Introduction to Multiagent Systems 2e

Overview

• How do agents reach agreements when they are self

interested?

• In an extreme case (zero sum encounter) no

agreement is possible — but in most scenarios, there is potential for mutually beneficial agreement on matters of common interest.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 1

Chapter 15 An Introduction to Multiagent Systems 2e

Overview

• The capabilities of:

– negotiation and – argumentation are central to the ability of an agent to reach such agreements.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 2

Chapter 15 An Introduction to Multiagent Systems 2e

Two pictures that summarise negotiation

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 3

Chapter 15 An Introduction to Multiagent Systems 2e

Mechanisms, Protocols, and Strategies

• Negotiation is governed by a particular mechanism, or

protocol.

• The mechanism defines the “rules of encounter”

between agents.

• Mechanism design is designing mechanisms so that

they have certain desirable properties. – Properties like Pareto efficiency

• Given a particular protocol, how can a particular

strategy be designed that individual agents can use?

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 4

Chapter 15 An Introduction to Multiagent Systems 2e

Auctions versus Negotiation

• Auctions are only concerned with the allocation of

goods: richer techniques for reaching agreements are required.

• Negotiation is the process of reaching agreements on

matters of common interest.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 5

Chapter 15 An Introduction to Multiagent Systems 2e

• Any negotiation setting will have four components:

– A negotiation set: possible proposals that agents can make. – A protocol. – Strategies, one for each agent, which are private. – A rule that determines when a deal has been struck and what the agreement deal is. Negotiation often proceeds in a series of rounds, with proposals at every round.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 6

Chapter 15 An Introduction to Multiagent Systems 2e

• There are a number of aspects of negotiation that

make it complex.

• Multiple issues

– Number of possible deals is exponential in the number of issues. (Like the number of bundles in a combinatorial auction) – Hard to compare offers across multiple issues The car salesman problem

• Multiple agents

– One-to-one negotiation

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 7

Chapter 15 An Introduction to Multiagent Systems 2e

– Many-to-one negotiation – Many-to-many negotiation

• At the simple end there isn’t much to distinguish

negotiation from auctions.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 8

Chapter 15 An Introduction to Multiagent Systems 2e

Negotiation for Resource Division

• We will start by looking at Rubinstein’s alternating
• ffers model.
• This is a one-to-one protocol.
• Agents are 1 and 2, and they negotiate over a series
• f rounds:

0, 1, 2, . . .

• In round 0, Agent 1 makes an offer x0.
• Agent 2 either accepts A, or rejects R.
• If the offer is accepted, then the deal is implemented.
• If not, we have round 1, and Agent 2 makes an offer.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 9

Chapter 15 An Introduction to Multiagent Systems 2e Agent 1 makes a proposal Agent 2 accepts Agent 2 rejects Agent 2 makes a proposal start Agent 1 rejects http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 10

Chapter 15 An Introduction to Multiagent Systems 2e

• The rules of the protocol don’t mean that agreement

will ever be reached. – Agents could just keep rejecting offers.

• If there is no agreement, we say the result is the

conflict deal Θ.

• We make the following basic assumptions:

– Disagreement is the worst ouctome Both agents prefer any agreement to none. – Agents seek to maximise utility Agents prefer to get larger utility values

• With this basic model, we get some odd results.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 11

Chapter 15 An Introduction to Multiagent Systems 2e

• Consider we are dividing a pie. . .

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 12

Chapter 15 An Introduction to Multiagent Systems 2e

• Model this as some resource with value 1, that is

divided into two parts. – Each part is between 0 and 1. – The two parts sum to 1 so a proposal is (x, 1 − x)

• The set of possible deals is:

{(x, 1 − x) : 0 ≤ x ≤ 1}

• If you are Agent 1, what do you offer?

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 13

Chapter 15 An Introduction to Multiagent Systems 2e

• Let’s assume that we will only have one round.

Ultimatum game

• Agent 1 has all the power.
• If Agent 1 proposes (1, 0), then this is still better for

Agent 2 than the conflict deal.

• Agent 1 can do no better than this either.
• So we have a Nash equilibrium.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 14

Chapter 15 An Introduction to Multiagent Systems 2e

• If we have two rounds, the power passes to Agent 2.
• Whatever Agent 1 proposes, Agent 2 rejects it.
• Then Agent 2 proposes (0, 1).
• Just as before this is still better for Agent 1 than the

conflict deal and so it is accepted.

• A bit of thought shows that this will happen any time

there is a fixed number of rounds.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 15

Chapter 15 An Introduction to Multiagent Systems 2e

• What if we have an indefinite number of rounds.
• Let’s say that Agent 1 uses this strategy:

Always propose (1, 0) and always reject any offer from Agent 2

• How should Agent 2 respond?
• If she rejects, then there will never be agreement.

– Conflict deal

• So accept. And there is no point in not accepting on

the first round.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 16

Chapter 15 An Introduction to Multiagent Systems 2e

• In fact, whatever (x, 1 − x) agent 1 proposes here,

immediate acceptance is the Nash equilibrium so long as Agent 2 knows what Agent 1’s strategy is.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 17

Chapter 15 An Introduction to Multiagent Systems 2e

Impatient players

• Since we have an infinite number of Nash equilibria,

the solution concept of NE is too weak to help us.

• Can get unqiue results if we take time into account.

For any outcome x and times t2 > t1, both agents prefer x at time t1.

• A standard way to model this impatience is to

discount the value of the outcome.

• Each agent has δi, i ∈ {1, 2}, where 0 ≤ δ < 1.
• The closer δi is to 1, the more patient the agent is.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 18

Chapter 15 An Introduction to Multiagent Systems 2e

• If agent i is offered x, then the value of the slice is:

– x at time 0 – δix at time 1 – δ2

i x at time 2.

. . . – δkx at time k

• Now we can make some progress with the fixed

number of rounds.

• A 1 round game is still an ultimatum game.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 19

Chapter 15 An Introduction to Multiagent Systems 2e

• A 2 round game means Agent 2 can play as before,

but if so, will only get δ2. Gets the whole pie, but it is worth less.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 20

Chapter 15 An Introduction to Multiagent Systems 2e

• Agent 1 can take this into account.
• If Agent 1 offers:

(1 − δ2, δ2) then Agent 2 might as well accept — can do no better.

• So this is now a Nash equilibrium.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 21

Chapter 15 An Introduction to Multiagent Systems 2e

• In the general case, agent 1 makes the proposal that

gives Agent 2 what Agent 2 would be able to enforce in the second round.

• Agent 1 gets:

1 − δ2 1 − δ1δ2

• Agent 2 gets:

δ2(1 − δ1) 1 − δ1δ2

• Note that the more patient either agent is, the more

pie they get.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 22

Chapter 15 An Introduction to Multiagent Systems 2e

Heuristic approach

• The approach we just talked about relies on strageic

thinking about the other player.

• A simpler approach is to use some heuristic

approximation of how the value of the pie varies for the players.

• Some common approximations:

– Linear – Boulware – Conceder

• We can see what these look like for buyers.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 23

Chapter 15 An Introduction to Multiagent Systems 2e http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 24

Chapter 15 An Introduction to Multiagent Systems 2e

• Linear

– Linear increase from initial price at the start time to reserve price at the deadline.

• Boulware

– Very slow increase until close to deadline and then an exponential increase.

• Conceder

– Inital exponential increase to close to the reserve price and then not much change.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 25

Chapter 15 An Introduction to Multiagent Systems 2e

Negotiation in Task-Oriented Domains

Imagine that you have three children, each of whom needs to be delivered to a different school each morning. Your neighbour has four children, and also needs to take them to school. Delivery

• f each child can be modelled as an indivisible task. You and your neighbour can discuss the

situation, and come to an agreement that it is better for both of you (for example, by carrying the

• ther’s child to a shared destination, saving him the trip). There is no concern about being able

to achieve your task by yourself. The worst that can happen is that you and your neighbour won’t come to an agreement about setting up a car pool, in which case you are no worse off than if you were alone. You can only benefit (or do no worse) from your neighbour’s tasks. Assume, though, that one of my children and one of my neigbours’s children both go to the same school (that is, the cost of carrying out these two deliveries, or two tasks, is the same as the cost of carrying out one of them). It obviously makes sense for both children to be taken together, and

• nly my neighbour or I will need to make the trip to carry out both tasks.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 26

Chapter 15 An Introduction to Multiagent Systems 2e

TODs Defined

• A task-oriented domain (TOD) is a triple

T, Ag, c where: – T is the (finite) set of all possible tasks; – Ag = {1, . . . , n} is set of participant agents; – c : ℘(T) → R+ defines cost of executing each subset of tasks:

• An encounter is a collection of tasks

T1, . . . , Tn where Ti ⊆ T for each i ∈ Ag.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 27

Chapter 15 An Introduction to Multiagent Systems 2e

Deals in TODs

• Given encounter T1, T2, a deal will be an allocation
• f the tasks T1 ∪ T2 to the agents 1 and 2.
• The cost to i of deal δ = D1, D2 is c(Di), and will be

denoted costi(δ).

• The utility of deal δ to agent i is:

utilityi(δ) = c(Ti) − costi(δ).

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 28

Chapter 15 An Introduction to Multiagent Systems 2e

• The conflict deal, Θ, is the deal T1, T2 consisting of

the tasks originally allocated. Note that utilityi(Θ) = 0 for all i ∈ Ag

• Deal δ is individual rational if it gives positive utility.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 29

Chapter 15 An Introduction to Multiagent Systems 2e

The Negotiation Set

• The set of deals over which agents negotiate are

those that are: – individual rational – pareto efficient.

• Individually rational: agents won’t be interested in

deals that give negative utility since they will prefer the conflict deal.

• Pareto efficient: agents can always transform a

non-Pareto efficient deal into a Pareto efficient deal by making one agent happier and none of the others worse off.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 30

Chapter 15 An Introduction to Multiagent Systems 2e

The Negotiation Set Illustrated

from B to C are utility for agent j utility for agent i utility of conflict deal for j utility of conflict deal for i deals on this line Pareto optimal, hence in the negotiation set this circle delimits the possible deals space of all conflict deal A B C D E

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 31

Chapter 15 An Introduction to Multiagent Systems 2e

The Monotonic Concession Protocol Rules of this protocol are as follows. . .

• Negotiation proceeds in rounds.
• On round 1, agents simultaneously propose a deal

from the negotiation set.

• Agreement is reached if one agent finds that the deal

proposed by the other is at least as good or better than its proposal.

• If no agreement is reached, then negotiation proceeds

to another round of simultaneous proposals.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 32

Chapter 15 An Introduction to Multiagent Systems 2e

• In round u + 1, no agent is allowed to make a proposal

that is less preferred by the other agent than the deal it proposed at time u.

• If neither agent makes a concession in some round

u > 0, then negotiation terminates, with the conflict deal.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 33

Chapter 15 An Introduction to Multiagent Systems 2e

The Zeuthen Strategy Three problems:

• What should an agent’s first proposal be?

Its most preferred deal

• On any given round, who should concede?

The agent least willing to risk conflict.

• If an agent concedes, then how much should it

concede? Just enough to change the balance of risk.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 34

Chapter 15 An Introduction to Multiagent Systems 2e

Willingness to Risk Conflict

• Suppose you have conceded a lot. Then:

– Your proposal is now near to conflict deal. – In case conflict occurs, you are not much worse off. – You are more willing to risk confict.

• An agent will be more willing to risk conflict if the

difference in utility between its current proposal and the conflict deal is low.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 35

Chapter 15 An Introduction to Multiagent Systems 2e

Nash Equilibrium Again. . . The Zeuthen strategy is in Nash equilibrium: under the assumption that one agent is using the strategy the

• ther can do no better than use it himself. . .

This is of particular interest to the designer of automated agents. It does away with any need for secrecy on the part of the programmer. An agent’s strategy can be publicly known, and no other agent designer can exploit the information by choosing a different strategy. In fact, it is desirable that the strategy be known, to avoid inadvertent conflicts.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 36

Chapter 15 An Introduction to Multiagent Systems 2e

Deception in TODs Deception can benefit agents in two ways:

• Phantom and Decoy tasks.

Pretending that you have been allocated tasks you have not.

• Hidden tasks.

Pretending not to have been allocated tasks that you have been.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 37

Chapter 15 An Introduction to Multiagent Systems 2e

Summary

• This lecture has looked at different mechanisms for

reaching agreement between agents.

• We started by looking at negotiation, where agents

make concessions and explore tradeoffs.

• Finally, we looked at argumentation, which allows for

more complex interactions and can be used for a range of tasks that include negotiation.

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/ 38