Use of Markov Chains to Design an Agent Bidding Strategy for - - PowerPoint PPT Presentation

Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions

Sunju Park Management Science and Information Systems Department Rutgers Business School, Rutgers University Edmund H. Durfee Artificial Intelligence Laboratory, University of Michigan William P. Birmingham Math & Computer Science Department, Grove City College

Presenter: TinTin Yu {tiyu@mtu.edu}

Introduction

Not like tradition auctions

Single seller and multiple buyers (e.g. eBay)

Continuous Double Auctions (CDA)

Buyers place bids, and sellers place offers to the same items. We have a match whenever a buyer’s bid is higher than a

seller’s offer.

(e.g. Name your price (hotel.com?)

Goal

To determine the optimal price/offer for a seller in order to gain

the maximum profit.

Definitions

Notation: bbssp

b: buyer’s bid; s: seller’s offer sp: seller’s offer that was just submitted bbssp: a queue in ascending order (of price)

Clearing Price (CP)

bspbs: When an offer is less than a bid sp<=CP<=b (the right most b) We use sp in this paper.

Definitions

Markov Chains (Markov state machine)

Probabilistic finite state machine Input is ignored We uses first-order Markov chain only

First-order means the probability of the present state is a

function only to its direct predecessor states.

p-strategy Algorithm (1/2)

p-strategy Algorithm (2/2)

Information used by p-strategy

Step1: Building Markov Chains (1/3)

Given a current state (bbs).

When the p-seller (a seller use p-strategy) submit its offer sp,

there are four possible next auction states.

We make these states the initial states of the Markov Chain.

Step1: Building Markov Chains (2/3)

• From the initial states, we keep populate the (bbss) queue by either

submitting a new buyer bid or a seller offer.

• If we have a match, it goes to the SUCCESS state.
• If it goes out of the bound (maximum number of standing offers), it goes to

the FAIL state.

Step1: Building Markov Chains (3/3)

• The MC model of the CDA with starting state (bbs) and the number of bids

and offers are limited to 5 each.

Step2: Compute Utilities (1/5)

Step2.1: The utilities function

Ps(p): probability of success at price p U(Payoffs(p)): utilities of payoff if the offer receives a match CP: clearing price C: cost TD(Δs/f): delay overhead

Step2: Compute Utilities (2/5)

Things we need to compute for each p

Step2: Compute Utilities (3/5)

Step2.2.1: Transition Probabilities

Going from state (bbs) to (bbssp) at time step n That is P(bbssp | bbs); Applying Baye’s rule; Evaluating using probability density function

(PDF), f(s); bababa…

Step2: Compute Utilities (4/5)

Step2.2.2: TD(Δs/f): delay overhead

Too complex to cover in details It involves building a transition probability matrix P from the states of the

Markov Chain we built in step1.

Here is listed equations: ω: reward = c (a constant) except for the initial states and the absorbing

states

μ: the number of visits to state (…) until it goes to S.

Step2: Compute Utilities (5/5)

Plug in the numbers and we will get a expected utility value

associated with price p.

The algorithm find the optimal price p by looping through all p in a

possible range.

Time complexity of the algorithm is O(ρ n3), where ρ is the number of

possible prices, n is the number of MC states.

Benchmark (1/6)

Agents used for comparison

FM: Fixed-Markup

bids its cost plus some predefined markup

RM: Random-Markup

bids its cost plus some random markup

CP: Clearing-Price

• btains a clearing-price quote (similar to FM agent)

OPT: Post-facto Optimal

• ur benchmark strategy. Given it “knows” exactly everything about

the future (no uncertainty at all), it returns the maximum profit an agent may have achieved.

Benchmark (2/6)

Benchmark (3/6): p-strategy vs other

Results: Arrival rate:

0.4=high 0.1=low

negotiation zone

narrow: =5

Benchmark (4/6): p-strategy vs other

Results: Arrival rate:

0.4=high 0.1=low

negotiation zone

narrow: =25

Benchmark (5/6): p-strategy vs itself

Results Profit of individual

p-agent decrease as the number

• f p-agents increase.

However, when there

is more buyers, p-agents are able to gain similar profit at the expense of buyers.

Benchmark (6/6):

CP vs multiple p and CP

Results CP-strategy agents are

able to raises profit as the number mixed p-agents and CP-agents increase.

Conclusion

Summary:

p-strategy is based on stochastic modeling of the auction process. It works while it does not need to consider much about the other

individual agents. Time complexity only depends on the number of MC states, not the number of agents.

It out performs other agents (FM/ RM/ CP)

Future Work

Similar strategy can be apply to buyers. Analysis shows an average of 20% gap between p-strategy and the

• ptimal one.

Ongoing work: hybrid strategy. This adaptive approach allow the agent

to figure out when to use stochastic model and when to use some simpler strategies.

Question to think about

Human can think very differently:

e.g. Selling a 50” plasma HDTV

Place a very low selling price like $1.00 without a

hidden limit.

Shipping cost = $3000.00 ?!

Can artificial intelligent agents think

• utside the box?

Your Questions

Bibliography

• Park, S., Durfee, E.H. and Birmingham, W.P. (2004) "Use of Markov Chains to

Design an Agent Bidding Strategy for Continuous Double Auctions", Volume 22, pages 175-214.