Optimal Non-parametric Learning in Repeated Contextual Auctions with - - PowerPoint PPT Presentation

Optimal Non-parametric Learning in Repeated Contextual Auctions with Strategic Buyer

Alexey Drutsa

Setup

Repeated Contextual Posted-Price Auctions

Different goods (e.g., ad spaces)

› described by 𝑒-dimensional feature vectors (contexts) from 0,1 % › are repeatedly offered for sale by a seller › to a single buyer over 𝑈 rounds (one good per round).

The buyer

› holds a private fixed valuation function 𝑤: 0,1 % → 0,1 › used to calculate his valuation 𝑤(𝑦) for a good with context 𝑦 ∈ 0,1 %, › 𝑤 is unknown to the seller.

At each round 𝑢 = 1, … , 𝑈,

› a feature vector 𝑦2 of the current good is observed by the seller and the buyer › a price 𝑞2 is offered by the seller, › and an allocation decision 𝑏2 ∈ {0,1} is made by the buyer:

𝑏2 = 0, when the buyer rejects, and 𝑏2 = 1, when the buyer accepts.

Seller’s pricing algorithm and buyer strategy

The seller applies a pricing algorithm 𝐵 that sets prices {𝑞2}289

:

in response to buyer decisions 𝐛 = {𝑏2}289

:

and observed contexts 𝐲 = {𝑦2}289

:

. The price 𝑞2 can depend only on

› past decisions {𝑏=}=89

2>9

› feature vectors {𝑦=}=89

2

› the horizon 𝑈

Strategic buyer

▌ The seller announces her pricing algorithm 𝐵 in advance

The buyer has some distribution (beliefs) 𝐸 about future contexts. In each round 𝑢, given the history of previous rounds, he chooses his decision 𝑏2 s.t. it maximizes his future 𝛿-discounted surplus: 𝔽BC~E F 𝛿=>9𝑏=(𝑤(𝑦=) − 𝑞=)

: =82

, 𝛿 ∈ (0,1]

pu publ blic

kn knowle ledge

pr priva vate

kn knowle ledge before game starts

Se Seller

Al Algo gorithm hm

Na Nature Buye Buyer Al Algo gorithm

round 𝑢 = 1

𝑦9 𝑞9 𝑏9 𝐸

Al Algo gorithm hm

Na Nature Buye Buyer

round 𝑢 = 2

𝑦J 𝑞J 𝑏J

Al Algo gorithm hm

Na Nature Buye Buyer

round 𝑢 = 3

𝑦L 𝑞L 𝑏L 𝑤 𝐸 𝑤 𝐸 𝑤

The game’s workflow and knowledge structure

Seller’s goal

The seller’s strategic regret: SReg 𝑈, 𝐵, 𝑤, 𝛿, 𝑦9::, 𝐸 : = ∑ (𝑤(𝑦2) − 𝑏2

RST𝑞2) : 289

We will learn the function 𝑤 in a non-parametric way. For this, we will assume that it is Lipschitz (a standard requirement for non-parametric learning): LipX 0,1 % ≔ 𝑔: 0,1 % → 0,1 |∀𝑦, 𝑧 ∈ 0,1 % 𝑔 𝑦 − 𝑔 𝑧 ≤ 𝑀 𝑦 − 𝑧 The seller seeks for a no-regret pricing for worst-case valuation function: supb∈cdSe

f,9 g ,Bh:i,ESReg 𝑈, 𝐵, 𝑤, 𝛿, 𝑦9::, 𝐸 = 𝑝 𝑈

Optimality: the lowest possible upper bound for the regret of the form 𝑃 𝑔(𝑈) .

Background & Research question

Background

[Kleinberg et al., FOCS’2003] Non-contextual setup (𝑒 = 0). Horizon-dependent optimal algorithm against myopic buyer (𝛿 = 0) with truthful regret Θ(log log 𝑈). [Mao et al., NIPS’2018] Our non-parametric contextual setup (𝑒 > 0). Horizon-dependent optimal algorithm against myopic buyer (𝛿 = 0) with truthful regret Θ(𝑈

g gph).

[Drutsa, WWW’2017] Non-contextual setup (𝑒 = 0). Horizon-independent optimal algorithm against strategic buyer with regret Θ(log log 𝑈) for 𝛿 < 1. [Amin et al., NIPS’2013] Non-contextual setup (𝑒 = 0). The strategic setting is introduced. ∄ no-regret pricing for non-discount case 𝛿 = 1.

Research question

The key approaches of the non-contextual optimal algorithms ([pre]PRRFES) cannot be directly applied to contextual algorithm of [Mao et al., NIPS’2018] In order to search the valuation of the strategic buyer without context:

› Penalization rounds are used › We do not propose prices below the ones that are earlier accepted

In the approach of [Mao et al., NIPS’2018]:

› Standard penalization does not help › Proposed prices can be below the ones that are earlier accepted by the buyer

▌ In this study, I overcome these issues and propose an optimal ▌ non-parametric algorithm for the contextual setting with strategic buyer

Novel optimal algorithm

Penalized Exploiting Lipschitz Search (PELS)

PELS has three parameters:

› the price offset 𝜃 ∈ 1, +∞ › the degree of penalization 𝑠 ∈ ℕ › the exploitation rate 𝑕: ℤz → ℤz

This algorithm keeps track of

› a partition 𝔜 of the feature domain 0,1 % › initialized to 4𝜃 + 6 𝑀 % cubes (boxes) with side length 𝑚 = 1/ 4𝜃 + 6 𝑀 :

𝔜 = 𝐽9 × 𝐽J × ⋯ × 𝐽% | 𝐽9, 𝐽J, … , 𝐽% ∈ 0, 𝑚 , 𝑚, 2𝑚 , … , 1 − 𝑚, 1

% .

Penalized Exploiting Lipschitz Search (PELS)

For each box 𝑌 ∈ 𝔜, PELS also keeps track of:

› the lower bound 𝑣† ∈ [0,1], › the upper bound 𝑥† ∈ [0,1], › the depth 𝑛† ∈ ℤz.

They are initialized as follows: 𝑣† = 0, 𝑥† = 1, and 𝑛† = 0, 𝑌 ∈ 𝔜. The workflow of the algorithm is organized independently in each box 𝑌 ∈ 𝔜.

› the algorithm receives a good with a feature vector 𝑦2 ∈ 0,1 % › finds the box 𝑌 ∈ 𝔜 in the current partition 𝔜 s.t. 𝑦2 ∈ 𝑌.

▌ Then, the proposed price 𝑞2 is determined only from the current state ▌ associated with the box 𝑌, while the buyer decision 𝑏2 is used ▌ only to update the state associated with this box 𝑌.

Penalized Exploiting Lipschitz Search (PELS)

In each box 𝑌 ∈ 𝔜, the algorithm iteratively offers exploration price: 𝑣† + 𝜃𝑀diam(𝑌)

▌ If this price is accepted by the buyer:

› the lower bound 𝑣† is increased by 𝑀diam(𝑌).

▌ If this price is rejected:

› the upper bound 𝑥† is decreased by 𝑥† − 𝑣† − 2(𝜃 + 1)𝑀diam(𝑌) › 1 is offered as a penalization price for 𝑠 − 1 next rounds in this box 𝑌

(if one of them is accepted, we continue offering 1 all the remaining rounds).

Penalized Exploiting Lipschitz Search (PELS)

▌ If, after an acceptance of an exploration price or after penalization rounds

we have 𝑥† − 𝑣† < (2𝜃 + 3)𝑀diam(𝑌),

▌ then PELS:

› offers the exploitation price 𝑣† for 𝑕(𝑛†) next rounds in this box 𝑌

(buyer decisions made at them do not affect further pricing);

› bisects each side of the box 𝑌 to obtain 2% boxes 𝔜† ≔ 𝑌9, … , 𝑌Jg

with ℓŽ-diameter equal to diam(𝑌)/2;

› refines the partition 𝔜† replacing the box 𝑌 by the new boxes 𝔜†.

These new boxes 𝔜†

› inherit the state of the bounds 𝑣† and 𝑥† from the current state of 𝑌, › while their depth 𝑛• = 𝑛† + 1 ∀𝑍 ∈ 𝔜†.

PELS is optimal

Theorem 1. Let 𝑒 ≥ 1 and 𝛿f ∈ 0,1 . Then for the pricing algorithm PELS 𝐵 with:

› the number of penalization rounds 𝑠 ≥ log’“

9>’“ J

› the exploitation rate 𝑕 𝑛 = 2”, 𝑛 ∈ ℤz, › the price offset 𝜃 ≥ 2/(1 − 𝛿f)

for any valuation function 𝑤 ∈ LipX 0,1 % , discount 𝛿 ≤ 𝛿f, distribution 𝐸 and feature vectors 𝑦9::, the strategic regret is upper bounded: SReg 𝑈, 𝐵, 𝑤, 𝛿, 𝑦9::, 𝐸 ≤ 𝐷 𝑂f 𝑈 + 𝑂f %

9 %z9 = Θ(𝑈 % %z9),

𝐷 ≔ 2%𝑠 2𝜃 + 3 + 𝑀>9 + 1 and 𝑂f ≔ 4𝜃 + 6 𝑀 %.

PELS: main properties and extensions

› Can be applied against myopic buyer (𝛿 = 0) (setup of [Mao et al., NIPS’2018]) › PELS is horizon-independent (in contrast to [Mao et al., NIPS’2018])

▌ What if the loss is symmetric?

› We can generalize the algorithm to classical online learning losses › For instance, we want to optimize regret of the form ∑

|𝑤(𝑦2) − 𝑞2|

: 289

› But interacting with the strategic buyer still › Slight modification of PELS has regret 𝑃(𝑈

g—h g ), which is tight for 𝑒 > 1.

adrutsa@yandex.ru

Thank you!

Alexey Drutsa Yandex