Learning Based Auction for Cognitive Radio Networks AK Oloyede and - - PowerPoint PPT Presentation

AK Oloyede and David Grace Communications Research Group Department of Electronics University of York 10th October 2013

Learning Based Auction for Cognitive Radio Networks

Outline

n Introduction n Cognitive Radio / Dynamic Spectrum Access n Auction Based Dynamic Spectrum Access n Bid Learning Model n Result n Future Work n Conclusions

2

Introduction

Traffic on mobile systems is growing rapidly

n

Global mobile data grew by 70% in 2012

n

Mobile video traffic exceeded 50 % for the first time in 2012

n

Average smartphone usage grew 81% in 2012

n

There were 161 million laptops on the mobile network in 2012, and each laptop generated 7 times more traffic than the average smartphone

n

Mobile network connection speeds more than doubled in 2012

Trend continues

n

Global mobile data traffic will increase 13-fold between 2012 and 2017. Mobile data traffic will grow at a compound annual growth rate (CAGR) of 66 % from 2012 to 2017, reaching 11.2 EB per month by 2017

n

The number of mobile-connected devices will exceed world’s population in 2013 and by 2017 there will be nearly 1.4 mobile devices per capita.

n

The Middle East and Africa will have the strongest mobile data traffic growth of any region at 77 %

• CAGR. This region will be followed by Asia Pacific at 76% and Latin America at 67%
• Source: Cisco report on major global mobile data traffic projections and growth trends

3

Implications of Mobile Traffic Growth

n Network operators would be forced to find alternatives n Dynamic Spectrum Access(DSA) and Spectrum sharing n Deployment of Cognitive Radio System n It might lead to more delay on the mobile network

leading to WWW(World Wide Wait)

4

Cognitive Radio and DSA

n Cognitive Radio systems are defined as wireless systems

that are designed to interact and observe the transmission environment

n Cognitive Radio would provide dynamic access to the radio

spectrum

n Access to the radio spectrum using DSA requires a fair

means of allocation. Thus, the use of an auction provides a fair allocation process by granting access to the highest bidder(s)

5

Revenue Growth and Data Traffic Growth

6

Source: Analysis of operator options to reduce the impact of the revenue gap caused by flat rate mobile broadband subscription By B. Molleryd, J. Markendahl and O.Makitalo

Auction

n An auction is a process of buying and selling by offering them up for

bid, taking bids, and then selling the item to the highest bidder (Wikipedia)

n There are different types of Auction process

n First price auction n Second Price Auction n sealed bid auction n Discriminatory auction n Uniform Auction single unit, multiple unit etc

n Auctions that maximize sellers revenue are referred to as optimal

and those that maximize social welfare are referred to as efficient.

7

DSA Architecture

12

8

1 1.4 1.8 1.3 1.2 1.7 1.5

1.8 1.7 1.5 1.4 1.3 1.2 1 1 2 RP=1.75

The Users Bid

9

​𝑐↓𝑛𝑗𝑜 =𝑁𝑗𝑜𝑗𝑛𝑣𝑛 ¡𝑞𝑝𝑡𝑡𝑗𝑐𝑚𝑓 ¡𝑐𝑗𝑒 ​𝑐↓𝑛𝑏𝑦 =𝑁𝑏𝑦𝑗𝑛𝑣𝑛 ¡𝑞𝑝𝑡𝑡𝑗𝑐𝑚𝑓 ¡𝑐𝑗𝑒 ¡

The Reserve Price

10

Transmi(ng ¡period ¡

The Bidding and Transmitting Period

n The bidding period is the time window allowed for the users who

intend to transmit on the spectrum to submit a bid

• Long bidding period would introduce more delay into the system

11

n The bidding period is dependent on the traffic load. n The transmission period is longer than the bidding period

Bidding ¡Period ¡

¡ ¡

Bidding ¡Period ¡

¡ ¡

Transmi(ng ¡period ¡

Bidding ¡Period ¡

¡ ¡

Transmi(ng ¡period ¡

Bidding ¡Period ¡

¡ ¡

Transmi(ng ¡period ¡

Learning Based Models

n Q Reinforcement Learning (QL)

​ ¡ ¡ ¡Q↑π (s,a)=R+γ​max┬a ⁠Q(​s↑′ ,​a↑′ )

n Bayesian framework for Rainforcement Learning (BRL)

¡P(A\B)=​P(B\A)×P(A)/P(B) ¡

Where P(A) is the prior probability distribution of hypothesis A, P(B) is the probability of the training data B (likelihood) and P(B\A) is the probability of A given B (Posterior probability)

12

The Bid Learning Model

13

n The Utility Function

Assumptions

n

We assume that users are price sensitive and therefore want to win the bid with the least possible amount

n

The utility function measures how much a winning bidder deviates from the lowest winning bid

n

The lower the value of the winning bid the higher the value of utility

n

If the user is not among the winning bidders then the utility is zero 𝑵={​𝒄↓ 𝒄↓𝟐 ,​𝒄↓ 𝒄↓𝟑 ,​𝒄↓ 𝒄↓𝟒 …​𝒄↓𝒏 𝒄↓𝒏 }

where 𝜺=​𝒏𝒋𝒐

𝒏𝒋𝒐⁠𝑵 ​𝒗↓ 𝒗↓𝒋 (𝒖)= ¡ ¡ ¡ ¡ ¡{█□​𝟑↑​𝜺/ 𝜺/​𝒄↓𝒋 𝒄↓𝒋 −𝟐@𝟏 █□ ¡ ¡ ¡ ¡𝑮𝒑𝒔 𝑮𝒑𝒔 ¡𝒙𝒋𝒐𝒐𝒋 𝒐𝒐𝒋𝒐𝒉 𝒐𝒉 ¡𝒄𝒋𝒆𝒆𝒇𝒔𝒕@ 𝒕@𝒑𝒖𝒊𝒇𝒔𝒙𝒋𝒕𝒇 𝒕𝒇

Learning Model

14

Learning Model

The summary of the Q learning process is as shown below: 1: Users pick a bid value 2: The utility function of the user based on the bid is calculated 3: Other records is also calculated in equation 4: The weight table is computed at time 𝑢 5: After 𝜐 trials, the best bidding value is exploited ​𝜌↑∗ =max(​𝑋↓𝑢 ) The summary of the Bayesian Learning process is as shown below:

1:𝑣 and the priori probability are calculated

2: The likelihood is generated from QL and converted into probability 3: Bayes rule is applied 4: Users pick the action with the highest utility and the winning percentage (​

𝜌↑∗ ).

15

Flow Chart

16

Q Learning and Bayesian Learning

17

45 46 47 48 49 50 51 52 53 54 55 20 40 60 80

(b) Price (Price Unit) Weight

45 46 47 48 49 50 51 52 53 54 55 20 40 60 80

(a) Price (Price Unit) Weight

100 Events 500 Events

Future Work

n Implication of learning on system performance such as

delay and energy consumption

n Compare different bidding strategies to explore the

capability of learning

n Problems with learning n Learning different parameters

18

Conclusions

n Bayesian learning and Q learning can be applied to an

auction based cognitive network

n Bayesian learning allows prior information to be

incorporated into Q learning to reduce the exploration time

n Bayesian learning converges faster than QL

19

Question and Suggestions

n

AK Oloyede aao500@york.ac.uk Communications Research Group University of York York

n

David Grace dg@ohm.york.ac.uk Communications Research Group University of York York

20