[PPT] - Pay-per-Question: Towards Targeted Q&A with Payments Steve Jan , PowerPoint Presentation

SLIDE 1

Pay-per-Question: Towards Targeted Q&A with Payments

Steve Jan, Chun Wang, Qing Zhang, Gang Wang

SLIDE 2

Online Question & Answer Services

Web search engines

– But they can not give users the customized answers

Online Q&A Service

– Quora: 1M questions / month – Stack Overflow: 200K questions / month

2

Are they good enough?

SLIDE 3

Scenario 1

I got a traffic ticket the other day

3

Q: What are the tips to fight this ticket on court?

Ask friends Post it online Ask lawyers They may not understand the law Should I trust them? Too expensive Too slow

SLIDE 4

Scenario 2

I feel mildly sick the other day

4

Q: What were causing my headache and nausea?

Ask friends Post it online Ask doctors They may not understand medic Should I trust them? Too expensive Too slow

SLIDE 5

Another Option

I can directly ask some experts:

– Convenient: use my smart-phone – Trustworthy: certified domain experts – Targeted: ask experts who I can trust – Cheaper: cheaper than making a real appointment

5

Targeted Q&A apps are for the rescue!

SLIDE 6

Targeted Q&A Apps

Social network: connect users to certified experts
Targeted: ask an question to a specific user
Pay a small amount of money to ask questions
Very popular

6

May June 2016

2,000,000 USD revenue 10,000,000 registered users 500,000 paid questions

Launched Fenda Campfire Whale Yam

July

SLIDE 7

How Fenda Works

7

MD. Yang

Price $10/Q Pay 10 USD

1,000 Listeners

$7 cents $7 cents

What were causing my headache and nausea?

Give answer via audio

Asker 10% Income 10% Income 7 cents * 1000 listeners = 70 USD Received*0.1 + question fee = 7 + 10 = 17 USD Paid: Received: Final Profit: Received – Paid = 70 - 17 = 53 USD 14 cents USD to Listen

Fenda has a unique monetary incentive model!

Verified

SLIDE 8

This Study

Research questions

– How does monetary incentive affect Q&A? – Are there any manipulative behavior from users? – How does the pricing strategy affect users’ engagement?

Data driven analysis

– Collect over 200K paid questions from two websites

Fenda (China), Whale (US)

8

SLIDE 9

Outline

Introduction
User behavior in targeted Q&A apps

– Role of experts – Impact of monetary incentive

Manipulative behavior
Pricing strategy

9

SLIDE 10

Datasets

10

Dataset #Questions Time Coverage #Users #Experts Fenda 212,000 05/16 – 07/16 30% 88,540 4,370 Whale 9,200 09/16 – 03/17

1,419

118

Collect Fenda and Whale

– Using open API with slow speed – Using data set of Whale as a comparison – Coverage is around 30%1

Experts are verified manually by websites

– People can also ask questions to the normal users

1Li Xuanmin. 2016. Putting a price on knowledge. http://www.globaltimes.cn/content/997510.shtml.

SLIDE 11

Role of Experts

Experts: 5% of total of users, contribute 95% of

revenue and answer 82% of questions

11

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

# Users Revenue ($) # Answered Questions

Normal Users Experts

Experts are extremely important How does targeted Q&A service retain the experts?

SLIDE 12

Motivations on Q&A Service

Answerers are motivated to answer their questions by

– Intrinsic reward (e.g. helping people) – Social reward (e.g. respect from others) – Extrinsic reward (e.g. money)

Targeted Q&A service primary use extrinsic reward
Existing research suggested extrinsic reward1 may leads to

– Less response delay – High answer quality

12

1Haiyi Zhu, Sauvik Das, Yiqun Cao, Shuang Yu, Aniket Kittur, and Robert Kraut. A Market in Your Social Network: The Effects of

Extrinsic Rewards on Friendsourcing and Relationships. In Proc. of CHI (2016).

Are they true on Fenda and Whale?

SLIDE 13

Short Response Time?

Fenda & Whale

– Short response time – But not shorter than Yahoo answers

Yahoo answers: large number
f potential answerers
Fenda & Whale: target one

specific answerer

13

10 20 30 40 50 60 70

Yahoo Answers Fenda Whale Google Answers Stack Overflow

Average Response Time Hours

1 2 3

Targeted Q&A service are faster than most of crowdsourcing service

SLIDE 14

High Answer Quality?

14

High answer quality: Other people are also

interested; Willing to pay for the answers

56% of answers: Have at least one listener
Among these listened answers, 71% can make

profits for askers (listening income > question fee)

Good question: good chance for making profits

Majority of targeted Q&A service questions are high quality

SLIDE 15

Outline

Introduction
User behavior in targeted Q&A apps
Manipulative behavior

– Bounty Hunters – Collaborative Users

Pricing strategy

15

SLIDE 16

Manipulative Behavior

Bounty hunters: users ask lots of questions for $
Several types of experts to ask questions to:

16

Type 1 Experts

Few listeners High price High chance not earning

Type 2 Experts

Many listeners High price No guarantee Earning

Type 3 Experts

Many listeners Low price High chance earning

Bounty hunter

SLIDE 17

Manipulative Behavior

Bounty hunters: users ask lots of questions for $
Several types of experts to ask questions to:

17

Ratio of Questions to Experts # of Questions that the user asked

This outlier

Asked 1300 questions Earned around $200

Average

Asked 2.5 questions Can not earn ($-1.95)

Act as spam to experts

SLIDE 18

Manipulative Behavior 2

Collaborative users: Answerer and asker work

together exclusively

18

User-1: Answerer Other askers User-2: Asker

Expensive questions

Increase perception

f popularity

Cheap questions Ask questions to draw attentions

SLIDE 19

Manipulative Behavior 2

Collaborative users: Answerer and asker work

19

# of Answered questions # of Questions by same person

User-1 answered 435 questions. User-2 asked User-1 307 questions. Together earn $950

Fake perception of popularity

SLIDE 20

Outline

Introduction
User behavior in targeted Q&A apps
Manipulative behavior
Pricing strategy

– How do users set their price of questions? – How does the pricing strategy affect their income and engagement?

20

SLIDE 21

Dynamic Pricing

Answerers can adjust their price dynamically
Examples:

21

$3

User 1

$2 $1

How many common pricing strategies are there?

Too many askers Too few askers

$10

User 2 Without change

SLIDE 22

Pricing Strategy

Cluster pricing history into groups
Construct 9 features based on the pricing history
For example: Top 3 features based on Chi-square

22

id Feature Name Description 1 Price Change Frequency # of price change / # answers 2 Price Up Frequency # price up / # answers 3 Price Down Frequency # price down / # answers

$3

User 1

$2 $1

User 1: [2/3, 1/3, 1/3, …]

Applied hierarchical clustering algorithm on these features

… … …

SLIDE 23

Three Different Strategies

Got 3 groups (strategies) with highest modularity

23

Group 1 Group 2 Group 3 frequently price up and down mostly price up rarely price up and down active users celebrities inactive users Improve users’ incomes and engagement Hurt users’ incomes and engagement Hurt users’ incomes and engagement

SLIDE 24

Conclusion

Targeted Q&A service

– Short response time – High answer and question quality – Some manipulative behavior

Future Q&A work

– Crowdsourcing v.s. Targeted – Add more dataset

24

SLIDE 25

Thank You

25

SLIDE 26

Reference

[1] Dan Wu and Daqing He. 2014. Comparing IPL2 and Yahoo! Answers: A

Case Study of Digital Reference and Community Based Question Answering. In Proc. of Iconf.

[2] Benjamin Edelman. 2011. Earnings And Ratings At Google Answers.

Economic Inquiry 50, 2 (2011), 309–320.

[3] Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and

Björn Hartmann. 2011. Design lessons from the fastest Q&A site in the west. In

Proc. of CHI.

26