Deliberative Democracy citizens active participation in decision - - PowerPoint PPT Presentation

▶
deliberative democracy
SMART_READER_LITE
LIVE PREVIEW

Deliberative Democracy citizens active participation in decision - - PowerPoint PPT Presentation

Nov 01, 2022 671 likes •1.12k views

Factful : Engaging Taxpayers in the Public Discussion of a Government Budget Juho Kim (MIT) Eun-Young Ko (KAIST) Jonghyuk Jung (KAIST) Chang Won Lee (KAIST) Nam Wook Kim (Harvard) Jihee Kim (KAIST) Deliberative Democracy citizens active

slide-1
SLIDE 1

Factful: Engaging Taxpayers in the

Public Discussion of a Government Budget

Juho Kim (MIT) Eun-Young Ko (KAIST) Jonghyuk Jung (KAIST) Chang Won Lee (KAIST) Nam Wook Kim (Harvard) Jihee Kim (KAIST)

slide-2
SLIDE 2

Deliberative Democracy

citizens’ active participation in decision making and related discussions

slide-3
SLIDE 3

Deliberative Democracy Online

Open Government Data UI for Discussion Support

Balancer [Munson et al., 2011] ConsiderIt [Kriplean et al., 2012]

data.gov.uk, data.gov, data.seoul.go.kr

Reflect [Kriplean et al., 2012]

slide-4
SLIDE 4

Government Budget

plan for government to best allocate resources But…

  • hard to comprehend
  • extremely complex
  • low interest & awareness
slide-5
SLIDE 5

Improve awareness & understanding of budgetary issues Build interactive systems for civic engagement Leverage open government data

slide-6
SLIDE 6

Understanding Taxpayers’ Challenges

  • Survey

– 182 respondents in Korea – Perception of the government budget + estimation quiz

  • Semi-structured interviews

– 5 taxpayers – 3 experts

slide-7
SLIDE 7
  • 1. Low awareness and interest in

budgetary issues

knowledgeable about budget info search for budget info

1 not really 7 very much 3.1 3.2

slide-8
SLIDE 8
  • 1. Low awareness and interest in

budgetary issues

interest in the budget budget info is useful

1 not really 7 very much 4.6 4.5

slide-9
SLIDE 9
  • 1. Low awareness and interest in

budgetary issues

  • Federal police budget estimation

[Mean: $8 $800B 00B, Median: $11B, Stdev: 2114] “I feel distant from all the big numbers that don’t really mean anything to me.”

$8.8B

slide-10
SLIDE 10
  • 2. Contextual information matters in
  • pinion formation.

“(Taxpayers) sometimes only see their own interests and fail to realize that compromises need to be made.”

slide-11
SLIDE 11
  • 2. Contextual information matters in
  • pinion formation.
slide-12
SLIDE 12
  • 2. Contextual information matters in
  • pinion formation.

Seeing other programs in the category sometimes affected respondents’ opinion.

slide-13
SLIDE 13
  • 3. News Outlets: Primary source for

learning about budgetary issues

  • 74% regularly read articles online

– U.S. [Purcell et al., 2012]: 50% via news sites, 10% social networks

  • News articles: more comprehensible, engaging
  • NONE had attempted to read govt. reports
slide-14
SLIDE 14
  • 3. News Outlets: Primary source for

learning about budgetary issues

  • Concerned about media biases
  • Others’ comments help recognize the potential

subjectivity, bias, or error in an article

  • U.S. survey [Purcell et al., 2012]
  • 37%: commenting important feature to have
  • 25%: commenting experience
slide-15
SLIDE 15
slide-16
SLIDE 16

Factful: Fact-Oriented Budgetary Discussions Online

enhanced news reader application

con

  • ntextual

bu budgetary facts rea eader er-in init itia iated fa fact-ch check cking

slide-17
SLIDE 17
slide-18
SLIDE 18

LAYER #1: CONTEXTUAL BUDGET FACTS

slide-19
SLIDE 19

Embedding Contextual Budget Facts

slide-20
SLIDE 20

Automatically Inserted Overview

5-year trend category info category breakdown

cat ategory detection: fit score based on word hit count

slide-21
SLIDE 21

Most Relevant Budget Programs

link to budget program webpage budget name, category, amount pro rogram suggestion: for each word in the article, compute TF-IDF score against each program

slide-22
SLIDE 22

Automatic Annotation

programs of similar size monetary value annotation with ru rule-ba based detector

slide-23
SLIDE 23

LAYER #2: READER-INITIATED FACT-CHECKING

slide-24
SLIDE 24

add a comment do fact-checking request fact-checking

Reader Activities

slide-25
SLIDE 25

Annotative Threaded Discussion

slide-26
SLIDE 26

do fact-checking request fact-checking

Fact-Checking Embedded in the Article

read w/ fact- checked result

slide-27
SLIDE 27

Open Government Data Article Text Analysis

Budget Data Processing Pipeline

  • pengov.seoul.go.kr
  • 76% of all internal documents

publicly accessible

  • 2014: $24B,

13 1st level categories, 4629 individual programs

  • Article text parsing
  • Category detection
  • Program suggestion
  • Monetary value detection
slide-28
SLIDE 28

Evaluation: with Factful…

  • H1. Readers will discuss with

mo more fact-ba based statements.

  • H2. Readers will discuss with

mo more evidence, and mo more kinds of evidence.

  • H3. Readers will become

mo more critical about the article.

slide-29
SLIDE 29

Evaluation Setup

Between-subjects, 38 participants

Commenting Commenting Fact-checking Commenting Fact-checking Contextual Info

Baseline Fact-checking

  • nly

Factful

slide-30
SLIDE 30

Tasks and Procedures

Three articles about Seoul’s budget & policies

Pre-Q Read Discuss Post-Q

slide-31
SLIDE 31

Discourse Analysis

  • 404 comments
  • Discussion coding:

each comment is coded with

  • ne of the 25 categories

[UnweightedCohen’s 𝜆: 0.613]

slide-32
SLIDE 32

Discussion Quality Assessment

  • 5 external raters
  • 10 questions about discussion quality (score between 1-10)
  • Overall quality
  • Criteria derived from deliberation lit. [Fishkin & Luskin, 2005]

informed, balanced, conscientious, substantive, comprehensive

slide-33
SLIDE 33

With Factful,

  • verall discussion quality was higher.

score

6.6 5.67 4.93 1 2 3 4 5 6 7 8 9 10

Baseline Fact-Checking Factful p < 0.05 p < 0.05

slide-34
SLIDE 34
  • H1. With Factful, discussions contained

more relevant, accurate information.

strongly agree

Discussants participated in the discussion with more relevant, accurate information.

strongly disagree

6.93 6.27 6.27 1 2 3 4 5 6 7 8 9 10

Baseline Fact-Checking Factful p < 0.05 p < 0.05

slide-35
SLIDE 35
  • H1. With Factful, participants added more

fact-oriented comments...?

2.46 1.77 1.67 1 2 3

Baseline Fact-Checking Factful

2.54 1.77 1.75 1 2 3

Baseline Fact-Checking Factful

# comments with objective supporting arguments # comments that asked for objective information

# comments / person

slide-36
SLIDE 36
  • H2. Factful discussions contained more

diverse perspectives and supporting evidence.

6.93 6.00 5.27 1 2 3 4 5 6 7 8 9 10

Baseline Fact-Checking Factful strongly agree strongly disagree

Discussants participated in the discussion with more diverse perspectives and supporting evidence.

p < 0.05 p < 0.05

slide-37
SLIDE 37
  • H3. With Factful, participants became

more critical about the article.

4.03 4.77 5.08 1 2 3 4 5 6 7

Baseline Fact-Checking Factful

I trust the content of the article.

strongly agree strongly disagree p < 0.01 p < 0.05

slide-38
SLIDE 38
  • H3. With Factful, participants held a

more critical view on the article. # comments criticizing the article

  • 0.8 / person in Factful vs 0 in Baseline

# comments criticizing other participants

0.46 0.31 1.25 1 2 Baseline Fact-­‑Checking Factful

# comments / person

slide-39
SLIDE 39

Role of Contextual Information

Automated annotations w/ similar sized programs Category overview “Without such information, it would be hard to determine if the given go government spending g is wo worth or not.” “It made reading through the article easier, because the budget terms and numbers in the article fe felt le less obscure.”

slide-40
SLIDE 40

Future Work

  • Live deployment
  • Crowdsourced fact-checking methods
  • Generalization

– different countries – other datasets

slide-41
SLIDE 41

Improve awareness & understanding of budgetary issues Build interactive systems for civic engagement Leverage open government data

Da Data-dr driven, so social, cr crowdsourced me mechanisms ms

slide-42
SLIDE 42

BudgetMap: Issue-Driven Navigation for a Government Budget. Nam Wook Kim, Chang Won Lee, Jonghyuk Jung, Eun-Young Ko, Juho Kim, Jihee Kim. CHI 2015 Extended Abstracts.

BudgetMap: Issue-Driven Budget Navigation

slide-43
SLIDE 43

Factful: Engaging Taxpayers in the

Public Discussion of a Government Budget

juhokim@mit.edu budgetwiser.org