PageRank for Argument Relevance Henning Wachsmuth & Benno Stein - - PowerPoint PPT Presentation

▶
pagerank for argument relevance
SMART_READER_LITE
LIVE PREVIEW

PageRank for Argument Relevance Henning Wachsmuth & Benno Stein - - PowerPoint PPT Presentation

Apr 17, 2023 343 likes •648 views

PageRank for Argument Relevance Henning Wachsmuth & Benno Stein Bauhaus-Universitt Weimar www.webis.de Goals 1. Information Retrieval: Future (better?) search engines. 2. Argumentation: Improvement due to larger corpora (the

slide-1
SLIDE 1

PageRank for Argument Relevance

Henning Wachsmuth & Benno Stein

  • Bauhaus-Universität Weimar
  • www.webis.de
slide-2
SLIDE 2

Goals

  • 1. Information Retrieval: Future (better?) search engines.
  • 2. Argumentation: Improvement due to larger corpora (the web).
  • 3. Timeliness: Provide argumentation on dynamic corpora (the web).
  • 4. Debating: Improve flexibility and fallback behavior.

2

Apr.’16 • B. Stein

slide-3
SLIDE 3

Goal: Future Search Engines

“[Current] Search engines will take you half way, at best. [to deliver material to address an argumentative information need effectively.]”

[Noam Slonim, 14.12.2015] ❑ Classical retrieval systems operationalize the probability ranking principle. ❑ Future retrieval systems will provide us with justifications / rationales.

➜ Information needs may be formulated in hypothesis form. ➜ Rank documents according to the strongest arguments—support or attack.

3

Apr.’16 • B. Stein

slide-4
SLIDE 4

4

Apr.’16 • B. Stein

slide-5
SLIDE 5

Goals (continued)

  • 1. Information Retrieval: Future (better?) search engines.
  • 2. Argumentation: Improvement due to larger corpora (the web).
  • 3. Timeliness: Provide argumentation on dynamic corpora (the web).
  • 4. Debating: Improve flexibility and fallback behavior.

5

Apr.’16 • B. Stein

slide-6
SLIDE 6

Goals (continued)

Inference, Validation

validated arguments, proof trees reader-centric arguments relevant facts and arguments candidate documents author-centric sources formalized arguments

Formalization, Contextualization Extraxtion, Mining Retrieval

User

Query Interaction Synthesis, Visualization

6

Apr.’16 • B. Stein

slide-7
SLIDE 7

What if we had perfect argument mining technology?

Premises Conclusion

Argument

7

Apr.’16 • B. Stein

slide-8
SLIDE 8

Argument Graphs over Document Sets

Hypo- thesis

...

Web pages

Premises Conclusion

Arguments

support support support support support attack attack attack

≈ ≈ ≈

8

Apr.’16 • B. Stein

slide-9
SLIDE 9

Argument Graphs over Document Sets

An operationalizable model with five building blocks, in a nutshell:

  • 1. Syntax. A canoncial argument structure.

ARGUMENT ::= ( CONCLUSION, {PREMISE}n

1 )

  • 2. Semantics. An interpretation function α for an argument set A.

α : A × A → {supports, attacks, unrelated}, where A is the set of all mined arguments in some document set. A query (= hypothesis of a user) is in the role of a conclusion.

  • 3. The induced argument graph G = (AD, Eα) for a document set D.

From the RMS theory: Eα is cleaned such that G becomes a DAG.

  • 4. Recursive relevance computation for each a ∈ A via PageRank (or friends).

See uses in bibliometrics, social networks, road networks, or neuroscience.

  • 5. Argument ground (a-priori) strength. ∀a ∈ A : S(a) ≡

max

d∈D, a∈d{RBM25(d)}.

Identify pay-off values with relevance scores under some retrieval model.

9

Apr.’16 • B. Stein

slide-10
SLIDE 10

Argument Graphs over Document Sets

An operationalizable model with five building blocks, in a nutshell:

  • 1. Syntax. A canoncial argument structure.

ARGUMENT ::= ( CONCLUSION, {PREMISE}n

1 )

  • 2. Semantics. An interpretation function α for an argument set A.

α : A × A → {supports, attacks, unrelated}, where A is the set of all mined arguments in some document set. A query (= hypothesis of a user) is in the role of a conclusion.

  • 3. The induced argument graph G = (AD, Eα) for a document set D.

From the RMS theory: Eα is cleaned such that G becomes a DAG.

  • 4. Recursive relevance computation for each a ∈ A via PageRank (or friends).

See uses in bibliometrics, social networks, road networks, or neuroscience.

  • 5. Argument ground (a-priori) strength. ∀a ∈ A : S(a) ≡

max

d∈D, a∈d{RBM25(d)}.

Identify pay-off values with relevance scores under some retrieval model.

10

Apr.’16 • B. Stein

slide-11
SLIDE 11

Argument Graphs over Document Sets

An operationalizable model with five building blocks, in a nutshell:

  • 1. Syntax. A canoncial argument structure.

ARGUMENT ::= ( CONCLUSION, {PREMISE}n

1 )

  • 2. Semantics. An interpretation function α for an argument set A.

α : A × A → {supports, attacks, unrelated}, where A is the set of all mined arguments in some document set. A query (= hypothesis of a user) is in the role of a conclusion.

  • 3. The induced argument graph G = (AD, Eα) for a document set D.

From the RMS theory: Eα is cleaned such that G becomes a DAG.

  • 4. Recursive relevance computation for each a ∈ A via PageRank (or friends).

See uses in bibliometrics, social networks, road networks, or neuroscience.

  • 5. Argument ground (a-priori) strength. ∀a ∈ A : S(a) ≡

max

d∈D, a∈d{RBM25(d)}.

Identify pay-off values with relevance scores under some retrieval model.

11

Apr.’16 • B. Stein

slide-12
SLIDE 12

Argument Graphs over Document Sets

An operationalizable model with five building blocks, in a nutshell:

  • 1. Syntax. A canoncial argument structure.

ARGUMENT ::= ( CONCLUSION, {PREMISE}n

1 )

  • 2. Semantics. An interpretation function α for an argument set A.

α : A × A → {supports, attacks, unrelated}, where A is the set of all mined arguments in some document set. A query (= hypothesis of a user) is in the role of a conclusion.

  • 3. The induced argument graph G = (AD, Eα) for a document set D.

From the RMS theory: Eα is cleaned such that G becomes a DAG.

  • 4. Recursive relevance computation for each a ∈ A via PageRank (or friends).

See uses in bibliometrics, social networks, road networks, or neuroscience.

  • 5. Argument ground (a-priori) strength. ∀a ∈ A : S(a) ≡

max

d∈D, a∈d{RBM25(d)}.

Identify pay-off values with relevance scores under some retrieval model.

12

Apr.’16 • B. Stein

slide-13
SLIDE 13

Argument Graphs over Document Sets

An operationalizable model with five building blocks, in a nutshell:

  • 1. Syntax. A canoncial argument structure.

ARGUMENT ::= ( CONCLUSION, {PREMISE}n

1 )

  • 2. Semantics. An interpretation function α for an argument set A.

α : A × A → {supports, attacks, unrelated}, where A is the set of all mined arguments in some document set. A query (= hypothesis of a user) is in the role of a conclusion.

  • 3. The induced argument graph G = (AD, Eα) for a document set D.

From the RMS theory: Eα is cleaned such that G becomes a DAG.

  • 4. Recursive relevance computation for each a ∈ A via PageRank (or friends).

See uses in bibliometrics, social networks, road networks, or neuroscience.

  • 5. Argument ground (a-priori) strength. ∀a ∈ A : S(a) ≡

max

d∈D, a∈d{RBM25(d)}.

Identify pay-off values with relevance scores under some retrieval model.

13

Apr.’16 • B. Stein

slide-14
SLIDE 14

PageRank for Argument Relevance

14

Apr.’16 • B. Stein

slide-15
SLIDE 15

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

15

Apr.’16 • B. Stein

slide-16
SLIDE 16

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

16

Apr.’16 • B. Stein

slide-17
SLIDE 17

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

17

Apr.’16 • B. Stein

slide-18
SLIDE 18

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

18

Apr.’16 • B. Stein

slide-19
SLIDE 19

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

19

Apr.’16 • B. Stein

slide-20
SLIDE 20

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

20

Apr.’16 • B. Stein

slide-21
SLIDE 21

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

21

Apr.’16 • B. Stein

slide-22
SLIDE 22

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

22

Apr.’16 • B. Stein

slide-23
SLIDE 23

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

23

Apr.’16 • B. Stein

slide-24
SLIDE 24

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

24

Apr.’16 • B. Stein

slide-25
SLIDE 25

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

25

Apr.’16 • B. Stein

slide-26
SLIDE 26

PageRank for Argument Relevance

p(di) = (1 − α) · 1 |D| + α ·

  • j

p(dj) |Dj|

di dj 1. ground relevance + attributed relevance 2. dj links to di ❀ increase PageRank(di) 3. reward exclusive links 4. uniform ground relevances (sum to 1)

ˆ p(ci) = (1 − α) · p(di) · |D| |A| + α ·

  • j

ˆ p(cj) |Aj|

ci cj 1. ground strength + attributed relevance 2. cj relies on ci as a premise ❀ increase ArgumentRank(ci) 3. reward few premises 4. ground strength ∼ PageRank 5. normalize by the average number of arguments per web page

Principle of “Reversal of Evidence”

26

Apr.’16 • B. Stein

slide-27
SLIDE 27

Selected Research Questions

27

Apr.’16 • B. Stein

slide-28
SLIDE 28

Selected Research Questions

❑ Which of the existing argument models are suited theoretically?

Which models are suited practically? (robustness)

❑ How to identify relations ❀ equivalence between conclusions and premises?

(textual entailment, paraphrase recognition, enthymeme handling)

❑ How to balance support relations, attack relations, and recursion depth? ❑ How to balance intra-document and extra-document relations? ❑ Can recursive argument relevance be a valid measure for argument quality? ❑ User perspective: To what kind of queries should the paradigm be applied?

28

Apr.’16 • B. Stein

slide-29
SLIDE 29

Credits (incomplete)

Argument Quality

❑ Prominent argument detection debates [Boltuži´ c/Šnajder 2015] ❑ Argument strength of essays [Persing 2015] ❑ Finding arguments via graph analysis and entailment [Cabrio/Villata 2012] [Dung 1995]

Link Analysis, Reason Maintenance

❑ PageRank, Hubs and authorities [Page 1999] [Kleinberg 1999] ❑ Assumption-based truth maintenance systems [Reiter/deKleer 1987] ❑ Non-monotonic reasoning [McDermott/Doyle 1980]

Mining Arguments and Relations

❑ Claims and Evidence in Wikipedia [Levy 2014] [Rinott 2015] ❑ Counter-considerations in microtexts [Peldszus 2015] ❑ Argument units and relations in essays [Stab/Gurevych 2014]

29

Apr.’16 • B. Stein