interfaces to formal argumentation Federico Cerutti xx iv mmxvi - - PowerPoint PPT Presentation

interfaces to formal argumentation

Federico Cerutti

xx • iv • mmxvi

How can we create an human understandable interface to defeasible reasoning in order to guarantee that human users will agree with the result of the automated reasoning procedures?

¬c p

S D s1 : ⇒ sAAA s2 : ⇒ sBBB r1 : sAAA ∧ ∼ exAAA ⇒ c r2 : sBBB ∧ ∼ exBBB ⇒ ¬ c r3 : ∼ exp ⇒ r1 ≺ r2 Args = {a1 = ⟨s1, r1⟩, a2 = ⟨s2, r2⟩, a3 = ⟨r3⟩}; a2 Args-defeats a1 a2 justified

the experiment

∙ Presenting each participant with a text, written in natural language, followed by a questionnaire ∙ Between subjects design across eight texts: each participant is shown a single (randomly selected) text ∙ Four domains:

• 1. weather forecast
• 2. political debate
• 3. used car sale
• 4. romantic relationship

∙ Two KBs: base case, and extended case ∙ The base case always consider two arguments a1 and a2 with two contradicting conclusions; and a preference in favour of a2 ∙ The extended case reinstates a1 (somehow)

¬c p y

¬c p ¬p

¬c p x

domain 1: weather forecast

The weather forecasting service of the broadcasting company AAA says that it will rain tomorrow (a1). Meanwhile, the forecast service of the broadcasting company BBB says that it will be cloudy tomorrow but that it will not rain (a2). It is also well known that the forecasting service of BBB is more accurate than the one of AAA (a3). However, yesterday the trustworthy newspaper CCC published an article which said that BBB has cut the resources for its weather forecasting service in the past months, thus making it less reliable than in the past (a4).

methodology

∙ Online questionnaire using Amazon’s Mechanical Turk ∙ Participants are asked to determine which of the following positions they think is accurate:

∙ PA: I think that AAA’s position is correct ∙ PB: I think that BBB’s position is correct ∙ PU: I cannot determine if either AAA’s or BBB’s position is correct

∙ Rate a statements in terms of relevance (for the conclusion) and agreement on a 7 points scale from Disagree to Agree for each statement

hypotheses

H1: In the base cases (Scenarios 1.B, 2.B, 3.B and 4.B), the majority of participants will agree with BBB’s statement (position PB) H2: In the extended cases (Scenarios 1.E, 2.E, 3.E and 4.E), the majority of participants will agree that they cannot conclude anything from the text (position PU) H3: The majority of participants who view a base case scenario will agree with the preference argument, and find it relevant

post hoc: motivations

Base Cases Extended Cases PA PB PU PA PB PU 1, weather 5.0 50.0 45.0 15.8 21.1 63.2 2, politics 5.3 63.2 31.6 21.1 10.5 68.4 3, buying car 0.0 68.2 31.8 23.8 23.8 52.4 4, romance 12.5 68.8 18.8 48.0 36.0 16.0 Distribution of the final conclusion PA/PB/PU

post hoc: distributions of base cases

15 30 45 60 U1 U2 U3 % Distributions of motivations for PU (scenarios 1.B and 3.B) 1.B 3.B

Agreement with the PU position in scenarios 1.B and 3.B: U1: lack of information, U2: domain specific reasons; U3: other

a critique to this approach

Are the naïve pieces of text we generated reasonable?

Speech Speech Text Text Meaning

Speech Recognition Natural Language Understanding Natural Language Generation Speech Synthesis

Document Planning Microplanning Linguistic realisation

Communicative goal(s) Document plans Sentence plans Surface text

content: express arguments and reasoning

a1 = ⟨⇒ sAAA sAAA∧ ∼ exAAA ⇒ c⟩ The weather forecasting service of the broadcasting company AAA says that it will rain tomorrow, therefore tomorrow should rain a2 = ⟨⇒ sBBB sBBB∧ ∼ exBBB ⇒ ¬c⟩ Meanwhile, the forecast service of the broadcasting company BBB says that it will be cloudy tomorrow but that it will not rain, therefore tomorrow should be cloudy but no rainy a3 = ⟨∼ exp ⇒ r1 ≺ r2⟩ It is also well known that the forecasting service of BBB is more accurate than the one of AAA Args{a1, a2, a3} a2 Args-defeats a1 a1 justified Since AAA’s and BBB’s conclusions are incompatible, and since BBB is more accurate than AAA, it is reasonable to tentatively conclude that tomorrow should not rain

structure: express arguments and reasoning

The weather forecasting service of the broadcasting company AAA says that it will rain tomorrow, therefore tomorrow should rain Meanwhile, the forecast service of the broadcasting company BBB says that it will be cloudy tomorrow but that it will not rain It is also well known that the forecasting service of BBB is more accurate than the one of AAA Since AAA’s and BBB’s conclusions are incompatible, and since BBB is more accurate than AAA, it is reasonable to tentatively conclude that tomorrow should not rain Tomorrow will be cloudy but not rainy according to BBB; opposite forecast is provided by AAA, known to be untrustworthy.

conclusions

Investigation into the relationship between formal systems of defeasible argumentation and natural language interfaces Results suggest a correspondence between the formal theory and its representation in natural language Humans evaluate preference depending on the context—Collateral knowledge Applying NLG studies to argumentation?

Cardiff Argumentation Forum 2016 6th-7th July 2016

http://www.cs.cf.ac.uk/caf2016/

The Second Summer School on Argumentation: Computational and Linguistic Perspectives

Potsdam, 8th-14th September 2016 http://ssa2016.west.uni-koblenz.de/?content=programme

COMMA 2016 Demonstration track deadline 22nd April 2016

http://www.ling.uni-potsdam.de/comma2016/

The First International Workshop on Systems and Algorithms for Formal Argumentation (SAFA) Deadline 1st July 2016

SAFA2016 will be hosted as a COMMA 2016 workshop, and will take place in Postdam, Germany, September 13, 2016. http://safa2016.west.uni-koblenz.de/

background on p&s

Rule-based argumentation framework Allows to express arguments in favour of preferences among rules Includes negation as failure an strong negation Although it is pre-Dung1995, it is easy to draw a correspondence with an abstract argumentation frameworks (there are some points where we should be cautious, but it is not the case of this work)

crash course on p&s

Each rule (strict or defeasible) as a set of antecedents and a consequent Arguments as sequence (instead of recursive structure like in ASPIC) of rules The conclusions of an argument is the set containing each consequent of each rule

• f the argument

Attacks on some antecedent or on some conclusion Semantics: grounded (skeptical) and stable (credulous)

domain 2: political debate

In a TV debate, the politician AAA argues that if Region X becomes independent then X’s citizens will be poorer than now (a1). Subsequently, financial expert (a3) Dr. BBB presents a document; which scientifically shows that Region X will not be worse off financially if it becomes independent (a2). After that, the moderator of the debate reminds BBB of more recent research by several important economists that disputes the claims in that document (a4).

domain 3: buying a car

You are planning to buy a second-hand car, and you go to a dealership with BBB, a mechanic whom has been recommended you by a friend (a3). The salesperson AAA shows you a car and says that it needs very little work done to it (a1). BBB says it will require quite a lot of work, because in the past he had to fix several issues in a car of the same model (a2). While you are at the dealership, your friend calls you to tell you that he knows (beyond a shadow of a doubt) that BBB made unnecessary repairs to his car last month (a4).

domain 4: romance

After several dates, you would like to start a serious relationship with J. but you turn to ask two friends of yours, AAA and BBB, for advice. You have known BBB for longer than you have known AAA (a3). AAA tells you that J is lovely and you should go ahead (a1), while BBB suggests that you should be very cautious because J might have a hidden agenda (a2). After some weeks, CCC, who is also a close friend of BBB, tells you that BBB has been into you for years; BBB is too shy to tell you about their feelings about you, but are still possessive of you (a4).