interfaces to formal argumentation Federico Cerutti xx • iv • mmxvi
How can we create an human understandable interface to defeasible reasoning in order to guarantee that human users will agree with the result of the automated reasoning procedures?
p ¬c
S D s 1 : ⇒ s AAA r 1 : s AAA ∧ ∼ ex AAA ⇒ c s 2 : ⇒ s BBB r 2 : s BBB ∧ ∼ ex BBB ⇒ ¬ c r 3 : ∼ ex p ⇒ r 1 ≺ r 2 A rgs = { a 1 = ⟨ s 1 , r 1 ⟩ , a 2 = ⟨ s 2 , r 2 ⟩ , a 3 = ⟨ r 3 ⟩} ; a 2 A rgs-defeats a 1 a 2 justified
the experiment ∙ Presenting each participant with a text, written in natural language, followed by a questionnaire ∙ Between subjects design across eight texts: each participant is shown a single (randomly selected) text ∙ Four domains: 1. weather forecast 2. political debate 3. used car sale 4. romantic relationship ∙ Two KBs: base case, and extended case ∙ The base case always consider two arguments a 1 and a 2 with two contradicting conclusions; and a preference in favour of a 2 ∙ The extended case reinstates a 1 (somehow)
p ¬c y
p ¬p ¬c
x p ¬c
domain 1: weather forecast The weather forecasting service of the broadcasting company AAA says that it will rain tomorrow ( a 1 ). Meanwhile, the forecast service of the broadcasting company BBB says that it will be cloudy tomorrow but that it will not rain ( a 2 ). It is also well known that the forecasting service of BBB is more accurate than the one of AAA ( a 3 ). However, yesterday the trustworthy newspaper CCC published an article which said that BBB has cut the resources for its weather forecasting service in the past months, thus making it less reliable than in the past ( a 4 ).
methodology ∙ Online questionnaire using Amazon’s Mechanical Turk ∙ Participants are asked to determine which of the following positions they think is accurate: ∙ P A : I think that AAA’s position is correct ∙ P B : I think that BBB’s position is correct ∙ P U : I cannot determine if either AAA’s or BBB’s position is correct ∙ Rate a statements in terms of relevance (for the conclusion) and agreement on a 7 points scale from Disagree to Agree for each statement
hypotheses H1: In the base cases (Scenarios 1 . B, 2 . B, 3 . B and 4 . B), the majority of participants will agree with BBB’s statement (position P B ) H2: In the extended cases (Scenarios 1 . E, 2 . E, 3 . E and 4 . E), the majority of participants will agree that they cannot conclude anything from the text (position P U ) H3: The majority of participants who view a base case scenario will agree with the preference argument, and find it relevant
post hoc: motivations Base Cases Extended Cases P A P B P U P A P B P U 1, weather 5.0 50.0 45.0 15.8 21.1 63.2 2, politics 5.3 63.2 31.6 21.1 10.5 68.4 3, buying car 0.0 68.2 31.8 23.8 23.8 52.4 4, romance 12.5 68.8 18.8 48.0 36.0 16.0 Distribution of the final conclusion P A / P B / P U
post hoc: distributions of base cases Distributions of motivations for P U (scenarios 1 .B and 3 .B ) 60 45 % 30 15 0 U1 U2 U3 1 .B 3 .B Agreement with the P U position in scenarios 1 . B and 3 . B: U1: lack of information, U2: domain specific reasons; U3: other
a critique to this approach Are the naïve pieces of text we generated reasonable?
Meaning Natural Language Natural Language Understanding Generation Text Text Speech Speech Recognition Synthesis Speech Speech
Communicative goal(s) Document Planning Document plans Microplanning Sentence plans Linguistic realisation Surface text
content: express arguments and reasoning a 1 = ⟨⇒ s AAA The weather forecasting service of the broadcasting s AAA ∧ ∼ ex AAA ⇒ c ⟩ company AAA says that it will rain tomorrow, therefore tomorrow should rain a 2 = ⟨⇒ s BBB Meanwhile, the forecast service of the broadcasting s BBB ∧ ∼ ex BBB ⇒ ¬ c ⟩ company BBB says that it will be cloudy tomorrow but that it will not rain, therefore tomorrow should be cloudy but no rainy a 3 = ⟨∼ ex p ⇒ r 1 ≺ r 2 ⟩ It is also well known that the forecasting service of BBB is more accurate than the one of AAA A rgs { a 1 , a 2 , a 3 } Since AAA’s and BBB’s conclusions are incompatible, a 2 A rgs-defeats a 1 and since BBB is more accurate than AAA, it is a 1 justified reasonable to tentatively conclude that tomorrow should not rain
structure: express arguments and reasoning The weather forecasting service of the broadcasting company AAA says that it will rain tomorrow, therefore tomorrow should rain Meanwhile, the forecast service of the broadcasting company BBB says that it will be cloudy tomorrow Tomorrow will be cloudy but not but that it will not rain rainy according to BBB; opposite forecast is provided by AAA, known It is also well known that the forecasting service of to be untrustworthy. BBB is more accurate than the one of AAA Since AAA’s and BBB’s conclusions are incompatible, and since BBB is more accurate than AAA, it is reasonable to tentatively conclude that tomorrow should not rain
conclusions Investigation into the relationship between formal systems of defeasible argumentation and natural language interfaces Results suggest a correspondence between the formal theory and its representation in natural language Humans evaluate preference depending on the context—Collateral knowledge Applying NLG studies to argumentation?
Cardiff Argumentation Forum 2016 6th-7th July 2016 http://www.cs.cf.ac.uk/caf2016/
The Second Summer School on Argumentation: Computational and Linguistic Perspectives Potsdam, 8th-14th September 2016 http://ssa2016.west.uni-koblenz.de/?content=programme
COMMA 2016 Demonstration track deadline 22nd April 2016 http://www.ling.uni-potsdam.de/comma2016/
The First International Workshop on Systems and Algorithms for Formal Argumentation (SAFA) Deadline 1st July 2016 SAFA2016 will be hosted as a COMMA 2016 workshop, and will take place in Postdam, Germany, September 13, 2016. http://safa2016.west.uni-koblenz.de/
background on p&s Rule-based argumentation framework Allows to express arguments in favour of preferences among rules Includes negation as failure an strong negation Although it is pre-Dung1995, it is easy to draw a correspondence with an abstract argumentation frameworks (there are some points where we should be cautious, but it is not the case of this work)
crash course on p&s Each rule (strict or defeasible) as a set of antecedents and a consequent Arguments as sequence (instead of recursive structure like in ASPIC) of rules The conclusions of an argument is the set containing each consequent of each rule of the argument Attacks on some antecedent or on some conclusion Semantics: grounded (skeptical) and stable (credulous)
domain 2: political debate In a TV debate, the politician AAA argues that if Region X becomes independent then X’s citizens will be poorer than now ( a 1 ). Subsequently, financial expert ( a 3 ) Dr. BBB presents a document; which scientifically shows that Region X will not be worse off financially if it becomes independent ( a 2 ). After that, the moderator of the debate reminds BBB of more recent research by several important economists that disputes the claims in that document ( a 4 ).
domain 3: buying a car You are planning to buy a second-hand car, and you go to a dealership with BBB, a mechanic whom has been recommended you by a friend ( a 3 ). The salesperson AAA shows you a car and says that it needs very little work done to it ( a 1 ). BBB says it will require quite a lot of work, because in the past he had to fix several issues in a car of the same model ( a 2 ). While you are at the dealership, your friend calls you to tell you that he knows (beyond a shadow of a doubt) that BBB made unnecessary repairs to his car last month ( a 4 ).
domain 4: romance After several dates, you would like to start a serious relationship with J. but you turn to ask two friends of yours, AAA and BBB, for advice. You have known BBB for longer than you have known AAA ( a 3 ). AAA tells you that J is lovely and you should go ahead ( a 1 ), while BBB suggests that you should be very cautious because J might have a hidden agenda ( a 2 ). After some weeks, CCC, who is also a close friend of BBB, tells you that BBB has been into you for years; BBB is too shy to tell you about their feelings about you, but are still possessive of you ( a 4 ).
Recommend
More recommend
