Previous class Cognitive adequacy (Strube) strong (SCA) vs weak - - PowerPoint PPT Presentation

▶
previous class
SMART_READER_LITE
LIVE PREVIEW

Previous class Cognitive adequacy (Strube) strong (SCA) vs weak - - PowerPoint PPT Presentation

Aug 04, 2023 560 likes •883 views

Previous class Cognitive adequacy (Strube) strong (SCA) vs weak (WCA) WCA dimensions Tasks (reading, writing, using) Facets Learning (acuainted, competent, expert) Productivity (rate of task completion over

slide-1
SLIDE 1

Previous class

  • Cognitive adequacy (Strube)

– strong (SCA) vs weak (WCA)

  • WCA

– dimensions

  • Tasks (reading, writing, using)
  • Facets

– Learning (acuainted, competent, expert) – Productivity (rate of task completion over time, error-free?) – Error detection and correction – Satisfaction

– WCA and syntax (keywords vs symbols, more vs less) – WCA and expressive power – WCA and computation

1

Tuesday, 25 March 2014

slide-2
SLIDE 2

WCA & Computation

  • The more services the better

– if delivered in reasonable time – different services for one language might have different performance characteristics

  • Users often accept greater computation time

– if they get perceived sufficient value – within sane bounds

  • Toleration of computation time varies

– with circumstances – greater toleration at development time

  • getting right trumps getting it fast
  • Toleration != contentment

– Even if they tolerate it, users prefer speed

2

Tuesday, 25 March 2014

slide-3
SLIDE 3

SCA & WCA

  • What is the relation, if any?
  • Possible relations

– helps vs. ensures – could be negative!

  • SCA could preclude WCA
  • Why might these hold?

3

→

SCA WCA SCA Yes (at least, in the same respect) ??? WCA ??? Yes (at least, in the same respect)

Tuesday, 25 March 2014

slide-4
SLIDE 4

Strube’s argument

  • First, we separate out certain WCA issues

– Many ways to make systems unusable, don’t do them!

  • Acknowledge issues of SCA

– SCA systems are much harder to develop

  • We may not even know how to do it!

– A SCA rep might not work!

  • Three big benefits of SCA:

– “enhanced validity” – “added flexibility and stability” – “better security and ease of use” – (This mix up some bits.)

4

Tuesday, 25 March 2014

slide-5
SLIDE 5

Strube’s Argument: Validity

  • A broadness consideration

– Knowledge is tangled up with other knowledge – Scoping is difficult – Modelling “how we think” induces naturally appropriate scoping

  • Which can be used to understand what we drop out
  • Big Issue:

– Experts often don’t do this!

  • Focus on the domain, rather than reflection
  • They use tools (books, programs, etc.)
  • We typically understand the domain better than people!

5

“validity refers to the content and representational format of domain-specific knowledge, to the strategies of reasoning employed, and to the conditions of use for both”

Tuesday, 25 March 2014

slide-6
SLIDE 6

Strube’s Arg: Flexibility & Stability

  • An emulation consideration

– We use lots of reps and reasoning strategies – We’re pretty good

  • Problems:

– Flexibility might be a coping, rather than enabling strategy

  • E.g., it’s the best way given human constraints!
  • But consider chess playing...programs don’t have our limitations

– so succeed with different strategies

– Multiple representations can inhibit engineering

  • Don’t Repeat Yourself (DRY)
  • How do we know that they are the same?

6

“The flexibility gained through selective application of different reasoning strategies, which in turn utilize different parts of knowledge [is] perhaps the most important single cause of our success in thinking and problem solving. It follows that the cognitive approach to knowledge engineering must strive for variety in reasoning, and must likewise try to embody domain- specific knowledge in multiple representation formats in the knowledge base.”

Tuesday, 25 March 2014

slide-7
SLIDE 7

Strube’s Arg: Ease of Use

  • A focused consideration

– We need explanations (of certain kinds) – Hard to see how to get them without SCA

  • Problems:

– Do we need these explanations?

  • A lot of modern day stuff is focused on debugging KRs
  • Software debuggers might be more relevant

– Do we need explanations?

  • Question answering systems may not!

–Or may be fine with just supplying facts – What about cost to the rest of the system?

7

A system that models the reasoning of real experts provides a sound base for giving explanations to its user, explanations that are both correct and understandable. By contrast, it is difficult to see how a system whose reasoning is utterly un-human might arrive at explanations that fulfil both requirements...A cognitive approach is necessary, at least to some degree, to make the explanation component of a system useful.

Tuesday, 25 March 2014

slide-8
SLIDE 8

Strube’s Argument

  • It’s not clear that for many applications

– We need SCA – We need SCA to achieve validity, flexibility, etc. – We can achieve validity, flexibility, etc. via SCA

  • Where might SCA still be useful?

– Modelling people!

8

Tuesday, 25 March 2014

slide-9
SLIDE 9

Definition Oriented Development (encore)

Sebastian Brandt brandt@cs.manchester.ac.uk

(Slides by Bijan Parsia, bparsia@cs.man.ac.uk)

Tuesday, 25 March 2014

slide-10
SLIDE 10

F

  • r

m a l i z e

The Simple Development Cycle

has knowledge

Conceptualize

∃P.C ⊑ ∀P.D A⊓B ⊑ ∀P.D ∃P(A⊔B)⊑¬D

has Knowledge

Communicate I n f e r e n c e

10

Tuesday, 25 March 2014

slide-11
SLIDE 11

Recall: Definition Oriented Dev

  • “Reduce” (certain kinds of) effort

– Local focus on what terms mean

  • Verification

– There are consequences to what we say – We can spot wrong links

  • Work on our part to detect problems
  • But inferred links are a subset of all links

– The reasoner can tell us about broken definitions

  • We still need to understand them!
  • Improve interaction

– The KR becomes “reactive”

  • Comes at a computational cost!

11

Tuesday, 25 March 2014

slide-12
SLIDE 12

Reduce which effort?

  • Target: Hierarchical controlled vocabularies

– Aka taxonomies (why do we want these?)

  • Without (logically encoded) definitions

– We must formulate the definitions – We must put terms “in their proper place” – We must assert every non-trivial “link” – We must check that these are the right links

  • Thus we must determine what all the right links are
  • We must also verify that the links we include are right
  • How much work?

– 100 terms ≈ 10,000 (1002) possible subsumptions!

  • But we don’t need to be naive about it

– How many terms do we need?

  • Current terminologies tend toward the hundreds of thousands
  • How about in principle?

Mediated by humans!

12

Tuesday, 25 March 2014

slide-13
SLIDE 13

Types of Bicycle injury

  • 1972 ICD-9 (E826) 8
  • 1990 READ-2 (T30..) 81
  • late 1990s READ-3 87
  • 1999 ICD-10 ……

http://www.cs.man.ac.uk/~rector/presentations/Reasoning-web-rector-GALEN-2006.ppt

Tuesday, 25 March 2014

slide-14
SLIDE 14

1999 ICD10: 587 codes

  • V31.22 Occupant of three-wheeled motor vehicle injured in

collision with pedal cycle, person on outside of vehicle, nontraffjc accident, while working for income

  • W65.40 Drowning and submersion while in bath-tub, street and

highway, while engaged in sports activity

  • X35.44 Victim of volcanic eruption, street and highway, while

resting, sleeping, eating or engaging in other vital activities

http://www.cs.man.ac.uk/~rector/presentations/Reasoning-web-rector-GALEN-2006.ppt

Tuesday, 25 March 2014

slide-15
SLIDE 15

It’s getting worse!

  • “With the move to ICD-10, the [1] code for suturing an

artery will become 195 codes, designating every single artery, among other variables, according to OptumInsight, a unit of UnitedHealth Group Inc. A single code for a badly healed fracture could now translate to 2,595 different codes, the firm calculates. Each signals information including what bone was broken, as well as which side of the body it was on.”*

  • A compositional problem

– We are using opaque names – Thus every combination needs a new name – Thus must trade off

  • Precision for number

15

*http://online.wsj.com/article/SB10001424053111904103404576560742746021106.htm

Tuesday, 25 March 2014

slide-16
SLIDE 16

Two Problems

  • We have a lot of terms

– lots and lots and lots – how do we manage them?

  • We have pressures to add more terms

– fundamental

  • medical knowledge expands
  • medical record keeping needs expand

– artificial (or “technical”)

  • primarily, combinatorial pressure

16 Edwin Smith Papyrus

http://www.flickr.com/photos/78572993@N00/2226696853/

Tuesday, 25 March 2014

slide-17
SLIDE 17

500 codes in pieces

  • 10 things to hit…

– Pedestrian / cycle / motorbike / car / HGV / train / unpowered vehicle / a tree / other

  • 5 roles for the injured…

– Driving / passenger / cyclist / getting in / other

  • 5 activities when injured…

– resting / at work / sporting / at leisure / other

  • 2 contexts…

– In traffic / not in traffic

17

V12.24 Pedal cyclist injured in collision with two- or three-wheeled motor vehicle, unspecified pedal cyclist, nontraffic accident, while resting, sleeping, eating or engaging in other vital activities

Slide from: http://www.cs.man.ac.uk/~rector/presentations/snomed-rector-history-and-future-of-terminology.ppt

10 * 5 * 5 * 2 = 500

Tuesday, 25 March 2014

slide-18
SLIDE 18

Some Empirical Data

  • We took a number of medical vocabularies

– Parsed their terms – Divided the number of terms by the size of the basis

18

Tuesday, 25 March 2014

slide-19
SLIDE 19

Terms as (class) expressions

  • The big move

– from names to expressions – from few names we induce many expressions

  • a terse representation of the term space
  • humans work this way!

– position driven by definition

  • Brings complexity

– Terms have different grammatical roles

  • classes, properties

– We have to use logicy stuff

  • reasoning!

19

Tuesday, 25 March 2014

slide-20
SLIDE 20

Coordination

  • “...one key problem limiting re-use and application-

independence is that the formalisms used force application-specific choices...one of the key application-specific decisions which formalisms force

  • n users is the organisation of a fixed taxonomy based
  • n indivisible representations of concepts. Inevitably,

indivisible representations of complex concepts conflate several ideas, and since any fixed

  • rganisation of such complex ideas is fundamentally

arbitrary, choices can only be made on the basis of the current application...avoiding such decisions is an important technique for achieving reuse.

20

http://www.opengalen.org/download/aime-95.pdf

Tuesday, 25 March 2014

slide-21
SLIDE 21

Forms of coordination

  • Pre-coordination

– No code without prior agreement

  • Only explicitly published codes
  • Maybe there’s an “other”
  • Integrating others is post-facto

– Code drift

  • Introducing new codes complicates old codings

– Though devtime definitions can help!

  • Post-coordination

– Core set of of terms

  • We can fix these, even

– A code is any valid expression

  • We don’t need an expression

– Many nonsensical codes! – Requires reasoning at “runtime”

21

Tuesday, 25 March 2014

slide-22
SLIDE 22

Coordination Example

  • Imagine a simple taxonomy

– Nuts and nut allergy

  • Notice the lack of specificity

– No PecanAllergy, HazelnutAllergy, etc. – Why not?

  • Consider a form using this

– Very simple – We have a patient with an almond allergy

  • Do we select

– Allergy?

22

Tuesday, 25 March 2014

slide-23
SLIDE 23

Coordination Example

  • Imagine a simple taxonomy

– Nuts and nut allergy

  • Notice the lack of specificity

– No PecanAllergy, HazelnutAllergy, etc. – Why not?

  • Consider a form using this

– Very simple – We have a patient with an almond allergy

  • Do we select

– Allergy? – Nut Allergy?

23

Tuesday, 25 March 2014

slide-24
SLIDE 24

Coordination Example

  • Imagine a simple taxonomy

– Nuts and nut allergy

  • Notice the lack of specificity

– No PecanAllergy, HazelnutAllergy, etc. – Why not?

  • Consider a form using this

– Very simple – We have a patient with an almond allergy

  • Do we select

– Allergy? – Nut Allergy? – Other?

24

Tuesday, 25 March 2014

slide-25
SLIDE 25

Coordination Example

  • All choices have problems

– The fact of (divergent) choice – The uncontrolled bits

  • Can definitions help?

25

Tuesday, 25 March 2014

slide-26
SLIDE 26

Coordination by definition

  • Note that now we have some expressive power

26

Tuesday, 25 March 2014

slide-27
SLIDE 27

The pre-coordination cycle

has knowledge Conceptualize

∃P.C

has 27

http://www.flickr.com/photos/seattlemunicipalarchives/4058808950/

No changes at runtime Code set changes over time

Tuesday, 25 March 2014

slide-28
SLIDE 28

The Post-coordination cycle

F

  • r

m a l i z e

has knowledge

Conceptualize

∃P.C ⊑ ∀P.D A⊓B ⊑ ∀P.D ∃P(A⊔B)⊑¬D

has

C

  • m

m u n i c a t e I n f e r e n c e

28

http://www.flickr.com/photos/seattlemunicipalarchives/4058808950/

W r i t e e x p r e s s i

  • n

s

Tuesday, 25 March 2014

slide-29
SLIDE 29

Runtime?

  • UI might be tricky...

29

Tuesday, 25 March 2014

slide-30
SLIDE 30

Some Pre-coordination Necessary

  • SNOMED is the canonical post-coordinated vocab

– Still has a medium coordination factor – The minimum basis might not be the sensible basis

30

Tuesday, 25 March 2014

slide-31
SLIDE 31

Runtime vs. Development Time

  • We can see benefits at each stage
  • Each stage has different requirements

– E.g., Runtime has higher performance demands – E.g., Development time has higher correctness demands

  • Beyond codes!

– We can look at definition structure

  • To structure other parts of forms!

31

Tuesday, 25 March 2014