INVENTIVE Evaluating PSI Ontologies by Mapping to the Common Sense - - PowerPoint PPT Presentation

inventive
SMART_READER_LITE
LIVE PREVIEW

INVENTIVE Evaluating PSI Ontologies by Mapping to the Common Sense - - PowerPoint PPT Presentation

INVENTIVE Evaluating PSI Ontologies by Mapping to the Common Sense Natalya KEBERLE Zaporozhye National Uni, Ukraine Vadim ERMOLAYEV Zaporozhye National Uni, Ukraine Wolf-Ekkehard MATZKE Cadence Design Systems GmbH, Germany May 24, 2007,


slide-1
SLIDE 1

INVENTIVE

Evaluating PSI Ontologies by Mapping to the Common Sense

Natalya KEBERLE Zaporozhye National Uni, Ukraine Vadim ERMOLAYEV Zaporozhye National Uni, Ukraine Wolf-Ekkehard MATZKE Cadence Design Systems GmbH, Germany

May 24, 2007, Kharkiv, Ukraine

slide-2
SLIDE 2

May 24, 2007 2 Evaluation of PSI Ontologies by Mapping to the Common Sense

Outline

  • Evaluation of ontologies

– Why, What and How-To?

  • Performance Simulation Initiative (PSI)

– PSI Ontologies Suite, reasons for evaluation of PSI ontologies

  • Common Sense as a “golden standard”

– What is Common Sense – Sources

  • Evaluation results

– PSI Meta – Mapping to upper level ontologies – Good mappings = close to common sense?

  • Conclusions and Outlook
slide-3
SLIDE 3

May 24, 2007 3 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation of Ontologies: Why?

  • Ontology - shared and agreed specification of

conceptualization [Gruber 1993]

  • Ontology – is a semiotic object [Gangemi et al, 2005]

It reflects the subjective views of its creators (knowledge engineers, domain experts etc)

  • There may be different ontologies for the same body of

knowledge

slide-4
SLIDE 4

May 24, 2007 4 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation of Ontologies: Why?

Making swing: strengthened wooden board, reliable ropes

slide-5
SLIDE 5

May 24, 2007 5 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation of Ontologies: Why?

Making swing: strengthened wooden board, reliable ropes

“strengthen” “reliable” 1 “reliable” 2 Ambiguity in terms … …wrong (and costly) solutions

slide-6
SLIDE 6

May 24, 2007 6 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation Dimensions

Conceptual modeling Usage of representation languages Suitability of the model w.r.t. a domain and use cases Suitability of the implemented model w.r.t. a domain and use cases Ontology engineering process

slide-7
SLIDE 7

May 24, 2007 7 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation of Ontologies: How to?

  • Logical evaluation

– Logical correctness of an ontology as a formal theory

  • Human expert evaluation

– Set of predefined criteria, domain standards, requirements

  • Data-driven evaluation

– Tagging of domain documents

  • Application-driven evaluation

– Plug the ontology into an application and evaluate results

  • “Golden Standard”

– Standard ontology required => well established domains

  • Set of metrics

– Structural, functional metrics, usability

slide-8
SLIDE 8

May 24, 2007 8 Evaluation of PSI Ontologies by Mapping to the Common Sense

Performance Simulation Initiative (PSI)

  • Internal Initiative of Cadence Design Systems, GmbH
  • Research and Development in Engineering Design

Performance Assessment and Management

  • A horizontal framework for R&D cooperation

– E.g., PRODUKTIV+ project (German Federal Ministry

  • f Education and Research)
  • Current PSI partners:

– VCAD, Cadence Design Systems, GmbH – Dept of Cybernetics and Gerstner Lab, Czech Technical Uni – CERTICON Corp. – Intelligent Systems Research Group, Zaporozhye National Uni

  • ZNU does knowledge modeling and management
slide-9
SLIDE 9

May 24, 2007 9 Evaluation of PSI Ontologies by Mapping to the Common Sense

PSI Ontologies Suite v.1.6

slide-10
SLIDE 10

May 24, 2007 10 Evaluation of PSI Ontologies by Mapping to the Common Sense

PSI Ontologies Suite v.1.6

The high-level structure of the PSI and PRODUKTIV+ Ontologies Suite. White packages represent the Core. Colored packages are the Extensions

slide-11
SLIDE 11

May 24, 2007 11 Evaluation of PSI Ontologies by Mapping to the Common Sense

PSI Ontologies: Evaluation Dimensions

Conceptual modeling Usage of representation languages Suitability of the model w.r.t. a domain and use cases Suitability of the implemented model w.r.t. a domain and use cases Ontology engineering process

slide-12
SLIDE 12

May 24, 2007 12 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation of PSI Ontologies: How to?

  • Logical evaluation

– Logical correctness of an ontology as a formal theory => Use it

  • Human expert evaluation

– Set of domain standards, requirements => no established standards => No

  • Data-driven evaluation

– Tagging of domain documents => documents are unstructured => No

  • Application-driven evaluation

– Plug the ontology into an application and evaluate results => => no applications yet => No

  • “Golden Standard”

– Standard ontology required => Use Common Sense

  • Set of metrics

– Structural, functional metrics, usability => May Be in Future

slide-13
SLIDE 13

May 24, 2007 13 Evaluation of PSI Ontologies by Mapping to the Common Sense

Common Sense

  • Scientific theories do not emerge in vacuum
  • There is some background knowledge = common sense
  • Scientists are aware of and may (not) use common

sense in their theory

  • Formalized (long way, but…) Common Sense:

– OpenCYC, SUMO, DOLCE, BFO, OCRHE,… but not so much

  • Drawbacks are:

– High level of abstraction in formalized common sense

slide-14
SLIDE 14

May 24, 2007 14 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation w.r.t. Common Sense

  • Use Upper Level Ontologies from different sources:

– Suggested Upper Merged Ontology (SUMO) – WordNet – Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) – Basic Formal Ontology (BFO) – Object-Centered High-Level Reference Ontology (OCHRE)

  • Map independently

– Only to DOLCE – Via WordNet to SUMO

  • Find upward cotopies first
  • Compare results
slide-15
SLIDE 15

May 24, 2007 15 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation of PSI Ontologies Suite w.r.t. Common Sense

  • Construction of PSI-Meta ontology – upward cotopies of

domain concepts

  • E.g.:

DesignArtifact’s upward cotopies are

slide-16
SLIDE 16

May 24, 2007 16 Evaluation of PSI Ontologies by Mapping to the Common Sense

Scenario 1: Mapping to SUMO via WordNet

  • WordNet – provides

almost all PSI concepts with their natural language semantics

  • SUMO – concepts and

instances in one semantic network – has benefited from harmonization with WordNet

slide-17
SLIDE 17

May 24, 2007 17 Evaluation of PSI Ontologies by Mapping to the Common Sense

Scenario 2: Mapping to DOLCE

  • DOLCE – provides formal

hierarchy of upper-level concepts

  • Does not use WordNet,

instead WordNet is “sweetened” with DOLCE

slide-18
SLIDE 18

May 24, 2007 18 Evaluation of PSI Ontologies by Mapping to the Common Sense

Evaluation Results

  • Quality of mappings to WordNet+SUMO and to DOLCE

is not the same:

– WordNet+SUMO is good in Processes, various Parameters – DOLCE is good in Abilities/Beliefs of Actor, in Tasks, in Descriptions

  • WordNet helps to resolve ambiguous concept names

– Manual work

  • Good mappings are for PSI Task, Actor, DesignArtifact
  • ntologies => real common sense orientation
  • Average quality mapping of Negotiation Process =>

underdevelopment of upper-level ontologies

slide-19
SLIDE 19

May 24, 2007 19 Evaluation of PSI Ontologies by Mapping to the Common Sense

Concluding Remarks

  • Evaluation of ontologies

– Is must-have for many real-world intelligent applications

  • Evaluation of ontologies for any domain

– Is hard, often manual, process

  • Evaluation of ontologies for a new domain

– May be checked against the Common Sense as a “golden standard”

  • Results of evaluation

– May influence both “golden standard” and domain ontology

slide-20
SLIDE 20

May 24, 2007 20 Evaluation of PSI Ontologies by Mapping to the Common Sense

Future Work

  • Evaluation of PSI Ontologies Suite against all evaluation

dimensions

  • Refining of PSI Ontologies Suite
  • Presentation of PSI Ontologies Suite for shared use
slide-21
SLIDE 21

May 24, 2007 21 Evaluation of PSI Ontologies by Mapping to the Common Sense

Questions please

Resources:

PSI: http://ermolayev.com/ISRG/ISRG-projects-PSI.htm E-paper: http://ermolayev.com/eva_personal/PS/PSI-ISTA-07_CR.pdf This presentation: http://ermolayev.com/eva_personal/PS/ISTA-2007-PSI-to-CommonSense.pdf