Q 2 1 1 TDT4250 - Modeling of Information Systems, Autumn 2006 - PDF document

TDT4250 - Modeling of Information Systems, Autumn 2006 Quality of models John Krogstie Professor, IDI, NTNU Senior Researcher, SINTEF ICT 1 TDT4250 - Modeling of Information Systems, Autumn 2006 Overview of presentation � What is quality ? � Detailed presentation of semiotic model quality framework (SEQUAL) Quality of models � Quality of modelling languages (briefly, more later) � � Based on chapter 3.1 in book manuscript Q 2 1 1

TDT4250 - Modeling of Information Systems, Autumn 2006 What is ’quality’ ? � Is ’The Lord of the Rings’ a good (set of) movie ? Q 3 TDT4250 - Modeling of Information Systems, Autumn 2006 Different views on quality exist � Product vs. Process quality (e.g. CMM) � According to requirements (ISO 9000) � The user is satisfied (Denning) � Properties of the product (-ilities) � Properties of a requirements specification or model (Davis/Pohl) � Quality related to different semiotic levels (Lindland, Stamper) � ”The quality that can be defined is not true quality” - Robert Pirsig Q 4 2 2

TDT4250 - Modeling of Information Systems, Autumn 2006 Properties of the product (design oriented) � How well does it function? � Efficiency, Integrity, Reliability, Survivability, Usability � How good is the design? � Correctness, Maintainability, Verifiability � How easy is it to evolve the product? � Expandability, Flexibility, Interoperability, Portability, Reusability Q 5 TDT4250 - Modeling of Information Systems, Autumn 2006 Properties of a requirements specification (Davis) � Complete � Correct � Understandable � Modifiable � Consistent � Unambigious � Reusable � Verifiable � etc. Q 6 3 3

TDT4250 - Modeling of Information Systems, Autumn 2006 Dimensions related to requirements specification process (Pohl) Q 7 TDT4250 - Modeling of Information Systems, Autumn 2006 Core framework on quality of models (1994) Q 8 4 4

TDT4250 - Modeling of Information Systems, Autumn 2006 The Method Evaluation Model for evaluating the early version of the framework PERFORMANCE PERCEPTIONS Actual P erceived INTENTIONS E fficiency E as e of Us e BEHAVIOUR Intention to Actual Us e Us ag e Actual P erceived E ffective- Us efulnes s nes s Q 9 TDT4250 - Modeling of Information Systems, Autumn 2006 Research Design Participants 194 third year IS students Participants trained Experimental how to us e the quality Materials Treatment framework 20 different cas es Case #1 Case #1 in textual form Case #1 random assignment 1. Modelling Task Experimental 2. E valuation Tas k Tasks 3. Post-task S urvey proces s models quality reviews survey res pons es Web-based Evaluation S ys tem Dependent Variables: D1 = S yntactic Quality Performance D2 = S emantic Quality Bas ed Variables D3 = Pragmatic Quality D4 = Overall Quality D5 = Perceived E ase of Use Perception Q D6 = Perceived Usefulnes s Bas ed Variables D7 = Intention to Use 10 5 5

TDT4250 - Modeling of Information Systems, Autumn 2006 Main results Reliability Analysis: Not consistent use of the framework by all � Validity Analysis: � Completeness: The three level found to be complete (for this kind of � use, i.e. with an objectively given domain) Necessity: All three levels necessary � Independence: The three levels independent � Interaction � S yntactic Quality + + + S emantic Pragmatic Quality Quality Q 11 TDT4250 - Modeling of Information Systems, Autumn 2006 Main results continue � Many errors in the models made � Many errors not reported � Most errors reported where errors of the type indicated � Participants found the framework � Easy to use � Useful � Intention to use � Need for specialisation of the framework Q 12 6 6

TDT4250 - Modeling of Information Systems, Autumn 2006 Some weaknesses with earlier work on quality of models � Only for some types of models (Requirements specification/design/completed product) � Cover only models in some languages � Uncoordinated list of wanted properties � Mix quality of process, tool, language, and model � Mix quality goals and means to achieve these goals � Is better suited for limited, constructed cases than real modelling problems in practice. Q 13 TDT4250 - Modeling of Information Systems, Autumn 2006 SEQUAL – A framework for understanding and assessing quality of models � For models as a knowledge representation in general � Can be extended and specialised towards specific types of model and modelling languages � Differentiate between goals of modelling and means to achieve these goals � Set-oriented definition to enable a formal discussion of the different quality levels � Differentiate between quality of different levels based on semiotic theory � Takes into account that models are socially constructed Q 14 7 7

TDT4250 - Modeling of Information Systems, Autumn 2006 Main elements of a modelling activity Goal of Modelling Persons Example of goal: Create a requirements Means for Area of interest specification for a travel representation Modeling task agency on the net . Tools Q 15 TDT4250 - Modeling of Information Systems, Autumn 2006 Sets in the quality framework � A: Actors that develops or has to relate to (parts of) the model. Can be persons or tools. � L: What can be expressed in the modelling language � M: What is expressed in the model � D: What can be expressed about the domain (area of interest) � K: The explicit knowlegde of the participating persons � I: What the persons in the audience interpret the model to say � T: What relevant tools interpret the model to say Q � G: The goals of the modelling All of these sets evolves as part of modeling 16 8 8

TDT4250 - Modeling of Information Systems, Autumn 2006 Different types of goals possible - example � A new work processes should be documented through the models. � The models developed should help sharing best practice between different units. � The models developed should be helpful in the process of refining the processes. � The models developed should teach the software developers about the domain. � The models developed should define the scope of the software application. � The models developed should help analyze and harmonize the current work processes. � The models developed should be used as a procedural tool in everyday work. � The models developed should support the use of the Q software application. 17 TDT4250 - Modeling of Information Systems, Autumn 2006 Who are the human actors (A)? � A subgroup of those that are influenced by the modelling (stakeholders). � Different types of stakeholders. E.g. in the context of system development: � System analysts, designers and programmers � Users � Managers Q 18 9 9

TDT4250 - Modeling of Information Systems, Autumn 2006 What is expressed in a model? � This is based on the model and the semantics of the language used for modelling. � Can differentiate between what is explicitly and implicitly expressed. � Language: Propositional logic � Explicit model: A, A -> B � Implicit model: B � Overall model: A, B, A -> B Q 19 TDT4250 - Modeling of Information Systems, Autumn 2006 Different domains relevant for IS-development � EIS: Existing Information System � ECIS: Existing Computer-based Information System � FIS: Future Information System � FCIS: Future Computer-based Information System � Requirements specification: Looking at the externally observable behaviour of the IS � Design: Looking at the internal fabric of the technical system � CIS: The actual computerised information system, including the data within the system. Q 20 10 10

TDT4250 - Modeling of Information Systems, Autumn 2006 Different domains for IS-development according to MDA-thinking in OMG � CIM (vs. FIS): Computational Independant Model � PIM (vs. FCIS - RE/Design): Platform Independant Model � PSM (vs. FCIS – Detailed design): Platform Specific Model Q 21 TDT4250 - Modeling of Information Systems, Autumn 2006 SEQUAL Perceived semantic Social actor explicit Social knowledge Pragmatic (learning) Goals of Social actor modelling interpretation Physical Organizational Pragmatic Pragmatic (human (action) understanding) Modeling Model Language domain externalization extension Semantic Syntactic Pragmatic Empirical (tool understanding) Technical Q actor interpretation 22 11 11

TDT4250 - Modeling of Information Systems, Autumn 2006 Overall structure of framework � Quality type (physical, emprical…) � One or more quality goals per quality type � Means to achieve this goal Beneficial existing quality � Model properties � Language properties � Modeling activities � Tool-support � Q 23 TDT4250 - Modeling of Information Systems, Autumn 2006 Model example to illustrate the different quality levels TITLE PERSON N WRITES M PAPER LANGUAGE Language: ER Domain: Conference organizing Q Goal: Design of database solution to support conference organizing 24 12 12