Ontology support for Ontology support for Management System Audit - - PowerPoint PPT Presentation
Ontology support for Ontology support for Management System Audit Management System Audit Programs Programs g Assisted Prot g Assisted Management System Auditing Management System Auditing Prot A. Gehrmann Gehrmann, S. , S.
Ontology support for Ontology support for Management System Audit Management System Audit Programs Programs
Prot Proté
ég
gé
é Assisted
Assisted Management System Auditing Management System Auditing A.
Gehrmann, S. , S. Ishizu Ishizu Aoyama Aoyama Gakuin Gakuin University, Japan University, Japan
Auditing and audit programs Auditing and audit programs
Caution: The term audit is used in many domains: : The term audit is used in many domains: Management, Computer security, Finance etc., Management, Computer security, Finance etc.,
We refer to Management System Audits as defined as defined in in ISO 19011:2002 ISO 19011:2002: :
– – systematic systematic, independent and , independent and documented process documented process for for
and evaluating evaluating it objectively to it objectively to determine the extent determine the extent to which to which audit criteria audit criteria are fulfilled; are fulfilled; ISO 19011:2002 clause 3.1 audit ISO 19011:2002 clause 3.1 audit
A set of audits for a defined purpose constitutes an audit program audit program; e.g. evaluation of effectiveness of ; e.g. evaluation of effectiveness of management system management system
Problem and approach Problem and approach
3rd
rd Party Management System Auditing is
Party Management System Auditing is criticized for not delivering values; we see criticized for not delivering values; we see the difficulty to deal with organizational the difficulty to deal with organizational complexity as one main obstacle to value complexity as one main obstacle to value-
adding auditing
We understand the management of complexity of organizations as a main complexity of organizations as a main factor for improvement and propose the factor for improvement and propose the use of an audit ontology and prot use of an audit ontology and proté
ég
gé
é for
for enhancing the value of auditing enhancing the value of auditing
Origins of complexity in 3 rd party auditing Origins of complexity in 3 rd party auditing
1. 1.Third party auditors have to
Third party auditors have to deal with hundred of less deal with hundred of less familiar domain concepts familiar domain concepts in a very short time, but as in a very short time, but as human beings can cope only with 7 (+ / human beings can cope only with 7 (+ / -
2) concepts at a time a time
2. 2.Management standards are
Management standards are generic in nature and give generic in nature and give raise to many interpretational issues raise to many interpretational issues, therefore , therefore fundamental concepts such as Quality, Contract, Design, fundamental concepts such as Quality, Contract, Design, I ntegrity and Availability of I nformation assets lacking I ntegrity and Availability of I nformation assets lacking
are not shared consistently between the auditee and are not shared consistently between the auditee and the auditors; leads to conceptual inconsistencies / the auditors; leads to conceptual inconsistencies / clashes clashes
3. 3.Many requirements
Many requirements might be applicable : Quality and might be applicable : Quality and I nformation Security, I T risk management based, I nformation Security, I T risk management based, Quality Manuals, I nternal Procedures, Auditee Quality Manuals, I nternal Procedures, Auditee’ ’s client s client’ ’s s specification, Auditee specification, Auditee’ ’s client s client’ ’s quality procedures s quality procedures
4. 4.Demand on
Demand on documentation documentation is high is high
5. 5.Organizational
Organizational complexity is high complexity is high (horizontal, vertical) (horizontal, vertical)
6. 6.Auditing needs
Auditing needs team communication team communication
Conceptual clashes: Availability Conceptual clashes: Availability
SP800-
30 (Appendix A):
The The security goal security goal that generates the that generates the requirement for protection against requirement for protection against Intentional or accidental attempts to Intentional or accidental attempts to – – Perform unauthorized deletion of Perform unauthorized deletion of data or data or – – Otherwise cause a denial of service Otherwise cause a denial of service
– – Unauthorized use of system Unauthorized use of system resources resources
I SO/ I EC 17799:2000 :
: ensuring ensuring that authorized that authorized users users have access have access to to information and associated information and associated assets when required assets when required
Auditing as on Auditing as on-
going knowledge acquisition with Proté
ég
gé
é
Phase 1 Phase 2 Phase 4 Phase 3
Auditing as on Auditing as on-
going knowledge acquisition with proté
ég
gé
é
Phase 1
Auditing as on Auditing as on-
going knowledge acquisition with proté
ég
gé
é
Phase 2
Auditing as on Auditing as on-
going knowledge acquisition with proté
ég
gé
é
Phase 3
Auditing as on Auditing as on-
going knowledge acquisition with proté
ég
gé
é
Phase 4
A case: The auditee A case: The auditee
Total Business Information Systems Ltd.-
Procurement Finance Technical service Hardware Software IT Security Network Development Windows Novell Medical General Installation Testing IT
5 Levels, 50 Engineers, 10 technical assistants, 10 clerical staff Service: Total network solutions including information security solution
The task ahead The task ahead
12 Interviews at 5 levels covering variety of engineering fields engineering fields
Time available is limited to 3 working days
2 auditors
CEO is non-
technician, lawyer
Managers: Former Hacker, MBA
Students, Part-
timer, non-
technical clerics
300 pages internal procedures and Management standard standard
Confirm findings Make conclusions Move in Organization
Gather facts Verify Common Understanding
Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
TBIS structure
Confirm findings Make conclusions Move in Organization
Conduct interview, Gather facts Verify Common Understanding
Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
Selecting stored requirements
Confirm findings Make conclusions Move in Organization
Gather facts Verify Common Understanding
Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context of a process
Select Right Level in organization Select Right interviewee
Selecting required processes and activities
Confirm findings Make conclusions Move in Organization
Conduct interview, Gather facts Verify Common Understanding
Link information Understand Organizational Structure
Identify Applicable Requirement
Interpret Requirement In context
Select Right Level in organization Select Right interviewee
Recording an interview
Confirm findings Make conclusions Move in Organization
Gather facts Refer to controlled concepts
Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
Access to controlled concepts
R e q u i r e m e n t s e l e c t i
C
c e p t v e r i f i c a t i
e q u i r e m e n t v e r i f i c a t i
Confirm findings Make conclusions Move in Organization
Gather facts
Verify Common Understanding Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
Confirm findings Make conclusions Move in Organization
Gather facts Verify Common Understanding
Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
The audit console in Protege
Organizational Units
Confirm findings Make conclusions Move in Organization
Gather facts Verify Common Understanding
Review situations, Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
Visualization of interview topics
Confirm findings Make conclusions Move in Organization
Gather facts Verify Common Understanding
Review situations, Link information Understand Organizational Structure Identify Applicable Requirement Interpret Requirement In context
Select Right Level in organization Select Right interviewee
Summary Summary -
Key functions of an audit ontology
Conduct systematically the audit audit
Document audit process for
Evaluating evidence
Determine the extent to which the audit criteria are which the audit criteria are fulfilled fulfilled
Proté
ég
gé
é for systematic
for systematic conduct and planning; conduct and planning; Prot Proté
ég
gé
é as organizer
as organizer
Proté
ég
gé
é as
as documentation tool documentation tool
Proté
ég
gé
é as evaluation
as evaluation support tool support tool
Proté
ég
gé
é for keeping
for keeping track audit findings track audit findings
Solutions for coping with complexity with a Solutions for coping with complexity with a Prot Proté
ég
gé
é audit ontology
audit ontology
1. 1.
Prot Proté ég gé é helps to helps to organize
concepts and make it possible to possible to manage hundreds manage hundreds of them at a time
2. 2.
An audit ontology helps to An audit ontology helps to identify conceptual clashes identify conceptual clashes and helps to understand generic concepts in the and helps to understand generic concepts in the context context
3. 3.
Audit Audit requirements are retrievable requirements are retrievable and their and their relationship are linked to concepts and required relationship are linked to concepts and required activities activities
4. 4.
Audit Audit documentation can be prepared documentation can be prepared on the fly by
using transformation for XML documents using transformation for XML documents
5. 5.
Teams can Teams can exchange exchange ontologies
for improved communication communication
6. 6.
Organizational Organizational complexity can be managed complexity can be managed by using by using an an organizational model
in the audit ontology
High technical requirement High technical requirement Understandability of Understandability of knowledge representation knowledge representation Instantaneous Instantaneous Necessary for auditing Necessary for auditing in a team in a team Communication Communication within team within team Need customization of user Need customization of user interface/print/representation interface/print/representation Knowledge base Knowledge base stored in XML stored in XML Audit findings and Audit findings and conclusions extracted conclusions extracted Is required but not a Is required but not a purpose in itself purpose in itself Audit Audit documentation documentation none none Domain vocabulary Domain vocabulary can be extended can be extended Usage in IT projects Usage in IT projects Part of system Part of system documentation documentation Currently not the focus Currently not the focus
chance chance Re Re-
usability of knowledge knowledge Requires a reasonable degree Requires a reasonable degree
ég
gé
é
Speed problems. Speed problems. Auditors have pre Auditors have pre-
defined concepts available available Creation of instances of Creation of instances of
Linking artifacts Linking artifacts On On-
site audit Requires understanding of Requires understanding of
Fast understanding Fast understanding by visualization and by visualization and taxonomies taxonomies Modeling of Modeling of
and organizational and organizational artifact artifact Audit planning Audit planning
Obstacles Obstacles Benefits Benefits Description Description Use/Phase Use/Phase
Usability of an audit ontology in prot Usability of an audit ontology in proté
ég
gé
é
Future applications / expectations Future applications / expectations
Expectation about features of proté
ég
gé
é :
: – – Speed Speed improvements (drawing, visualization) improvements (drawing, visualization) – – Possibility for Possibility for customizing customizing interface for knowledge interface for knowledge acquisation acquisation – – Build Build-
in documentation customizing documentation customizing
Implementation in OWL for reasoning and consistency OWL for reasoning and consistency
Remote login and sharing ontology over distributed clients clients
Import of industry ontologies SUO
Mobile devices: tablet computer
Proté
ég
gé
é as server component for customized clients
as server component for customized clients tool (files) for simplifying interface tool (files) for simplifying interface