Information security audits & certification Security in - - PowerPoint PPT Presentation

1

Information security audits & certification

Security in Organizations 2011 Eric Verheul

2

Literature

Main literature for this lecture: 1. NOREA beroepsregels

http://www.norea.nl/Norea/Thema's/Gedrags- +en+beroepsregels/Richtlijn+Assurance-opdrachten

2. TTP.NL schema (http://www.ecp.nl/sites/default/files/TTP- NL_Scheme_version_8.1_final__June_2010_.pdf ) 3. Common Criteria part 1 (http://standards.iso.org/ittf/PubliclyAvailableStandards/c050 341_ISO_IEC_15408-1_2009.zip )

Variants on ISO 2700*

3

Assignment #5

• Assignment #3 is on Blackboard
• It uses VMWARE image. This is available:
• Through Klaus/DVD
• On-line

sftp://lilo.science.ru.nl/vol/xpsoftware/sio2009/image_1111 09/*.*

• Note: starting the VMWARE image takes time; first start

the image then read the assignment

Variants on ISO 2700*

4

Outline

• Audit introduction
• IT security audits in general
• management system certification audits
• IT security product certification audits (‘common criteria’)
• Recap & Practicum

5

Types of audits

• The audits we are discussing include:
• IT security audits in general,
• management systemcertification audits,
• IT security product certification audits
• As there is – as far as we know – no common terminology

used for these three types of audits simultaneously, we will introduce our own terminology. This is actually based on a combination of terms taken from these audit types.

Audit introduction

6

Terminology

• An audit is the process in which an competent, impartial

judgment (‘opinion’) is formed on one or more aspects of an

• bject (‘criteria’).
• The result of an audit is typically a document in which the

auditor expresses his opinion, the supporting findings and the limitations that apply.

• The opinion provides assurance to the auditee itself or to a

third party.

• The assurance can be either positive or negative:
• Positive assurance - An affirmative statement or opinion given by the

auditor, generally based on a high level of work performed.

• Negative assurance - A statement indicating that nothing came to the

auditor's attention indicating that the subject matter in question did not meet a specified criteria.

Audit introduction

7

Terminology

Audit Object Audit Criteria Audit Scheme Auditor Opinion (report)

Independent

• verseer

(e.g. association

• rganization)

Scheme maintainer (e.g. association

• rganization)

Audit introduction

Criteria maintainer

8

Terminology

• The audit process should be reproducible and should not

depend on the (qualified) auditor.

• An opinion can also take the form of a ‘certificate’.
• Audits are historically associated with accounting: a financial

audit of the financial accounts (‘jaarrekening audit’) performed by (registered) accountants. In this situation the criteria are based on the laws on accounting (‘Wet op de jaarrekening’). In the accounting context the term ‘audit’ is a very sensitive notion.

Audit introduction

9

Terminology

• The audit is performed for a client, that also sponsors the

audit.

• The aspects that form the basis of the audit are formulated as

a set of criteria (audit criteria), determined prior to the actual audit and agreed upon with the client. In Dutch these criteria are sometimes ‘de gehanteerde (audit) norm’.

• The set of criteria could be an open standard, a tailored

version of it, or even some assertions made by the client

• management. In the latter case, the opinion can be a

statement of the auditor that the assertions are correct.

• The object type can vary, examples are: a person, a product,

a process, a system or an organization.

Audit introduction

10

Audit schemes

• Closely linked with the audit criteria is the audit scheme used.

These are rules describing how the audits shall be conducted and what requirements should be met by the auditor

• rganization itself
• An audit scheme provides a ‘manual’ for conducting audits

and typically answers questions like:

• What steps shall an audit have?
• When is a criterion met?
• What qualifications should an auditor have?
• When can the auditor ‘built’ on prior work done by other auditors?
• When can an opinion be provided and what can be part of it?

Audit introduction

11

Audit schemes

Important general topics in audit schemes are:

• impartiality requirements of auditors and the organizations

they work for,

• confidentiality,
• providing auditees the opportunity to respond to findings

(‘hoor en wederhoor’)

• ethics, e.g., ‘do not audit your own work’,
• quality, e.g. filing of evidence

Audit introduction

12

Audit schemes

The audit scheme can be:

• an open standard itself, e.g.,
• ISO 19011 ‘Guidelines for quality and/or environmental management

systems auditing’

• ISO/IEC 17021 ‘Requirements for bodies providing audit and

certification of management systems,

• and its particularization ISO 27006 ‘Requirements for bodies providing

audit and certification of information security management systems’

• a dedicated document, e.g., the TTP-NL scheme ‘Scheme

For Certification of Certification Authorities against ETSI TS 101 456’

• or it could be part of the rules of conduct of the professional

associations (‘beroepsverenigingen’) of auditors, e.g. of NOREA (http://www.norea.nl/Norea/Thema's/Gedrags- +en+beroepsregels/Richtlijn+Assurance-opdrachten) or ISACA (www.isaca.org).

Audit introduction

13

Terminology

Object Opinion Criteria Source: https://cert.webtrust.org/SealFile?seal=304&file=pdf Scheme

Audit introduction

14

Terminology

Audit Object Audit Criteria Audit Scheme Auditor Opinion (report)

Independent

• verseer

(e.g. association

• rganization)

Scheme maintainer (e.g. association

• rganization)

Audit introduction

Criteria maintainer

15

Outline

• Audit introduction
• IT security audits in general
• management systemcertification audits,
• IT security product certification audits (‘common criteria’)
• Recap & Practicum

16

IT (security) audits

• An IT security audit is a particular type of an IT audit.
• An IT audit is also known as an EDP audit and focuses on the

following aspects of IT systems (cf. COBIT):

• Effectiveness
• Efficiency
• Compliance
• Reliability
• Confidentiality
• Integrity
• Availability
• An IT audit can therefore include much more than information

security.

IT security audits in general

17

IT audit aspects

• Effectiveness

Deals with information being relevant and pertinent to the business process as well as being delivered in a timely, correct, consistent and usable manner.

• Efficiency

Concerns the provision of information through the optimal (most productive and economical) usage of resources

• Reliability

Relates to systems providing management with appropriate information for it to use in operating the entity, in providing financial reporting to users

• f the financial information, and in providing information to report to

regulatory bodies with regard to compliance with laws and regulations

• Compliance

Deals with complying with those laws, regulations and contractual arrangements to which the business process is subject; i.e., externally imposed business criteria

IT security audits in general

18

IT effectiveness

IT security audits in general

19

IT audit aspects

• Confidentiality

Concerns protection of sensitive information from unauthorized disclosure.

• Integrity

Relates to the accuracy and completeness of information as well as to its validity in accordance with the business' set of values and expectations.

• Availability

Relates to information being available when required by the business process, and hence also concerns the safeguarding of resources.

IT security audits in general

20

IT security audits

• IT security audits (aka IT security reviews) concentrate on

information security aspects, i.e.:

• Confidentiality
• Integrity
• Availability
• Sometimes IT security audits are called IT security reviews to

prevent confusion with financial audits.

• IT security audits can be:
• technically oriented; then the objects are IT systems, e.g., a whole IT

infrastructure, a network, a Windows environment, a specific application

• process oriented; then the objects are IT processes, e.g., a security

management process, a change management process.

• The audit criteria are typically formulated in information

security objectives or security controls, e.g. based on ISO 27002.

IT security audits in general

21

Example of technical IT Security criteria

IT security audits in general

22

Example of non-technical IT Security criteria

IT security audits in general

23

Audit evidence

• Practically speaking, the auditor should:
• determine the scope of the audit (e.g., Windows based office

automation network),

• agree the audit criteria with the audit sponsor and put them in a table

and compare the criteria with the object setting.

• But what should an auditor accept as compliance evidence?

IT security audits in general

24

Audit evidence

• What if the IT administrator says in an interview: ‘Sure, we

have this password policy and account lockout setting’?

• What if there is an official document stating compliance with

these setting?

• When should you believe this setting is actually

implemented?

IT security audits in general

25

Audit evidence

• What if the IT administrator shows you the Windows settings.

Can you be sure that this will not be changed tomorrow?

IT security audits in general

26

The three audit assurance levels

• In IT audits one therefore distinguishes three types of audit

assurance ‘levels’:

• Design

The auditor has reviewed the relevant design based on documentation and interviews but not on actual inspections. In effect, the auditor can not provide assurance is designed is actually implemented.

• Existence

The auditor has additionally performed inspections of system settings, paper archives and other things providing him with assurance that the design is at least implemented during the audit.

• Operational Effectiveness

The auditor has additionally looked for evidence that the implemented controls were effective for a certain amount of time.

• These audit levels build upon each other, i.e. you can only

have Design, Design + Existence or Design + Existence + OE

• The audit level is an integral part of the opinion report!

IT security audits in general

27

Terminology

Object Opinion Criteria Source: https://cert.webtrust.org/SealFile?seal=304&file=pdf Scheme

IT security audits in general

28

The three audit assurance levels

• In IT audits one therefore distinguishes three types of audit

assurance ‘levels’:

• Design

The auditor has reviewed the relevant design based on documentation and interviews but not on actual inspections. In effect, the auditor can not provide assurance is designed is actually implemented.

• Existence

The auditor has additionally performed inspections of system settings, paper archives and other things providing him with assurance that the design is at least implemented during the audit.

• Operational Effectiveness

The auditor has additionally looked for evidence that the implemented controls were effective for a certain amount of time.

IT security audits in general

29

The opinion

• It is vital that the opinion minimally states:
• For who the audit was conducted (client) by whom

(auditor)

• The objective of the audit
• The object and its boundaries
• The period in which the audit was performed
• The followed procedures, e.g., documentation review,

interviews, inspections etc.

• The audit criteria used and the related audit scheme
• The assurance level of the audit (‘design’, ‘existence’ or

‘operational effectiveness’

• The opinion itself and any reservations or limitations

regarding the opinion.

• Optionally one can supplement the opinion with

recommendations however some schemes preclude on grounds of impartiality.

IT security audits in general

30

Is Penetration Testing ‘auditing’?

• A penetration test is a method of evaluating the security of a

computer system or network by simulating an attack from a malicious source. (source: wikipedia).

• One could say a penetration test concentrates on ‘existence’

and ‘operational effectiveness’ of information security and not

• n documented ‘design’.
• There are implicit ‘criteria’ and ‘frameworks’ such as The

Open Source Security Testing Methodology Manual (OSSTMM) and Guideline on Network Security Testing (NIST SP 800-42) and the Testing Guide of the Open Web Application Security Project (OWASP).

• There also exist professional associations of penetration

testers.

• Dependence on the competence of penetration tester is

higher than in a typical audit, making the reproducibility difficult.

IT security audits in general

31

Outline

• Audit introduction
• IT security audits in general
• management systemcertification audits
• IT security product certification audits (‘common criteria’)
• Recap & Practicum

32

Certification of management systems

• A management system is framework of policies, procedures

guidelines and associated resources to achieve the

• bjectives of the organization. (source: ISO 27000)
• An Information Security Management System (ISMS) is that

part of the overall management system, based on a business risk approach, to establish, implement, operate, monitor, review, maintain and improve information security. (source: ISO 27001)

• Certification of a management system, such as a quality or

environmental management system of an organization, is one means of providing assurance that the organization has implemented a system for the management of the relevant aspects of its activities, in line with its policy. (source: ISO 17021).

management system certification audits

33

Limitation

• We limit our selves to these two IT security (management)

systems:

• ISO 27001 ‘Information security management systems’

which we have focused on in the first three lectures

• TTP.NL which relates to the European guideline on

electronic signatures.

• Certification ‘auditors’ are typically called Certification Bodies

and need to have management systems of their own to reflect the proper process of certification. The basis for the management system for Certification Bodies is described in ISO 17021.

management system certification audits

34

IT security management certification framework

Management System Standard (ISO, ETSI) Certification Scheme Certification Body Certificate (report)

Scheme maintainer (ISO, ECP-EPN)

Accreditation Body (RVA)

management system certification audits

Criteria Maintainer (ISO, ETSI)

35

Accreditation

• The management systems of the certification Bodies are

inspected by the Dutch Accreditation Council (Raad voor Accrediatie, see www.rva.nl) in a similar way as the Certification Bodies perform certification.

• This process is called accreditation. There is actually a

standard for Accreditation Bodies themselves (ISO 17011 ‘General requirements for accreditation bodies accrediting conformity assessment bodies’).

• (Nearly) every country has its own ‘Accreditation Council’ and

mutual agreements exist.

• The national councils also perform ‘peer reviews’.

management system certification audits

36

The schemes

ISO 27001 scheme Criteria: ISO 27001 Scheme:

• maintained by ISO
• ISO 27006 which is based on ISO 17021 (who) and ISO 19011 (how)
• Accreditation Body: RVA

TTP.NL scheme Criteria: ETSI TS 101 456 Scheme:

• Maintained by ECP-EPN
• TTP.NL scheme which is based on ISO 17021 (who) and ISO 19011 (how)
• Accreditation Body: RVA

management system certification audits

37

Audits within a certification

There exist several types of audits within in the context of a certification:

• Trial audits (optional)
• Initial audits (consisting themselves of documentation and implementation

audits)

• Follow-up audits
• Surveillance audits
• Recertification audits very similar to initial audits.
• Special audits, when major changes take place in the client’s organization

management system certification audits

38

Certification overview

Documentation

Audit

Implementation

Audit

Certification

Decision

Trial-audit

(optional)

Surveillance Audit (yearly) Surveillance Audit (yearly)

Recertification

• A certificate is valid for three years.

Follow-up audits

management system certification audits

39

Audit plan

• For each type of audit, the audits makes an audit plan prior to

conducting it.

• The audit plan describes:
• the timing of the audit,
• the topics of the audit (preferably in reference with the criteria)
• the staff (internal/external) that needs to be interviewed (when/where)
• any visits or inspections (in implementation and surveillance audits)
• The audit plan is interactively compiled with the client,

typically through email.

management system certification audits

40

Opening meetings

• Each audit commences with an opening meeting. Typical

things addressed are:

• introduction of team members
• scope and methodology of the audit
• (understanding of) the audit plan
• any unresolved issues from earlier audits
• timing of the closing meeting

management system certification audits

41

Closing meetings

• Each audit ends with an closing meeting. In this meeting the

lead auditor provides its general impression on the

• rganization and more specifically the negative findings.
• There exist two types of negative findings:
• Minor nonconformities also known as deficiencies
• Major nonconformities
• Each of these is separately documented in a (draft) non-

conformity report and discussed with the client.

• The findings are (preferably) formally accepted (‘signed’) in

the closing meeting.

• Only negative findings are documented.

management system certification audits

42

Documentation audit

• Part of initial audit, also known as Stage I audit in ISO 27006.
• The auditor reviews the documentation and keeps interviews

to check consistence with the audit criteria.

• This audit will familiarize the auditor with the organization and

will allow him to formulate attention points for the Stage II (‘implementation’) audit.

ISO 27001 ETSI TS 101456

management system certification audits

43

Implementation audit

• Part of initial audit, also known as Stage II audit in ISO

27006.

• The auditor checks the existence of controls in consistence

with the documentation.

ISO 27001 ETSI TS 101456

management system certification audits

44

Certification decision

• The certification manager certification decides upon certification based on

the stage I and stage II reports.

• The certification manager must not have been part of the audit team.
• The client documents for each Non-Conformity a Corrective Action Report

(CAR) which includes a cause analysis, a corrective action and its planning.

• It is impossible to be certified if there still exist major non-conformities;

these need to be addressed and reassessed (‘follow-up audit’) before certification.

• The certificate is valid for three years and every year surveillance audits

are conducted (typically much smaller than an implementation audit).

• After three years a recertification audit is conducted, similar to the initial
• ne.

management system certification audits

45

Surveillance audit

• At least once a year, the certification body carries out a surveillance audit

consistent with the same requirements the initial audit was conducted.

• The surveillance is a combination of a documentation and implementation

audit.

• These periodic assessments serve to make sure all requirements are

assessed at least once during the certificates’ period of validity

• Each surveillance audit will address fixed elements as well:
• ‘open’ non-conformities
• the internal audits carried out by the organization,
• the complaints of customers,
• management reviews of the management system

management system certification audits

46

Audit time

• To conduct certification audits the auditor has rather limited

time, implying that the implementation audits are only of limited depth.

ISO 27006

management system certification audits

47

Outline

• Audit introduction
• IT security audits in general
• management system certification audits
• IT security product certification audits (‘common criteria’)
• Recap & Practicum

48

IT security product certification framework

IT product Common Criteria

(ISO15408 )

CC-CEM

(ISO 18045)

Evaluator (laboratory) CC Certificate

Scheme maintainer (ISO, BSI, TNO)

Accreditation Body (BSI, RVA)

IT security product certification audits (‘common criteria’)

49

NL product example (fox-it)

IT security product certification audits (‘common criteria’)

http://dl.dropbox.com/u/6343869/My%20First%20Common%20Criteria.pdf

50

IT product security

• Several governments have early recognized the inherent security risk of

computer systems, e.g.:

• the risks of not having the ‘right’ controls in the systems (security

functionality)

• the risks of not having adequate assurance that controls are properly

implemented (assurance)

• Security of a system is function of security functionality and assurance

Systems A and B could have the same security functionality (e.g. a password based authentication mechanism) but if the system A’s development of A is more thorough than that of system B; system A is probably more ‘secure’ than B.

• What is required is an IT-product security certification framework

enabling:

• ‘users’ to formulate their security needs in requirements for IT products
• manufacturers to develop (potential) conformant IT products
• technical laboratories to independently evaluate these IT products

against the set requirements

• ‘authorities’ to certify these IT products based on the evaluation report
• ‘users’ to apply these IT products in the right way (accreditation)

IT security product certification audits (‘common criteria’)

51

TCSEC

• In the 1980s the US defense department initiated the Trusted Computer

System Evaluation Criteria (TCSEC) program for assessing the effectiveness of computer security controls built into an operation system.

• It is commonly known as the Orange Book based on the color of its cover.

The Orange Book focuses on OS and leaves out many important information security aspects (such as networks!). This gave rise to many

• ther (colored) books resulting in what called the Rainbow series.
• The Orange Book distinguishes the following OS types:
• D: minimal protection
• C[1-2]: Discretionary protection: users can decide which information is

accessible by others’

• B[1-3]: Mandatory protection: information is labeled with classifications, e.g.

restricted, confidential, secret, and the system enforces access based on the clearance of users

• A[1]: Verified protection: builds further on B but includes formal design and

verification techniques.

• The classes are a rigid combination of security functionality and security

assurance.

IT security product certification audits (‘common criteria’)

52

ITSEC

• The critique on TCSEC is that it is rather rigid
• TCSEC focuses on confidentiality
• The TSCEC ratings are a fixed combination of functionality and assurance.
• TCSEC does not provide users flexibility in describing security requirements

different than those in TCSEC.

• In the 1990s France, Germany, the Netherlands, and the United Kingdom

published their own evaluation framework called the Information Technology Security Evaluation Criteria (ITSEC).

• ITSEC is more flexible and allows users more flexibility in describing their

security requirements than TCSEC. Moreover ITSEC separates functionality and assurance. ITSEC introduces 7 assurance classes E0 – E6 where E0 represents the lowest and E6 the highest assurance.

• ITSEC suggest a comparison between its assurance classes and the

implicit assurance classes in the TCSEC classes (D, C1, C2, B1, B2, B3, A1).

IT security product certification audits (‘common criteria’)

53

ITSEC assurance classes

IT security product certification audits (‘common criteria’)

54

Common Criteria

• According to some TCSEC is too hard and ITSEC is too soft.
• The Common Criteria for Information Technology Security Evaluation or

simply Common Criteria (or CC) is based on three underlying IT-product security certification frameworks: ITSEC (EU), TCSEC (US) and CTCPEC (Canada).

• The CC are published as ISO standards (ISO/IEC 15408):
• Part 1: Introduction and general model
• Part 2: Security functional requirements
• Part 3: Security assurance requirements
• The guidelines for the CC evaluators (Methodology for IT security

evaluation) is also published as an ISO standard (ISO 18045)

• They can be freely downloaded from

http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html

IT security product certification audits (‘common criteria’)

55

Short history

IT security product certification audits (‘common criteria’)

56

Protection Profile (PP)

• Typically a ‘user community ’ compiles a Protection Profile (PP) for a TOE

(Target of Evaluation) type, e.g., a firewall or a smartcard application (e.g., an SSCD).

• A PP defines an implementation-independent set of IT security

requirements for a category of TOEs which are intended to meet common consumer needs for IT security.

• A PP contains
• The TOE description,
• The TOE environment (including threats)
• Security Functional Requirements (SFRs) as specified in CC-part2
• Security Assurance Requirements (SARs) as specified in CC-part3
• Security Requirements for the IT Environment
• Security Requirements for the non-IT Environment
• A rationale
• Additional SFRs and SARs can be formulated.

IT security product certification audits (‘common criteria’)

57

Security Functional Requirements (SFRs) and Assurance Requirements (SARs)

Security Functional Classes Security Assurance Classes Class FAU: Security audit Class ACM:Configuration management Class FCO: Communication Class ADO:Delivery and

• peration

Class FCS: Cryptographic support Class ADV:Development Class FDP: User data protection Class AGD:Guidance documents Class FIA: Identification and authentication Class ALC:Life cycle support Class FMT: Security management Class APE:Protection Profile evaluation Class FPR: Privacy Class ASE:Security Target evaluation Class FPT: Protection of the TSF Class ATE:Tests Class FRU: Resource utilisation Class AVA:Vulnerability assessment Class FTA: TOE access Class FTP: Trusted path/channels

IT security product certification audits (‘common criteria’)

58

Evaluation Assurance Levels

• A PP also defines an Evaluation Assurance Level which in fact is a

package of SARs. The CC distinguishes 7 EAL levels from EAL1 to EAL7.

IT security product certification audits (‘common criteria’)

59

Security Targets

• In some cases the Protection Profiles contains more SARs than

necessary for a certain EAL level. In that case one uses the term ‘augmented’. So EAL 4 augmented (or EAL4+) means all SARs required in EAL 4 plus some additional ones.

• When creating a product in compliance with a PP, the manufacturer

creates a Security Target for its product. The manufacturer refers to the PP.

• In the evaluation process the product (TOE) is evaluated against the

SFRs by the evaluator (‘laboratory’) in accordance with the SARs.

• Based on the evaluation report typically another party certifies the

product, but in some schemes it is the laboratory itself. In Germany, the Bundesamt für Sicherheit in der Informationstechnik performs the accreditation of the laboratories (‘Prüfstelle’) and issues the certificates based on the evaluations.

IT security product certification audits (‘common criteria’)

60

PP example

IT security product certification audits (‘common criteria’)

61

PP example

IT security product certification audits (‘common criteria’)

62

Security Target example

IT security product certification audits (‘common criteria’)

63

Security Target example

IT security product certification audits (‘common criteria’)

64

Certificate example

IT security product certification audits (‘common criteria’)

65

NL Certificate example

IT security product certification audits (‘common criteria’)

http://dl.dropbox.com/u/6343869/My%20First%20Common%20Criteria.pdf

66

Certification of Secure Signature Creation Devices (SSCDs)

• An SSCD is a combination of hardware (‘chip’),

‘generic operation system’ and application.

• Nowadays many chip applications are applets based
• n the Java platforms (‘Javacards’). In many cases

the chip and the Java virtual machine (called ‘JCOP’ for NXP) are separately certified.

• In the Dutch context the certification of the SSCD

can be replaced by (roughly) by a suitably certified platform (chip + java VM) and a tested JAVA applet.

• See http://www.ecp.nl/sites/default/files/TTP-

NL_GuidanceNote2_June_2010.pdf

IT security product certification audits (‘common criteria’)

67

Links for certified products

• http://www.commoncriteriaportal.org/products.html
• https://www.bsi.bund.de/cln_134/DE/Themen/Zertifiz

ierungundAkkreditierung/ZertifizierungnachCCundIT SEC/ZertifizierteProdukte/zertifizierteprodukte_node. html

IT security product certification audits (‘common criteria’)

68

Outline

• Audit introduction
• IT security audits in general
• management system certification audits
• IT security product certification audits (‘common criteria’)
• Recap & Practicum

69

• Recap & Practicum
• See blackboard
• Please submit to f.vandenbroek@cs.ru.nl before

28 November 2011

• Room: 02.047