Sampling Strategies in Financial Statement Audits Devising a Sampling - - PowerPoint PPT Presentation

Presenting a live 110‐minute teleconference with interactive Q&A

Sampling Strategies in Financial Statement Audits

Devising a Sampling Methodology That Meets AICPA Standards and Fortifies the Auditor's Opinion

1pm Eastern | 12pm Central | 11am Mountain | 10am Pacific WEDNESDAY, MARCH 23, 2011

Today’s faculty features:

1pm Eastern | 12pm Central | 11am Mountain | 10am Pacific

Jeanne Yamamura Director of Professional Services Mark Bailey & Co. Reno Nev Jeanne Yamamura, Director of Professional Services, Mark Bailey & Co., Reno, Nev. Laura Schweitzer, Director, PricewaterhouseCoopers, Washington, D.C. Lyn Graham, CPA, PhD., CFE, Bentley University, Waltham, Mass. Trevor Stewart, Senior Research Fellow, Rutgers University, New Brunswick, N.J.

For this program, attendees must listen to the audio over the telephone.

Trevor Stewart, Senior Research Fellow, Rutgers University, New Brunswick, N.J. Please refer to the instructions emailed to the registrant for the dial-in information. Attendees can still view the presentation slides online. If you have any questions, please contact Customer Service at1-800-926-7926 ext. 10.

Conference Materials

If you have not printed the conference materials for this program, please complete the following steps:

• Click on the + sign next to “Conference Materials” in the middle of the left-

hand column on your screen hand column on your screen.

• Click on the tab labeled “Handouts” that appears, and there you will see a

PDF of the slides for today's program.

• Double click on the PDF and a separate page will open.

Double click on the PDF and a separate page will open.

• Print the slides by clicking on the printer icon.

Continuing Education Credits

FOR LIVE EVENT ONLY

Attendees must listen to the audio over the telephone. Attendees can still view

the presentation slides online but there is no online audio for this program. Please refer to the instructions emailed to the registrant for additional

• information. If you have any questions, please contact Customer Service

at 1-800-926-7926 ext. 10. at 1 800 926 7926 ext. 10.

Tips for Optimal Quality

S d Q lit S

• und Qualit y

For this program, you must listen via the telephone by dialing 1-866-871-8924 and entering your PIN when prompted. There will be no sound over the web connection. co ect o . If you dialed in and have any difficulties during the call, press *0 for assistance. You may also send us a chat or e-mail sound@

straffordpub.com immediately so

we can address the problem.

Viewing Qualit y

To maximize your screen, press the F11 key on your keyboard. To exit full screen, press the F11 key again press the F11 key again.

S li St t i i Fi i l Sampling Strategies in Financial Statement Audits Seminar

March 23, 2011 Laura Schweitzer, PricewaterhouseCoopers laura.schweitzer@us.pwc.com Lyn Graham, CPA, Bentley University lgrahamcpa@verizon.net Jeanne Yamamura, Mark Bailey & Co. yamamura@unr.edu Trevor Stewart, Deloitte (Retired), Rutgers University trsny@verizon.net y

Today’s Program

Background On Relevant Guidance; Evolution Of Sampling Techniques

[Lyn Graham]

Slide 7 – Slide 16 Important Sampling Concepts

[Laura S chweit zer]

Slide 17 – Slide 23 The Audit Risk Model And Its Applicability

[Trevor S t ewart ]

Slide 24 – Slide 32 Current Sampling Priorities And Best Practices

[Trevor S t ewart , Laura S chweit zer and Jeanne Y amamura]

Frequently Faced Issues [All speakers] Slide 33 – Slide 49 Slide 50 Slide 52 Frequently Faced Issues [All speakers] Slide 50 – Slide 52

BACKGROUND ON RELEVANT

Lyn Graham, CPA, Bentley University

BACKGROUND ON RELEVANT GUIDANCE; EVOLUTION OF SAMPLING TECHNIQUES

A di Obj i Audit Objective

I. Gather sufficient evidence to support an audit opinion that the financial statements are free of material misstatement the financial statements are free of material misstatement II. Seek a high assurance (or a low risk) g ( )

• III. Sampling tests of controls and tests of balances and

transactions are important sources of audit evidence.

8

Implications To Entities And Auditors

I. Lower-risk entities require less auditor testing, and that

can reduce audit costs can reduce audit costs. II. Entities with reliable controls can reduce audit costs; risks ; are “covered” by controls.

• III. Internal auditors’ attention to controls and financial

reporting accuracy will allow external auditors to rely on their work and reduce audit costs.

9

Further Implications

I. All public companies must report on the effectiveness of internal controls (SEC requirement – SOX Section 404).

• A. Some non-public companies also report.

II. Auditors of accelerated filers also separately report.

• III. Tests of controls provide the support for the company

assertion re: controls’ effectiveness.

• IV. Quality testing by entities can reduce auditor testing and

reduce auditor costs.

10

R T d A d I li i Recent Trends And Implications

I. More attention was given to controls after frauds and business failures such as Enron Worldcom etc failures such as Enron, Worldcom, etc.

• A. Many studies of fraud and misstatement

II Improving controls reduces business and audit risks and audit costs II. Improving controls reduces business and audit risks and audit costs.

• III. Year one investment costs vs. subsequent returns from improved

financial reporting processes

• IV. Role of the COSO framework
• V. Separate sampling guidance for compliance audits under OMB A-133

11

Professional Audit Sampling Standards

I. Audit sampling (SAS 39, 111, 107) II. AICPA Audit Sampling Guide (2008)

• A. Sufficiency of sample sizes to meet audit objectives
• B. Determining sample sizes – tables, formulae
• C. Evaluating sample results and implications
• D. Practical application issues

12

A di Ri k M d l Audit Risk Model

I. Audit risk (AR) is the risk that the auditor may unknowingly I. Audit risk (AR) is the risk that the auditor may unknowingly fail to appropriately modify his or her opinion on financial statements that are materially misstated [SAS 107, para. 2]. II A 5% dit i k i ll id d l i k II. A 5% audit risk is generally considered low-risk. AR = IR x CR x DR I. Components of audit risk (AR)

• A. Inherent Risk (IR) Risk Of Material
• A. Inherent Risk (IR) Risk Of Material

Misstatement

• B. Control Risk (CR)
• C. Detection Risk (DR) –Analytics and Detail Tests

13

A di Ri k M d l A di S Audit Risk Model: Audit Strategy

Risk of Material

IR x CR = RMM x DR = AR (5%)

Inherent Risk Control Risk Risk of Material Misstatement Detection Risk 100% 100% 100% 100% 5% 5% 100% 100% 100% 100% 5% 5% 80% 50% 50% 40% 12.5% 12.5% 50% 10% 10% 5% N/A N/A

14

SAS A di A SAS 111 Appendix A

15

How Does The Audit Risk Model Aff A di ? Affect Audit?

I. Sampling procedures are used in:

• A. Tests of controls
• B. Tests of details (substantive)

II. Analytical procedures are also substantive tests and are a factor (DR = AP x Substantive Details Test Risk) III Tests of controls do not always involve sampling

• III. Tests of controls do not always involve sampling.
• A. Automated controls vs. manual controls

B Control environment assessments – Competence

• B. Control environment assessments Competence
• IV. Special consideration for small populations

16

IMPORTANT SAMPLING

Laura Schweitzer, PricewaterhouseCoopers

IMPORTANT SAMPLING CONCEPTS

Sa m p ling Term inology Sa m p ling Term inology

Audit sam pling Audit sam pling

• Audit sampling is the application of audit procedures to less than

100% of the population.

• Audit samples can be non-statistical or statistical (probability based).
• Anytime a sample is selected, there is some risk that the estimates

derived from the sample do not accurately reflect the population. In p y p p statistical audit sampling, it is customary to quantify this risk using two parameters:

• Risk of incorrect acceptance

s o co ect accepta ce

• Precision

PwC 18 March 2011 Sampling Basics

Sa m p ling Term inology (Cont ) Sa m p ling Term inology (Cont.)

Risk of incorrect acceptance Risk of incorrect acceptance

• The risk that the sample supports the conclusion that the population

is not materially misstated, when in fact it is misstated (this is the confidence level in statistical terms) confidence level in statistical terms).

• Typically, 90% and 95% are used.

Precision

• The range around the sample estimate
• We typically want to be as precise as possible, within time and budget

constraints.

PwC 19 March 2011 Sampling Basics

Sa m p ling Term inology (Cont ) Sa m p ling Term inology (Cont.)

Expected m isstatem ent

• The expected amount of error in the population

Tolerable m isstatem ent h f i h l i h i d d bl

• The amount of error in the population that is deemed acceptable to

the auditor. The tolerable misstatement should include the expected misstatement and an allowance for sampling risk (precision in statistical terms) statistical terms).

PwC 20 March 2011 Sampling Basics

Sa m p ling Pa ra m eters Sa m p ling Pa ra m eters

The selection of sam pling param eters affects the sam ple size: The selection of sam pling param eters affects the sam ple size: Precision Sample Size Confidence Sample Size Attribute Expected Accuracy Rate Sample Size y Variability Sample Size

PwC 21 March 2011 Sampling Basics

Sa m p ling Pla n Ba sics Sa m p ling Pla n Ba sics

A sam pling plan should address the following: A sam pling plan should address the following:

• Objective
• What is the purpose of the sample?
• Is sampling appropriate?
• What will be measured from the sample?
• Population definition
• Target parameters
• How much uncertainty is tolerable?

PwC 22 March 2011 Sampling Basics

Sa m p ling Pla n Ba sics (Cont ) Sa m p ling Pla n Ba sics (Cont.)

A sam pling plan should address the following (Cont.): A sam pling plan should address the following (Cont.):

• Sample design
• Sample size
• What sample size is necessary to achieve the target sampling

parameters? parameters?

• Sample selection methods
• Extrapolation methodology
• How will sample results be evaluated?

PwC 23 March 2011 Sampling Basics

THE AUDIT RISK MODEL AND

Trevor Stewart, Deloitte (Retired), Rutgers University

ITS APPLICABILITY

Revised Audit Sampling Guide

• Updated from 2008 edition
• Expanded controls guidance

Expanded controls guidance

– Small population guidance

• New tables and more guidance

I l d id /t f i k t t d d

• Includes guidance/terms from risk assessment standards
• Multi-location auditing guidance

Technical Notes

• n the AICPA Audit Guide
• Companion publication
• Contains a detailed technical analysis of the tables in

the Sampling Guide

Audit Sampling

New Edition as of May 1, 2008

the Sampling Guide

• Includes the Excel and Excel VBA algorithms used

to compute the tables, thus providing extensibility beyond the tabulated values il bl f li f h

Trevor R. Stewart

Deloitte & Touche LLP Member of the 2008 Audit Sampling Guide Task Force

• PDF available free online from the AICPA at

http://www.aicpa.org/Publications/AccountingAuditing /KeyTopics/Pages/AuditSampling.aspx

25

The Audit Risk Model (ARM), AU 312.21-.26 AR RMM × DR =

AR = Audit Risk RMM = Risk of Material Misstatement IR = Inherent Risk

AR RMM × DR

IR × CR AP × TD = =

IR Inherent Risk CR = Controls Risk DR = Detection Risk AP = Analytical Procedures Risk In planning an audit sample to achieve AR, h RMM h b d d AP i AP Analytical Procedures Risk TD = Test of Details Risk

Acceptable Sampling Risk

where RMM has been assessed and AP is known, TD is the acceptable sampling risk

• f incorrect and can be calculated by

applying the ARM in reverse.

cceptab e Sa p g s AR TD RMM AP  

pp y g

For example, 5% AR If 5%, 50%, 30 5% 33% 5 %, the 0% 3 % n AR A TD RMM AP R RMM AP        

26

Types Of Tests

(AICPA Sampling Guide §2.09-2.12)

Tests of controls

• Provide evidence about the effectiveness of the design, implementation or operation of a

control in preventing or detecting material misstatements in a financial statement assertion

• Are necessary when the audit strategy is to rely on the effectiveness of the control
• Some controls cannot be tested using audit sampling

Substantive tests

• Are audit procedures designed to obtain evidence about the validity and propriety of the

accounting treatment of transactions and balances or to detect misstatements (they may also reveal deficiencies in controls) h di i i d i il i l i b d ll hi h i il h

• The auditor is interested primarily in a conclusion about dollars, which is not necessarily the

case in tests of controls.

• Substantive tests include (1) tests of details of transactions and balances, and (2) analytical

procedures procedures. Dual-purpose tests

• Test the effectiveness of controls and also whether a recorded balance or class of transactions

is materially misstated. In sampling, the same sample is used for both purposes. is materially misstated. In sampling, the same sample is used for both purposes.

• Requires a preliminary judgment about the effectiveness of controls — which may need to be

revised, thus also affecting the associated substantive test

27

Design Of Attribute Samples For Controls Testing

Table A.2 Statistical sample sizes for tests of controls: 5% risk of overreliance (abbreviated table)

2% 3% 4% 5% 6% 7% 8% 9% 10% 0.00% 149 (0) 99 (0) 74 (0) 59 (0) 49 (0) 42 (0) 36 (0) 32 (0) 29 (0) Tolerable Deviation Rate Expected Deviation Rate 0.25% 236 (1) 157 (1) 117 (1) 93 (1) 78 (1) 66 (1) 58 (1) 51 (1) 46 (1) 0.50% 313 (2) 157 (1) 117 (1) 93 (1) 78 (1) 66 (1) 58 (1) 51 (1) 46 (1) 0.75% 386 (3) 208 (2) 117 (1) 93 (1) 78 (1) 66 (1) 58 (1) 51 (1) 46 (1) 1.00% 590 (6) 257 (3) 156 (2) 93 (1) 78 (1) 66 (1) 58 (1) 51 (1) 46 (1) 1.25% 1,030 (13) 303 (4) 156 (2) 124 (2) 78 (1) 66 (1) 58 (1) 51 (1) 46 (1) 1 50% 392 (6) 192 (3) 124 (2) 103 (2) 66 (1) 58 (1) 51 (1) 46 (1) 1.50% 392 (6) 192 (3) 124 (2) 103 (2) 66 (1) 58 (1) 51 (1) 46 (1) 1.75% 562 (10) 227 (4) 153 (3) 103 (2) 88 (2) 77 (2) 51 (1) 46 (1) 2.00% 846 (17) 294 (6) 181 (4) 127 (3) 88 (2) 77 (2) 68 (2) 46 (1) 2.25% 1,466 (33) 390 (9) 208 (5) 127 (3) 88 (2) 77 (2) 68 (2) 61 (2) 2.50% 513 (13) 234 (6) 150 (4) 109 (3) 77 (2) 68 (2) 61 (2) 2.75% 722 (20) 286 (8) 173 (5) 109 (3) 95 (3) 68 (2) 61 (2) 2.75% 722 (20) 286 (8) 173 (5) 109 (3) 95 (3) 68 (2) 61 (2) 3.00% 1,098 (33) 361 (11) 195 (6) 129 (4) 95 (3) 84 (3) 61 (2) 3.25% 1,936 (63) 458 (15) 238 (8) 148 (5) 112 (4) 84 (3) 61 (2) 3.50% 624 (22) 280 (10) 167 (6) 112 (4) 84 (3) 76 (3) 3.75% 877 (33) 341 (13) 185 (7) 129 (5) 100 (4) 76 (3) 4.00% 1,348 (54) 421 (17) 221 (9) 146 (6) 100 (4) 89 (4)

Example (see Table 3.3 in Guide):

• Deviation rates: 2% expected, 5% tolerable

S l i 181 it

5.00% 1,580 (79) 478 (24) 240 (12) 158 (8) 116 (6)

• Sample size = 181 items
• Expected number of deviations in sample, 2% × 181 = 4

28

Evaluation Of

Table A.3 Statistical sampling results evaluation table for tests of controls: Upper limits at 5% risk of overreliance (abbreviated table)

Actual Number of Deviations Found, k Sample

Evaluation Of Attribute Samples For

1 2 3 4 5 6 7 8 9 10 20 14.0 21.7 28.3 34.4 40.2 45.6 50.8 55.9 60.7 65.4 69.9 25 11.3 17.7 23.2 28.2 33.0 37.6 42.0 46.3 50.4 54.4 58.4 30 9.6 14.9 19.6 23.9 28.0 31.9 35.8 39.4 43.0 46.6 50.0 35 8 3 12 9 17 0 20 7 24 3 27 8 31 1 34 4 37 5 40 6 43 7 , Sample Size, n

p Controls Testing

35 8.3 12.9 17.0 20.7 24.3 27.8 31.1 34.4 37.5 40.6 43.7 40 7.3 11.4 15.0 18.3 21.5 24.6 27.5 30.4 33.3 36.0 38.8 45 6.5 10.2 13.4 16.4 19.2 22.0 24.7 27.3 29.8 32.4 34.8 50 5.9 9.2 12.1 14.8 17.4 19.9 22.4 24.7 27.1 29.4 31.6 55 5.4 8.4 11.1 13.5 15.9 18.2 20.5 22.6 24.8 26.9 28.9 60 4 9 7 7 10 2 12 5 14 7 16 8 18 8 20 8 22 8 24 8 26 7 60 4.9 7.7 10.2 12.5 14.7 16.8 18.8 20.8 22.8 24.8 26.7 65 4.6 7.1 9.4 11.5 13.6 15.5 17.5 19.3 21.2 23.0 24.7 70 4.2 6.6 8.8 10.8 12.7 14.5 16.3 18.0 19.7 21.4 23.1 75 4.0 6.2 8.2 10.1 11.8 13.6 15.2 16.9 18.5 20.1 21.6 80 3.7 5.8 7.7 9.5 11.1 12.7 14.3 15.9 17.4 18.9 20.3 90 3 3 5 2 6 9 8 4 9 9 11 4 12 8 14 2 15 5 16 9 18 2

Example:

• Sample size = 100

90 3.3 5.2 6.9 8.4 9.9 11.4 12.8 14.2 15.5 16.9 18.2 100 3.0 4.7 6.2 7.6 9.0 10.3 11.5 12.8 14.0 15.2 16.4

5% 95% 5% 95%

▲ ▲

Sample size 100

• Number of deviations found = 3
• Most likely population deviation rate is 3/100 = 0.03
• Upper 5% limit = 0.076

▲ 0.03 ▲ 0.076

• Evaluation can be depicted as a probability distribution

with a peak at 0.03 and 95th percentile at 0.076.

In Excel (Sampling Guide, Technical Notes), BETAINV(1−risk,1+k,n−k) = BETAINV(95%,1+3,100−3) = 0.076 29

Guidance For Small Population Test Levels

(AICPA Sampling Guide Table 3.5)

T bl 3 5 Table 3.5 Small Population Sample Size Table C l F d S l Control Frequency and Population Size Sample Size Q l (4) 2 Quarterly (4) 2 Monthly (12) 2–4 Semimonthly (24) 3–8 y ( ) Weekly (52) 5–9

30

Compact MUS Sample Size Table

(AICPA Sampling Guide Table C.2)

Confidence factors for monetary unit sample size design

5% 10% 15% 20% 25% 30% 35% 37% 50% 0.00 3.00 2.31 1.90 1.61 1.39 1.21 1.05 1.00 0.70 Ratio of Expected to Tolerable Misstatement Risk of Incorrect Acceptance

Step Example

• 1. Determine allowable Risk
• f Incorrect Acceptance

37%

0.05 3.31 2.52 2.06 1.74 1.49 1.29 1.12 1.06 0.73 0.10 3.68 2.77 2.25 1.89 1.61 1.39 1.20 1.13 0.77 0.15 4.11 3.07 2.47 2.06 1.74 1.49 1.28 1.21 0.82 0.20 4.63 3.41 2.73 2.26 1.90 1.62 1.38 1.30 0.87 0.25 5.24 3.83 3.04 2.49 2.09 1.76 1.50 1.41 0.92 0.30 6.00 4.33 3.41 2.77 2.30 1.93 1.63 1.53 0.99 6 92 4 95 3 86 3 12 2 57 2 14 1 79 1 67 1 06

• 2. Determine Tolerable

Misstatement (%) 3%

• 3. Determine the Ratio of

Expected Misstatement to 0.20

0.35 6.92 4.95 3.86 3.12 2.57 2.14 1.79 1.67 1.06 0.40 8.09 5.72 4.42 3.54 2.89 2.39 1.99 1.85 1.14 0.45 9.59 6.71 5.13 4.07 3.29 2.70 2.22 2.06 1.25 0.50 11.54 7.99 6.04 4.75 3.80 3.08 2.51 2.32 1.37 0.55 14.18 9.70 7.26 5.64 4.47 3.58 2.89 2.65 1.52 0.60 17.85 12.07 8.93 6.86 5.37 4.25 3.38 3.09 1.70

p Tolerable Misstatement

• 4. Look up Factor

1.30

• 5. Calculate Sample Size

1.30/0.03 = 44 Factor Sample Size Tolerable Misstatement % 

Dual-purpose samples p p p This table may also be used to determine the sample size for a dual-purpose sample, in which case the more stringent of the two purposes would be used to determine the total sample size.

31

MUS Evaluations: Projected

AICPA Sampling Guide Table C.3 MUS: Confidence factors for sample evaluation (abbreviated table)

Risk of Incorrect Acceptance

Projected Misstatement And Upper Limit

5% 10% 15% 20% 25% 30% 35% 37% 50% 3.00 2.31 1.90 1.61 1.39 1.21 1.05 1.00 0.70 1 4.75 3.89 3.38 3.00 2.70 2.44 2.22 2.14 1.68 2 6.30 5.33 4.73 4.28 3.93 3.62 3.35 3.25 2.68 Risk of Incorrect Acceptance Number of Overstatements, k

Upper Limit

3 7.76 6.69 6.02 5.52 5.11 4.77 4.46 4.35 3.68 4 9.16 8.00 7.27 6.73 6.28 5.90 5.55 5.43 4.68 5 10.52 9.28 8.50 7.91 7.43 7.01 6.64 6.50 5.68 6 11.85 10.54 9.71 9.08 8.56 8.12 7.72 7.57 6.67 7 13.15 11.78 10.90 10.24 9.69 9.21 8.79 8.63 7.67 8 14.44 13.00 12.08 11.38 10.81 10.31 9.85 9.68 8.67

Example

• Sample design and results

S li i l $1 000

9 15.71 14.21 13.25 12.52 11.92 11.39 10.92 10.74 9.67 10 16.97 15.41 14.42 13.66 13.02 12.47 11.98 11.79 10.67

– Sampling interval = $1,000 – Number of overstatements = 3 – Allowable risk = 5%

• Sample evaluation

Projected misstatement is 3 × $1 000 $3 000

5% 95% 5% 95%

$’000

▲ ▲ – Projected misstatement is 3 × $1,000 = $3,000 – Factor = 7.76 – Upper 5% limit is $1,000 × 7.76 = $7,760 – Calculation a bit more complicated when partial errors are encountered ▲ $3,000 ▲ $7,760

• Evaluation can be depicted as a probability distribution

with peak at $3,000 and 95th percentile at $7,760

In Excel (Sampling Guide, Technical Notes): GAMMAINV(1−risk,1+k,SamplingInterval) = GAMMAINV(95%,1+3,1000) = 7760 32

Trevor Stewart, Deloitte (Retired), Rutgers University Laura Schweitzer, PricewaterhouseCoopers

CURRENT SAMPLING

Laura Schweitzer, PricewaterhouseCoopers Jeanne Yamamura, Mark Bailey & Co.

PRIORITIES AND BEST PRACTICES PRACTICES

Reliance On Internal Controls Reliance On Internal Controls

• Design and implementation assessment is required: Essential to

understanding the business

• Tests of operating effectiveness are required if a controls reliance strategy

Tests of operating effectiveness are required if a controls reliance strategy is implemented (which depends on D&I).

– Also required for an assertion regarding controls (AT 501 or AS 5)

• Controls reliance is pretty much essential for large, complex entities with

high transaction volumes such as financial institutions high transaction volumes, such as financial institutions .

• What controls to test

– The “important” ones – Risk assessment: Likelihood and magnitude of possible misstatement

• The role of “walk-throughs” discussed in Audit Sampling Guide, §3.25

– Design and implementation – Operating effectiveness – Automated IT environment with good general controls versus manual environment – Automated IT environment with good general controls versus manual environment – How much assurance?

• In the end, the auditor’s assessment of RMM (= IR × CR) is a professional

judgment, informed in part by the results of sampling.

34

Audit Sampling

The application of an audit procedure to less than 100% of the items … for the purpose of evaluating some characteristic

Statistical sampling Non-statistical sampling p g

• Any approach to sampling that has the following

characteristics: – Random selection of sample, and U f b bili h l l l p g

• Audit sampling that is non-

statistical

• SAS 111 states, “… non-

statistical sampling – Use of probability theory to evaluate sample results, including measurement of sampling risk

• Types of test

– Control tests statistical sampling …

• rdinarily … would result in

a sample size comparable to the sample size from an efficiently designed statistical

• Attribute sampling most commonly used
• But, MUS may be more suitable for dual-purpose

tests

– Substantive tests efficiently designed statistical sample, considering the same sampling parameters.”

• Various approaches often

d i d f t ti ti l Substantive tests

• MUS: Monetary unit sampling
• Classic variables sampling

– Dual-purpose tests

S t t t ( t l d b t ti ) li d t

derived from statistical sampling

• Sample size penalties often

built in to account for non-

• Separate tests (control and substantive) applied to

same selected sample item

• MUS may be most useful selection method

statistical selection and evaluation

35

Pop ula tion Definition Pop ula tion Definition

A population that is not defined properly can lead to A population that is not defined properly can lead to m isleading results. Consider the following:

• What is the time period of interest?
• Is the population available electronically?
• What is the sampling unit (for example, transaction)?
• Are population data available for the testable unit (e g transaction
• Are population data available for the testable unit (e.g., transaction,

journal entry)?

• Is the population accurate and complete?
• Does the population include items that are not of interest?
• Are data available for stratification purposes?

PwC 36 March 2011 Sampling Basics

Ap p rop ria te Sa m p le Designs Ap p rop ria te Sa m p le Designs

There are m any appropriate sam ple designs There are m any appropriate sam ple designs

• Often, a simple random sample is used
• A stratified random sample may be used:
• To increase the precision of the sample results, or
• When estimates are required for sub-groups of the population.
• Dollar-unit sampling

Dollar unit sampling

• Population items with larger dollar values have a higher probability of

selection. Cluster/ multi stage sampling

• Cluster/ multi-stage sampling
• Cluster and multi-stage sampling may be useful when population data are

available at a various levels (for example, data are available at the journal entry level but testing occurs at a transaction level) or when there are

PwC

entry level, but testing occurs at a transaction level), or when there are multiple locations, contracts, etc.

37 March 2011 Sampling Basics

Sa m p le Selection Method s Sa m p le Selection Method s

Various sam ple selection m ethods include:

• Random number generator (readily available in multiple software

packages)

• Systematic sampling techniques in which every nth item is selected

after a random starting place is determined g p

• Dollar unit sampling uses a random starting place and a systematic

sampling technique in which every nth dollar is selected.

PwC 38 March 2011 Sampling Basics

SOX Requirements SOX Requirements

• Sarbanes-Oxley Act of 2002
• Management required to:

– Perform a formal assessment of ICFR – Include tests that confirm the design and operating effectiveness

• f controls
• Auditors required to:

– Evaluate management’s assessment process – Obtain reasonable assurance that no material weaknesses exist as of the assessment date

39

The Role Of Sampling The Role Of Sampling

• Management

– Assesses whether controls operating effectively as of Assesses whether controls operating effectively as of assessment date – May use samples to verify that controls operating effectively Question then arises: How many need to be tested? – Question then arises: How many need to be tested? – Related question re-testing of internal controls “fixed” during year

40

The Role Of Sampling (Cont.) The Role Of Sampling (Cont.)

• Auditors

– Obtain evidence to verify that control has operated effectively for a “sufficient” period of time – Will typically use samples to test yp y p – Same question: How many need to be tested?

• Management’s assessment process affects the auditor’s testing
• Management s assessment process affects the auditor s testing.
• If process was performed properly and documented sufficiently,

dit b bl t d th i t ti auditors may be able to reduce their testing.

41

Sampling For Smaller Firms A d C i And Companies

Widespread misunderstandings exist 1. Selection of specific items vs. audit sampling 1. Selection of specific items vs. audit sampling Example: – Small audit client – Substantive audit (no planned reliance on internal controls) Substantive audit (no planned reliance on internal controls) – Selection of sample of 30 cash disbursements to test

• perating expenses
• Auditor selects repair and maintenance items, investment

ud to se ects epa a d a te a ce te s, est e t expense items and the remainder “other” items

• This is not a case of audit sampling!

42

Sampling For Smaller Firms A d C i (C t ) And Companies (Cont.)

Widespread misunderstandings exist (Cont.) 2. Use of “haphazard selection” – By definition, selection without conscious bias – Used in error when selecting items from specific accounts believed to be more likely to contain misstatement

43

Sample Size Determination Sample Size Determination

• Switch to non-statistical sampling
• Related changes in sample sizes

– Smaller (and smaller and smaller) Arbitrary – Arbitrary

• Documentation differences

44

SAS 111 (AU 350) SAS 111 (AU 350)

• An auditor who applies non-statistical sampling uses professional

judgment to relate [the identified] factors in determining the j g [ ] g appropriate sample size. Ordinarily, this would result in a sample size comparable to the sample size resulting from an efficient and effectively designed statistical sample, using the same sampling parameters.

45

SAS 111 (AU 350), Cont. SAS 111 (AU 350), Cont.

• Factors to be considered (and documented) for test of controls

– Tolerable rate of deviations – Likely rate of deviations – Allowable risk of assessing control risk too low Allowable risk of assessing control risk too low

46

SAS 111 (AU 350), Cont. SAS 111 (AU 350), Cont.

Example: Assume 10% risk of assessing control risk too low

• A sample size of 30 implies:
• A sample size of 30 implies:

– Tolerable rate between 7% and 8% and expected population error rate = 0.0% OR OR – Tolerable rate between 10% and 15% and expected population error rate = 0.25%

• A sample size of 60 implies:

A sample size of 60 implies: – Tolerable rate between 3% and 4% and expected population error rate = 0.0% OR – Tolerable rate between 6% and 7% and expected population error rate = 0.25%

47

Practical Guidance For Auditors A d C i And Companies

• Plan up-front what you are going to do

– Wholly substantive approach

• Directed testing or audit sampling

Directed testing or audit sampling

• If non-statistical sample, document factors and rationale for

sample size determination Testing internal controls – Testing internal controls

• Which controls?
• If testable, how frequently are they performed?

– Document!

48

Training And Understanding Training And Understanding

• Use of statistical sampling software or pre-printed forms
• Adoption of non-statistical sampling
• Tendency to believe that no knowledge necessary!

49

Lyn Graham, CPA, Bentley University Laura Schweitzer, PricewaterhouseCoopers Trevor Stewart, Rutgers University

FREQUENTLY FACED ISSUES

g y Jeanne Yamamura, Mark Bailey & Co.

Q

F l F d Q i /I Frequently Faced Questions/Issues

• Is a projection from the sample to the population required,

even if the misstatement found is small in amount? ― Thoughts from today’s speakers Thoughts from today s speakers

• Can auditors look to the sample projection or deviation rates
• r misstatements, in order to assess the severity of control

deficiencies? ― Thoughts from today’s speakers What is the greatest misunderstanding in the area of the

• What is the greatest misunderstanding in the area of the

economics of sampling? ― Thoughts from today’s speakers

51

Frequently Faced Questions/Issues (Cont.)

• Can I extrapolate dollars in error, using an attribute sample?

― Thoughts from today’s speakers C I l l it ?

• Can I replace sample items?

― Thoughts from today’s speakers

• What happens if the population is incorrectly defined (i e
• What happens if the population is incorrectly defined (i.e.,

the population includes items it shouldn’t or excludes a portion)? ― Thoughts from today’s speakers

• What do I do with the sample results?

Th h f d ’ k ― Thoughts from today’s speakers

52