Improving the accuracy of estimates for complex sampling in auditing - - PowerPoint PPT Presentation

Improving the accuracy of estimates for complex sampling in auditing1.

• Y. G. Berger1
• P. M. Chiodini2
• M. Zenga2

1University of Southampton (UK) 2University of Milano-Bicocca (Italy)

14-06-2017

1The research leading to these results has received support under the European Commissions 7th Framework Programme (FP7/2013-2017) under grant agreement n312691, InGRID Inclusive Growth Research Infrastructure Diffusion. Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 1 / 26

Overview

1

Audit Sampling

2

Definitions

3

Simulation study

4

Conclusions and Future research

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 2 / 26

AUDIT SAMPLING

In auditing the goal is to verify if the values of the accounts reported by the company are not materially misstated To determine the total error in the amount reported by the company auditors should audit all accounts in the population of accounts. This is not possible as it is too costly and time expensive! In practice auditors verify only a sample of accounts to estimate the error of the total population of accounts (total amount error & error rate) Common statistical methods to select an audit sample are by without replacement or by probability proportional to size (PPS)

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 3 / 26

SAMPLING PLANS IN AUDITING

For the auditor the only interest is to verify that the error rate is within a pre-assigned value, if the observed rate is greater than the pre-assigned value a census is made. Practically speaking all the sampling methods are reliable for audit sampling, by the way Monetary Unit Sampling (MUS) - a particular case of the PPS - is the most popular! This sampling method directs efforts towards high-valued items which contain the greatest potential of large overstatement

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 4 / 26

SAMPLING PLANS IN AUDITING

The way the auditors make their choices in terms of sampling strategies is frequently based on personal experience! In general: book values distributions are highly positively skewed with different percentage of errors Two main scenarios can be met:

• a population with relatively large number of small accounts combined

with high rate of mistakes

• a population with relatively large number of accounts combined with a

small rate of mistakes

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 5 / 26

DEFINITIONS

An accounting population consist of N line items with book (or recorded) values, y1, y2, , yN and total book amount Ty defined by: Ty =

N

• i=1

yi. The audited (true) amount of the N line items in the population is denoted by x1, x2, ..., xN and the total audited amount is: Tx =

N

• i=1

xi.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 6 / 26

DEFINITIONS

The error in item i, is zi = yi − xi, 1 ≤ i ≤ N. When zi > 0, the i-th item is said to be overstated and when zi < 0, it is understated. When zi = 0, the account is said to be error free. The total error amount is defined as: Tz =

N

• i=1

zi. For yi = 0, ti = zi

yi = yi−xi yi

is called the fractional error or taint. The values (x1, x2, ..., xN ) are unknown before sampling, whereas (y1, y2, ..., yN) are known. It is assumed that the amount of any

• verstatement does not exceed the stated recorded value.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 7 / 26

DEFINITIONS

The purpose of the audit is to estimate the total error amount Tz: Tz =

n

• i=1

zi =

n

• i=1

ti × yi

• btained by the examination of a sample of n items of the account.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 8 / 26

Stringer Bound (Stringer, 1963)

Let T1, ..., Tn be the independent random variables which represent the taintings. The distribution of these taintings is some unknown mixture of distributions on the interval [0; 1], so that Pr(0 ≤ Ti ≤ 1) = 1. Let denote µ = E(Ti) and let 0 ≤ t1:n ≤ t2:n ≤ ... ≤ tn:n ≤ 1 be the

• rdered statistics of (T1, T2, ..., Tn).

For α ∈ (0; 1) and i = 0, 1, ..., n − 1 let p = pn(i; 1 − α) be the unique solution of:

i

• k=0

n k

• pk(1 − p)n−k = α

with pn(n; 1 − α) = 1.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 9 / 26

Stringer Bound (Stringer, 1963)

The Stringer method for obtaining an upper bound for the total

• verstatement error can be obtained by combining the upper limits

for the sample error rates with the taints:

ˆ Tz = Typn(0, 1 − α) + Ty

n

• i=1

[pn(i, 1 − α) − pn(i − 1, 1 − α)]tn−i+1:n then pn(i, 1 − α) is the (1 − α) upper confidence limit for the binomial parameter when i errors are observed in a sample of size n.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 10 / 26

Stringer Bound (Stringer, 1963)

Equivalently for a given α, n and number of errors i, it is possible to find the value pn(i, 1 − α) that satisfies:

i

• j=0

n j

• [pn(i, 1 − α)]j[1 − pn(i, 1 − α)]n−j = α.

The Stringer bound is sometimes calculated using the Poisson approximation for obtaining the upper confidence limits pn(i, 1 − α).

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 11 / 26

Empirical Likelihood estimator

EL is a non-parametric likelihood. Hartley and Rao in 1968 first introduced it in the context of survey sampling as scale-load approach. From early 2000 the EL approach has been introduced also in survey sampling literature. EL approach provides non-parametric confidence intervals similar to the parametric likelihood ratio intervals.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 12 / 26

Empirical Likelihood estimator

The shape and the orientation of the EL intervals are completely determined by the data. Chen et al. (2003) obtained EL intervals on the population mean for populations containing many zero values thats the case of audit sampling. Parametric likelihood ratio intervals based on parametric mixture distributions perform better than the standard normal theory intervals in terms of coverage, but EL intervals perform better under deviations from the assumed mixture model, by providing non-coverage rate below lower bound closer to the nominal value and also larger lower bound.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 13 / 26

EL approach (Berger, 2016)

Let U be a finite populations of N units. θ0 is the unique solution G(θ) = 0, G(θ) =

• i∈U

gi(θ) The empirical log-likelihood function is defined l(m) = log(

• i∈S

mi) =

• i∈S

log(mi) (1) mi is estimated by the value ˆ mi which maximize l(m) subject

mi ≥ 0

• i∈S

mic = C

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 14 / 26

EL approach (Berger, 2016)

The solution is given by ˆ mi = {(t + η)Tci}−1 = (πi + ηTci)−1 where πi = npi.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 15 / 26

Maximum EL estimator

The empirical log-likelihood ratio function is defined by ˆ r(θ) = 2{l( ˆ m) − l( ˆ m∗, θ)} where l( ˆ m) =

i∈S

log( ˆ mi) l( ˆ m∗, θ) =

i∈S

log( ˆ m∗

i (θ))

ˆ m∗

i (θ) is the value that maximize (1) subject to

• mi ≥ 0

i∈S

mic∗

i = C ∗ with c∗ i = (cT i , 0)T and C ∗ = (C T, 0)T

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 16 / 26

HOW TO EVALUATE AUDITING RISK

Previous studies has already demonstrated that different sampling plans such as Systematic Sampling or Probability Proportional to Size plans (such as Unrestricted Random Sampling, Lahiri Sampling, ...) are all reliable even for all sample size! We evaluate the efficiency of EL Bound respect the Stringer Bound. The parametr of interest is the ratio of error-per-Euro (Taint)

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 17 / 26

SIMULATION STUDY

We simulated data using a real accounting population of credit invoices of an audited society.

Value

5000 4000 3000 2000 1000

Frequency

40,0 30,0 20,0 10,0 0,0

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 18 / 26

SIMULATION STUDY

X ∼ LogNormal(7.001, 1.71). N = 10000. m = 1000 samples. Error randomly associated to the Xi (no fraudulent hypothesis) Error rates :

+ 5%, + 10% + 20%.

Taint simulated values:

• 0.1-0.3;
• 0.5-0.7;
• 0.2-0.7;

Sample fractions:

* 0.05 * 0.1

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 19 / 26

SIMULATION STUDY

To compare different estimators Tightness, MSE and Coverage Probability have been computed Tightness is used to indicate how close the bound value is to the true error rate:

(AV (ˆ t)−t) t

· 100 If it is small the sampling method is said to be tight and if it is large is said to be conservative Variability of the bound: this is an indicator of the uncertainty of the bound. This is measured by Mean Squared Error (MSE): 1 m

m

• i=1

(ˆ ti − t)2 where ˆ ti is the estimated value for the error rate at the ith replicate. Coverage probability: for a specific bound it refers to the proportion of replications for which a bound is greater than or equal to the true population error amount. A bound is considered unreliable if its coverage is significantly below the specified nominal coverage, otherwise it is reliable.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 20 / 26

Case Study I: Taint 0.1-0.3

EL Stringer % Error % n/N Tight. MSE COV Tight. MSE COV 5 5 35.16% 8.745E-05 0.976 49.13% 8.012E-04 1 5 10 24.12% 3.275E-05 0.999 32.16% 3.121E-04 1 10 5 23.75% 1.311E-04 0.978 30.88% 6.532E-04 1 10 10 15.84% 2.003E-04 0.987 21.38% 9.041E-04 0.996 20 5 15.50% 3.096E-04 0.969 19.44% 8.521E-04 0.988 20 10 34.78% 1.281E-04 0.962 47.20% 8.301E-04 0.991

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 21 / 26

Case Study II: Taint 0.5-0.7

EL Stringer % Error % n/N Tight. MSE COV Tight. MSE COV 5 5 35.18% 9.451E-05 0.964 61.31% 1.713E-03 1 5 10 24.12% 7.921E-04 0.963 41.21% 1.312E-03 1 10 5 23.89% 4.251E-05 0.9106 39.90% 9.101E-04 0.9984 10 10 16.50% 1.741E-04 0.984 25.86% 1.700E-04 1 20 5 15.43% 7.096E-05 0.978 23.46% 3.512E-04 1 20 10 10.99% 2.283E-05 0.984 15.73% 7.192E-04 1

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 22 / 26

Case Study III: Taint 0.2-0.7

EL Stringer % Error % n/N Tight. MSE COV Tight. MSE COV 5 5 37.37% 2.217E-04 0.971 70.82% 1.653E-03 1 5 10 24.77% 4.628E-05 0.99 38.27% 7.510E-04 1 10 5 25.23% 1.115E-04 0.971 36.36% 1.210E-03 0.998 10 10 16.44% 4.941E-05 0.97 24.96% 2.612E-04 1 20 5 16.91% 2.140E-04 0.912 24.09% 1.300E-03 0.989 20 10 11.09% 8.162E-05 0.953 16.27% 6.010E-04 0.995

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 23 / 26

Results

* EL is generally better than SB for estimating error rate * EL is tighter for the estimation of the real error rate. SB is more conservative! * MSE of SB is greater respect EL. * EL and SB can be considered reliable.

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 24 / 26

Conclusions and Future research

* We introduced di EL Bound to estimate the Upper Bound for the estimation of the error rate * In general EL Bound perform better respect the Stringer Bound. + Other distributions (Dagum, Gamma,...) for the X values. + Other hypothesys on the generation of the error (on the queues,...)

Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 25 / 26

References

Arens, A.A., Loebbecke, J.K., (1981). Applications of Statistical sampling to Auditing, Prentice-Hall, Inc.. Berger, Y. G., De La Riva Torres, O. (2015). Empirical likelihood confidence intervals for complex sampling designs, Journal of the Royal Statistical Society: Series B (Statistical Methodology),78(2),319-341. Chen, J., Chen, S. Y., and Rao, J. N. K. (2003). Empirical likelihood confidence intervals for the mean of a population containing many zero values. The Canadian Journal of Statistics 31, 53-68. Chiodini P.M., Zenga M. (2014) Efficiency of the sample plans for symmetric and non-symmetric distributions in auditing: a comparison. in Contribution to sampling Statistics, ed Mecatti F., Conti PL., Ranalli MG Horgan, J.M. (2008). Monetary-unit sampling old and new, School of Computing, Dublin City University, Dublin ,Ireland. Nandram, H.N., (2009). Applications of Statistical sampling to Auditing, Monetary unit sampling: Improving estimation

• f the total audit error, Advances in Accounting, incorporating Advances in International Accounting, 25, 174-182.

Owen Art B. (1988). Empirical likelihood ratio confidence intervals for a single functional, Biometrika 75(2), 237-49 Rao, J. N. K. (2006). Empirical Likelihood Methods for Sample Survey Data:An Overview AUSTRIAN JOURNAL OF STATISTICS, 35(2-3), 191196 Stringer, K.W. (1963). Practical aspects of statistical sampling in auditing. Proceedings of the Business and Economics Statistics Section, 405411. ASA Berger, Chiodini, Zenga Improving the accuracy of estimates 14-06-2017 26 / 26