Auditing : the good, the bad and the ugly A. Feinberg 14 Sep 2017 - - PowerPoint PPT Presentation

14 Sep 2017

Auditing : the good, the bad and the ugly

1

• A. Feinberg

14 Sep 2017

 I must pay a debt of gratitude to

DeWitt Beeler, who in an article published in the 1999 May Issue of Quality Progress provided me with a fresh look at auditing and infused me with the motivation to improve my auditing skills.

2

14 Sep 2017

 While I may tonight be looking

critically at aspects of how audits are traditionally conducted, I have the utmost respect for the conscientious people who perform the audit

• function. I do not mean in any way to

denigrate their efforts.

3

14 Sep 2017

 So much of what we do in our work lives

today is mandated.

 We have a doctor friend who is

absolutely inundated with paperwork. She works as a family physician during the day and then spends hours pretty much every evening doing the required paperwork.

4

 For all the effort we expend on

mandated tasks such as audits, I think it absolutely imperative to take a hard look at the assumptions, methods, and

• utput of activities in which we

engage.

 If I can stimulate thinking on these as

regarding the audit process I will judge tonight’s talk a success.

5

14 Sep 2017

Territory to Cover –  Purpose - What is the purpose of an

audit?

 Process - How effective are our

audits?

 Pitfalls - What are some of the

• bstacles that prevent us from

achieving our audit goals?

6

We hunger for relevance and purpose in

• ur lives.

When we know we are making a positive

impact due to the results of our efforts we are motivated to perform at the top of our game.

7

Life in a corporate environment of which

we are only play a seemingly minor role can at times appear to devolve into a series of meaningless exercises.

Scott Adams’ Dilbert cartoon series

explored these feelings and gave expression to them.

8

9

But that’s not the ‘Dream Job’ is it? It is more likely the “Nightmare Job”

… you work for a month and at the end it doesn’t matter if from day 1 you had just made up the results.

10

14 Sep 2017

 How many of you have either conducted

an audit or been involved in an audit?

 What have been your impressions of the

audit function?

11

14 Sep 2017

 Obviously I don’t take the pointy

haired boss’ approach to the audit work we do: I believe that there are benefits that can be derived from the audit function when its’ objectives and limitations are kept in mind.

12

14 Sep 2017

 A well conceived and executed audit

can yield information that is important to the healthy operation of an

• rganization; whether that
• rganization exists for the purpose of

manufacturing a product or providing a service.

13

 If we are going to take a fresh look at the

audit function we’ve first got to clear the land a bit.

You might even say we need to “Drain the

Swamp” not of people, but of assumptions that can’t endure the light of day.

Let’s look at some common misconceptions

about audits. Beeler calls them the “Big Lies” about audits.

14

Audits are about “Compliance” or the

extent to which procedures incorporate requirements.

Audits should focus exclusively on

“Facts”

Auditors should not make “Judgments”

15

Audits are a verification activity. Audits are the primary tool to drive

Continuous Improvement.

Internal Audits should emulate their

Registrar’s Audit.

16

14 Sep 2017

Companies / Organizations exist to provide a product or service for their customers. In order to do that there are four essentials that must be met.

17

14 Sep 2017

Customer Expectations - the organization must be able to satisfy and exceed. Compliance - meet any and all requirements as imposed by regulatory bodies. Consistency - must be able to deliver the same product or service reliably day to day, week to week and year to year. Cost control - All the above has to be accomplished at a cost the market will bear.

18

14 Sep 2017

 Although human error is all around us, in an

• rganization it is usually not actually the root cause of

problems.

 It is instead only one of multiple causes that must be

dealt with.

 Unfortunately, organizations try to change human

behavior rather than the system within which the individual is working, then are frustrated when problems recur.

 However, industries can significantly reduce the rate of

human error by focusing on business process design while better considering human factors.

19

14 Sep 2017

• Quality Systems are designed and

implemented in order to provide the apparatus that will accomplish the 4 musts of a successful

• rganization.
• It is the job of managers to maintain and

improve their systems, but in order to do so they need information.

20

14 Sep 2017

• Just as a doctor runs a battery of tests during a

typical physical to gain information on the health of your organization, i.e. your body, in

• rder to determine the health of the myriad of

systems which comprise you so can a manager gain needed information from the audit process.

21

14 Sep 2017

 In order to produce their product in a consistent, cost

effective manner that doesn’t destroy the environment

• r unnecessarily place their employees lives at risk

management establishes systems and appoint managers over those systems to ensure all is running as intended.

 Managers and their teams rely on information to fine

tune and keep the engine running with a minimum of delays, disruptions and hopefully zero disasters.

22

Therefore audits in order to be successful

must focus not on compliance, nor on verification activities, nor simply on facts, but on the entire universe of the system

• r systems they are auditing.

Whatever is critical to the manager

should be to the auditor as well. The auditor should be the manager’s eyes, ears, and to a certain extent brain.

23

Not that the auditor thinks for the

manager, but instead interprets and presents the information collected from the audit to reach measured conclusions concerning the systems being audited.

24

So the assumption that Auditors should not

make judgments is entirely in error.

As a society we shrink from the word

“judgment”. It conjures up someone who is intolerant of other people’s views.

But as auditors we must employ the highest

form of judgment, i.e ‘the ability to make considered decisions or come to sensible conclusions about the systems we are charged with evaluating.’

25

 The cardinal truth about auditing is that it should

be a process of inferential reasoning.

 The inputs are the observations we make during

an audit.

 We compare those observations with established

standards of quality organizations, goals established by internal documentation, and our

• wn knowledge base to come up with

conclusions about the system(s) we are auditing.

 These conclusion(s) are then relayed to the

management team as actionable intelligence.

26

Often though the final audit report is a list

• f “findings” that can be perceived by

management as simple lapses in their process that can easily be corrected by retraining a particular operator, making a process work instruction clearer, etc. but which do not address the underlying causes for the lapse, nor provide information that will direct management where and how to spend their resources.

27

Imagine a football team that consistently

has a losing record.

The coach and general manager of the

team has a lot of work to do.

But where to start? Where to expend

precious resources?

Let’s imagine an audit of our team we’ll

call the 'Spiritless Warriors is commissioned.

28

The final audit report goes something

like this:

• 1. Finding - left tackle missed Tuesday’s practice.
• 2. Finding - projector is not being maintained

according to team’s maintenance records.

• 3. Finding - Trainer failed to record weight lifting

sessions for the entire defensive backfield.

• 4. Finding- three members of the offensive line

were late for Sunday’s pre-game meeting.

29

All the findings are “facts” observed by

• ur intrepid auditor. But they are facts

bereft of meaningful conclusions.

They are the skeletal bones of the team’s

systems without the throbbing beat of the heart.

30

 What if the final audit report looked more like this:

• 1. Team practices are poorly organized and players’ time is

not well utilized. This results in low team morale and inconsistent attendance.

• 2. Assistant coach seemed unprepared for the team’s film
• session. He was unfamiliar with the projection device

and also how to operate it. Film session concluded early and players expressed consternation about not getting enough opponent information to be properly prepared.

• 3. Team Trainer has not kept up with recording individual

weight training workouts. Players have picked up on this attitude and show little desire to improve.

31

 In the first report the emphasis was only on

• bservable facts.

 In the revised edition of the audit report the

emphasis switched to the systems on which the team depends for its success: i.e. its practices, pre-game film reviews and the weight training systems.

 It is obvious which report would be more useful

to the team management. And due to the clarity

• f the report it becomes obvious where $$ and

time should be invested to improve the probability of gridiron victories.

32

Challenges faced by the audit team include:

• 1. Limited time horizon for observation of the

processes we are auditing. We’re looking at a photograph not a full length video.

• 2. Time limits imposed on the audit. Our window of
• pportunity for the audit is typically relegated to a

days time frame, not weeks or years.

• 3. There is often restricted access to certain areas of

the organization being audited.

• 4. The information is collected via a sampling

process.

33

Therefore, there is the possibility of

incorrect inferential conclusions.

This is the risk that we take into account

and have to live with if our audit reports are ever going to be meaningful in a deeper way.

34

Think of it as the alpha v. beta conundrum

that we face with any statistical sampling process.

An alpha error means we do not accept a

part that we should have.

A beta error results in acceptance of a

part that we should have been rejected.

35

In the audit culture of today we often

choose to accept the Alpha error over the Beta.

When we limit our findings to

perfunctory matters there is very little probability we will be wrong.

However, there is also very little

probability we will be providing any information of useful value as a result of

• ur audit.

36

This is an area to which we need to devote

some interest, thought, and resources in

• rder to make the audit process more

meaningful.

You can’t ask a racehorse to pull a plow and

vice-versa.

It is incumbent upon management to

provide the resources whereby we better choose and train individuals who will become competent auditors.

37

Since auditing is no easy matter, we must

find individuals that exhibit at least some

• f the following characteristics:
• 1. Vision – be able to see peripherally as well as

in a straight line.

• 2. Logical - able to think critically and evaluate

systems in light of standards of excellence.

• 3. Versatile - must possess the ability to collate,

assimilate and conclude based on the sampling

• f observations collected at the audit.

38

14 Sep 2017

Our audits need to be system focused and answer the following three questions concerning the systems of interest:

• Adequacy - Do the various parts of the system

being examined have the potential to succeed?

• Implementation - are the systems being

implemented as was intended?

• Effectiveness - are the systems as implemented

achieving the results that were intended?

39

14 Sep 2017

Audits can and should be the go-to tool

Managers rely upon to judge the welfare of the system for which they are responsible.

If the audit is a third party audit then a

similar goal is desired for the customer or agency that has commissioned the audit.

When this philosophy underpins the audit

function it elevates it to the useful information gathering device it can be and restores the proper organizational significance that many of our audits of today lack.

40

14 Sep 2017

Questions ???? Comments ????

41