AECOM Independent Assessment of WRPS CAS Effectiveness Results - - PowerPoint PPT Presentation

Tank Operations Contract 1

Michael Peloquin

(509) 539-5357 Michael_G_Peloquin@rl.gov

WRPS Contractor Assurance Manager November 7, 2017

AECOM Independent Assessment of WRPS CAS Effectiveness Results

Tank Operations Contract 2

CAS EFFECTIVENESS ASSESSMENT

• May 15 through May 22, 2017
• The Review Team was led by AECOM N&E Technical

Services and included

– 3 management representatives from other contractors – 3 internal functional area representatives

• DOE-ORP was an invited observer for this review.
• First AECOM assessment using EFCOG Best Practice.

Tank Operations Contract 3

PURPOSE

• The review was practically focused on front line

performance where implementation of CAS processes

• ccurs

– CAS Effectiveness with respect to governance and transparency – Field Operations including

• Operational drill and emergency preparedness drill and exercise

program performance as implemented within field operations

• Conduct of Operations program performance as implemented within

field operations

• The assessment examined practices and behaviors of

performance at various levels of assurance

• The assessment also allowed for a review of safety culture

Tank Operations Contract 4

LAYERS OF SUPPORT

Workers/Technical Staff

Work/Project Delivery

Functional/Support Organizations Supervisors/Line Managers Audit/Assurance Organization Corporate Client Regulator External Internal

Performanc nce A Assur uranc nce La Layers of S Suppo upport

Tank Operations Contract 5

CAS EFFECTIVENESS CROSSWALK

Tank Operations Contract 6

LINES OF INQUIRY

• Objectives 1-7 aligned with the CAS effectiveness

attributes as defined in the best practice

• Objective 8-10 added by corporate assessment team
• Objective 9:

– A routine operations drill program and an EP drill and exercise program have been established and implemented. – Records for each program are adequate to demonstrate the effectiveness of completed drills and exercises as well as planning for future drills and exercises.

• Objective 10: The formality and discipline of operations

are adequate to conduct work safely.

Tank Operations Contract 7

LINES OF INQUIRY OBJECTIVE CE8: The CAS Effectiveness attributes and sub-elements are or will timely be effectively promulgated within WRPS. CRITERIA

• The CAS Effectiveness attributes and sub-elements are

documented in company programs, policies, an/or procedures and/or gap analyses have been performed, gaps have been identified and plans established to close gaps in a timely manner

• WRPS Senior Management is sufficiently aware,

understands, and supports implementation of the CAS Effectiveness attributes

Tank Operations Contract 8

RESULTS AND CONCLUSIONS

• The Assessment Team determined that all 10 Review

Objectives were met

– 9 noteworthy practices – 16 issues

• None of the issues were considered to have sufficient safety

significance to warrant immediate action and 2 of the 16 issues were for parent company consideration

• WRPS demonstrates a mature and continually-improving

Contractor Assurance System utilized by management to:

– (a) reinforce expectations and drive improvement in focus areas and – (b) focus risk based decisions and actions on outcomes and performance optimization – thereby minimizing risk.

Tank Operations Contract 9

RESULTS AND CONCLUSIONS

• Senior WRPS managers were found to be directly involved

with and fully attentive to current issues, their mitigation and their resolution. They were also found to fully support implementation of the CAS effectiveness attributes.

• With respect to the emergency drill and exercise program, a

healthy emergency preparedness program is being implemented and is focused heavily on continual process improvement and analysis of trends.

• There were no significant concerns identified regarding

sustainability of these two programs.

Tank Operations Contract 10

RESULTS AND CONCLUSIONS

• Formality and discipline of operations are generally

adequate to conduct work safely – but do not currently meet management expectations based on recent lock/tag and procedure compliance issues – Management has been proactive in taking immediate actions to ensure the negative trend is mitigated. – The maturity of the contractor assurance system provides the foundation and tools necessary for monitoring Conduct of Operations health to ensure work is performed safely. – In particular, field oversight through the Management Observation Program provides an opportunity for management engagement in the field to continue reinforcement of expectations.

Tank Operations Contract 11

RESULTS AND CONCLUSIONS

• Noteworthy CAS practices such as:

– The Management Observation Program; – The Trend Analysis and Collective Significance Review processes; – PER screening for immediate safety and non- compliance concerns; – Integration of risk management, internal audit, and contractor assurance; – Use of mockups and/or oral boards to establish and assure competency of involved personnel in response to some issues and events.

Tank Operations Contract 12

DISCUSSION FOR EFCOG CONSIDERATION

• Assessment activities are based on risk but do not

specifically include scheduling of high consequence activities.

• An assessment approach should consider consequence

irrespective of the probability of occurrence for a defined list

• f high consequence activities.
• The element of responsiveness consists of more than just

timeliness; other factors should also be considered

– Compensatory measures – Extent of Condition – Scope – Budget

Tank Operations Contract 13

DISCUSSION FOR EFCOG CONSIDERATION

• What is the definition of risk and risk informed within the

context of 226.1 B

• How do we leverage CAS processes to address risk

– Operating experience/Lessons learned – Assessment program – Cause analysis – Trend & corrective action program – Performance indicators

• Consider a best practice to address risk integration with

CAS

Tank Operations Contract 14

DISCUSSION FOR EFCOG CONSIDERATION

• What kind of information is provided to the LLC Board to

improve governance and their understanding of performance

– Quarterly Risk Reports – Quarterly Performance Trends and Recurring Occurrence Analysis Report – Quarterly Performance Evaluation and Measurement Plan Critical Self-Assessment – Performance Indicators – Independent/External assessment results

Tank Operations Contract 15

DISCUSSION FOR EFCOG CONSIDERATION

• DOE-O-226.1B CRD states

– “The contractor is responsible for flowing down the requirements of this CRD to subcontractors at any tier to the extent necessary to ensure the contractor’s compliance with the requirements. – Contractors must monitor and evaluate all work performed under their contracts, including the work of subcontractors, to ensure work performance meets the applicable requirements for environment, safety, and health, including quality assurance and integrated safety management; safeguards and security; cyber security; and emergency management.”

• EFCOG should consider a best practice to address this

language

Tank Operations Contract 16

LESSONS LEARNED

• Allow 2 weeks for the field assessment to provide better
• pportunity for follow-up reviews and management

interviews for validation of results prior to the exit interview.

• Assessment of the field activities in the early part of the

assessment was a strength, however, assessment team needs to circle back to the program for context and clarity.

• Documents should be provided prior to the review and then

contract period of performance start date should be approximately one week prior to the on-site review in order to enable the Team to review documents before arriving on- site

Tank Operations Contract 17

LESSONS LEARNED

• Consideration should be given for using surveys prior to the

assessment visit.

– Questions prompting binary answers with room for comment could be used to find common themes to interview and document review lines of inquiry. – Potential to piggyback on existing organizational surveys (e.g. Safety Culture), analyzing the results for relevance or adding some applicable questions whenever possible.

• Consideration should be given to combining Organizational

Learning and Management Leadership given the extent of their overlap

• Consideration should be given to addressing Work by

Others corporately rather than project by project

Tank Operations Contract 18

Thank You