Organisational Failures in Accident Reports Michle Jeffcott & - - PDF document

1

Organisational Failures in Accident Reports

Michéle Jeffcott & Chris Johnson

Department of Computing Science, University of Glasgow, Glasgow, G12 8QQ. Tel: +44 141 330 0917, Fax: +44 141 330 4913 Email: {shellyj, johnson}@dcs.gla.ac.uk

Accident investigation aims to identify the root causes that lead to major

• incidents. These factors include problems in the design and operation of human

computer interfaces. They also include the organisational factors that are increasingly considered as important in accident causation and in our understanding of human computer interaction in complex working environments. However, the organisational “causes” of major accidents are still poorly investigated in comparison to technical failures. This results in disproportionate feedback, and a lack of improvement in organisational functioning. It also hides many of the contextual factors that jeopardise successful human computer

• interaction. This paper, therefore, shows how a design rationale notation that was

developed within HCI can also be applied to represent and reason about these wider systemic causes of failure. This semi-formal notation is useful because it can be difficult for readers to trace the complex arguments about human ‘error’ and system ‘failure’ that are scattered throughout the body of a lengthy text- based documents. In particular, we show how Conclusion-Analysis-Evidence (CAE) diagrams provide a graphical overview of evidence and lines of argument about human and organisational failure. A Marine Incident Investigation Unit's (MIIU) report into a fire on the Aurora Australis is used as a case study to illustrate the argument in this paper. Keywords: Organisational failure, Design Rationale, Human ‘Error’. 1. Introduction According to Reason, until the 1930’s investigation of British railroad accidents mainly focused on technical failure, which was typical of many industries. This lead, in the

2 following decades, to attempts to make technical systems error proof by developing elaborate in-built defences. But it proved increasingly difficult to reduce accidents by technical safeguards, and attention turned to the role of the human operator as a causal factor in accidents. As well as human fallibility, engineers' lack of consideration of the needs, capabilities and restrictions of humans when designing technical systems was also blamed. But the onus lay on the operator, who by the early 1980’s, was reported to be responsible for 80 to 100% of accident causes (Wagenaar, 1983). It seemed that the basic operators and maintenance personnel were becoming the scapegoats. More recent accidents, such as Chernobyl and King’s Cross, have shifted the emphasis

• nce again to those who make the decisions at a management and organisational level.

This mirrors the increasing interest in organisational and contextual factors within human computer interaction (Beyer and Holtzblatt, 1998). Reason (1998) concludes that the importance of organisational and management factors as a precursor to human “error” is now widely accepted. But it remains to be seen if organisational failure is actually properly investigated and subsequently represented in accident reports. A series of organisational failures make up an organisational accident, which Reason (1998) defines as: "An organisational accident has multiple causes involving people operating at different levels of their respective companies. They are events in which no

• ne person's failure was a sufficient cause and trace back into many parts of

the organisation, from the operator to the manufacturer, and - by implication - the regulator." Reason adds that understanding and limiting the occurrence of organisational accidents are the major challenges to be faced in the new millennium. This paper addresses and suggests improvements in the study of organisational failure in accident reports and so aids the first of Reason's challenges. The relevance of this work for HCI can be explained in two ways. Firstly, HCI is playing an increasingly important role in many accidents (Norman, 1990). They therefore provide important information for future interface development. Secondly, it is critical that we identify and report on these

• rganisational and managerial aspects that lead to failure in HCI if we are to

understand the true context in which accidents occur. 1.1 Marine Incident Investigation Unit (MIIU) Case Study A report prepared by the Marine Incident Investigation Unit (MIIU) is used to illustrate the argument in this paper. It is a lengthy 60-page document, with an additional 15 page 'Investigation-In-Confidence' following the main body of the report. The supplementary investigation records a seconded investigator's special study on the fuel system, whose failure caused the diesel oil leak. The general report presents the crucial time scale of the events leading to and during the incident, the action's of the crew and equipment used to fight the fire, and analysis of the effectiveness of the fire fighting

3 effort and the causes of the fire itself. The MIIU summarises the incident development as follows: "At about 0230 on 22 July 1998 a fire broke out in the engine room of the Antarctic research and supply vessel Aurora Australis. The ship was about 1300 miles south of Tasmania with 54 special purpose personnel (or expeditioners), 24 crew and an ice pilot on board. About 25 minutes before the outbreak of the fire, the duty engineer had been woken by an alarm visited the machinery control room and inspected the engine

• room. He cancelled the alarm and returned to his cabin at 0213. At that time,

everything in the engine room appeared to be normal. At 0225 the duty engineer was roused by a second alarm and, returning to the engine room, he discovered a fire at the forward end of the port main engine, around the turbocharger. The engine was stopped and the fire alarms sounded. The fire at the turbochargers was attacked by engineers using portable extinguishers and apparently extinguished. A few moments later, however, at about 0236, a fireball erupted and the engineers were forced to evacuate the engine room." (MIIU, 1999). This accident report provides an appropriate case study because it typifies the complex combinations of human “error” and systems “failure” that characterise many major

• accidents. It also reflects the problems in team working and in accessing appropriate

information sources that exacerbate the initial causes of an incident. It is also an appropriate case study because the format of this report typifies the presentation problems that affect the “usability” of these technical documents. These problems can prevent readers from gaining an accurate understanding of the implications that such failure has for the future design and operation of interactive systems. The Maritime Incident Investigation Unit (MIIU) presents their reports as a long text document, occasionally broken up with photographs and diagrams. It is presented as a single document, but can be roughly divided into two main sections. The first describing the Incident Development, detailing the time of events, their location, those involved and the actions they took. It is very much a narrative account of the event in

• precision. The second section consists of the Comment and Analysis and ends with a

list of conclusions, or root causes, about the incident. Although the conclusions are set out explicitly at the end of the report, the reader has no way of tracing the steps the investigator(s) took to reach these conclusions. The conclusions are based on the analysis and evidence, which is scattered all throughout the body of the report. CAE diagrams allow the reader to follow, which points of analysis are, being considered and where the supporting/negating evidence comes from, for each relevant conclusion. A potential problem with this is that the reader can not interpret and use the evidence, free of the investigator's argument, which means that any unwanted bias in the report can not be avoided. The more explicitly the argument is presented, the more the reader is left to rely simply on the analysis of the investigator, and is reduced to an essentially passive role. However, relying on an

4 implicit argument approach can create problems when people are forced to recall important evidence that is presented many pages before the concluding chapters. Problems can also arise if readers are never expressly told which items of evidence support a particular conclusion (Johnson, 1999). 2 Conclusion-Analysis-Evidence (CAE) Diagrams Conclusion-Analysis-Evidence (CAE) diagrams are based on concepts that were

• riginally developed within human computer interaction and design rationale

(Johnson, 1999, Shum, 1991). These diagrams distinguish between the evidence and analysis that are used to support specific conclusions in accident reports. Many root- cause analysis techniques, such as those developed by the US Department of Energy (1992), advocate this separation. CAE diagrams, therefore, provide a graphical

• verview of the argument that is presented in accident reports. The following pages

use them support an analysis of how well organisational failures were represented and investigated in a complex accident report, such as MIIU Report 135 on the Aurora Australis. As mentioned, CAE diagrams provide an application of techniques that were originally developed to support more “conventional” forms of human computer interaction. Design Rationale (DR) is concerned with developing effective methods and computer- supported representations for capturing, maintaining and re-using records of why designers made particular decisions (Shum, 1991). MacLean et al. (1991) presented a series of objectives for an approach to representing DR, which they call Design Space Analysis (DSA), characterised by the semi-formal, argumentation-based notation called Questions, Options, Criteria (QOC). QOC, developed by Rank Xerox Cambridge EuroPARC, rests largely on the 3 node types: Questions, Options and Criterion and 2 link types to express the Option-Criterion relationship: supports (solid line) and objects to (dotted line). It begins with a key development question, linked to the Options that support the Question and then all for and against Criteria used to make the decision. In

• ur novel application, they are used to record the reasons for failure rather than

potentially successful designs. Figure 1 presents an example of this QOC notation. It illustrates some of the design options that might improve situation awareness amongst vessels on the Australian Barrier reef (Johnson, 1999a). The first option is to force all ships to notify their position to a computer-based monitoring system. This is supported by the criteria that it would provide an external means of ensuring that crews comply with regulations. The Reefrep system could monitor and log the reporting behaviour of each vessel. The development of such a system is not supported by the affect that it would have upon crew workload. The second design option is to use crew training procedures as a means of ensuring adequate levels of situation awareness. This is not supported by the possibility of performing external checks.

5

Q: How t o imp rove sit uat ion awareness amongs t crews on t he Ree f? O: f orce all ships to not if y t heir posit ion to t he Reef rep report ing syst em. O: Rely upon training procedures t o ensure that crew complet e review of cu rrent sit ua tions when hand-over o f wat ch t akes place. Cr: ext e rnal checks can be conduct ed t o ensure compliance. Cr: low levels of crew workload. +

• Figure 1: QOC diagram shows design options for situation awareness

CAE diagrams were originally developed from the QOC notation. The incident investigators report conclusions (representing the incident's root causes) take the place

• f the key development question. QOC is used in 'externalising ideas' (Shum, 1991),

and similarly, the advantage that CAE brings is to make the investigator's analysis and general argument explicit. This is beneficial in view of the current accident report

• format. The following procedure is used during the generation of CAE diagrams

(Johnson, 1999): 1. List all of the conclusions that are identified in the report. This is usually a trivial exercise because most accident reports list their conclusions in the final chapter. 2. List each line of analysis that supports or weakens the conclusions identified in stage 1. This is, typically, a non-trivial exercise. Few accident reports explicitly record the lines of analysis that support their findings. The reader is expected to piece together the implicit inferences that support specific conclusions. 3. List the evidence that either weakens or strengthens each line of

• analysis. This is relatively straightforward because stage 2 helps to identify

the evidence that is needed to support a particular line of argument. The warrant or source for the evidence (e.g., expert witness, simulation, and regulation) is also noted. 4. Construct a graph based on the products of the first three stages. A separate CAE diagram is produced for each of the conclusions identified in stage 1. These are drawn as the roots for each of the graphs. The arguments for

6 and against a conclusion are connected to the root. Solid lines connect arguments that support a conclusion. Arguments that weaken a conclusion are connected by dotted lines. Finally, each item of evidence is linked to the relevant arguments. As before, evidence in support of an argument is shown by a solid line whilst negative or contradictory evidence is shown by a dotted line. CAE diagrams present the information in an accident report in a more concise format than a typical lengthy text document. This can be useful to both the original investigator/s and to subsequent readers. For the investigator, CAE diagrams can help to clarify lines of analysis using available evidence. It may force them to better justify and develop the conclusions that they arrive at. In this, it mirrors QOC, which Maclean (1991) believed enabled better means of representing and reformulating views of the design space around an artefact, encouraging the designer to widely explore Options that reflect the key dimensions in the design space. One of the key hypotheses which prompted research into supporting argumentation in accident reports was that by making the structure of arguments explicit, they can be more rigorously constructed and communicated (Brown, 1986; Smolensky et al., 1987). In contrast to previous work on design rationale, our concern was to better understand the causes of failure in human-computer interaction rather than document the reasons for a successful design. CAE diagrams were, therefore, constructed for the main conclusions in the MIIU report. Before focussing on the organisational precursors of failures in human computer interaction and group work, it is important to demonstrate that CAE diagrams can also be used to analyse the more “technical” causes of systems failures that currently dominate most accident reports. For example, the CAE diagram in Figure 2a focuses

• n the finding that the fire was due to a failure in the fuel hose and that:

"Failure of the hose was due to its age and wear and tear." This conclusion is supported by the results of torsion tests and by observations about their lack of sheathing. However, this argument does not directly explain why the potential failure was not detected during routine maintenance cycles. Figure 2a shows how CAE diagrams can be used to analyse these concerns that can arise from an initial analysis of the arguments in an accident report.

7

CONCLUSION 1:

Failure of flexible hose was due to its age and wear and tear.

A1: The two (fuel) hoses, under the bottom plates near the fuel filter pack of the port engine, were blackened and had been affected by the fire, but nevertheless appeared old. (PAGE 26) The basic cause of the failure was that the hose had been in service well beyond the end of its useful

• life. (PAGE 44)

E1: The evidence that the hose that failed on 22 July 1998 was fitted, at the latest, on 14 Dec. 1992, and possibly as early as

• Jan. 1991. The condition of the

hose (as reported in the LR) is consistent with it having been in service for a number of years. From this evidence, it appears that the hoses were 6yrs, 8 months old. (PAGE 34) E2: Many of the reinforcing wires were considerably corroded, and there was substantial wear of the wires where they had chafed against one another. Some wires appeared to have been reduced in cross section by more than 50% by mechanical

• wear. (PAGE 8 LR)

A2: The primary cause of failure of the hose was due to wear and corrosion between the overlapping wires, followed by fatigue an, finally, to ductile

• verload when the

remaining cross-section of each wire could no longer support the load on the

• wire. The cause of the

fatigue was not essentially any excessive loading, rather it was consequent upon the loss of cross section of the wires. (PAGE 43/44) E3: The consultant could draw no firm conclusions regarding the cause of the corrosion. It may have been long term caused by the deterioration in the rubber protection allowing the atmosphere to attack the steel. Equally, the corrosion may have been rapid, when the plastic components of the hose broke down upon exposure to the heat

• f the fire. The depth of the

corrosion, however, suggested that it had occurred over a period of time, indicating the former cause as the more likely. (PAGE 43) A3: The investigators suspected that residual torsion in the hose following some maintenance action might have had a bearing on the failure. (PAGE 8 LR) [However, there is little doubt that this problem was not the prime cause

• f failure (PAGE 3 LR)]

E5: The predominance of failed wires were those counted in a clockwise direction, which accounted for 95 fractured wires. In the anti-clockwise direction there were 35 failed wires. This would suggest, though not

• verwhelmingly, that residual

torsion in the hose may have been a factor in the failure. (PAGE 43)

Figure 2a. Flexible fuel hose failure due to 'Wear and Tear.' The condition of the flexible fuel hoses was emphasised throughout the report, culminating in a 15-page investigation by a seconded expert about the physical failure

• f the hoses. Drawing the CAE diagram in Figure 2a, highlighted the many different

ways in which evidence about the condition of the hoses was repeatedly emphasised throughout the report. Figure 2b, in contrast, shows how the same diagrammatic technique can be used to collate evidence in support of arguments that were not explicitly considered in the conclusions of the final report. In particular, it presents

8 arguments that were made against fitting the hoses in the first place rather than focussing on the poor condition of the hoses themselves.

CONCLUSION 1: Flexible hoses replaced original fixed pipework, and later, stainless steel convoluted pipes, as they were judged to be more suitable in situations of vibration and twist. (Rewording of PAGE 32/Lab Report LR) A1: The results of the torsion test, carried out as part of the lab investigation, clearly support warnings given by the manufacturers and suppliers against mounting this type of hose in situations where it is likely to get twisted. (It appears that less than 3˚/m should be regarded as a significant amount of twist) (PAGE 13 LR) E1: There was no indication

• f excessive bending ( of the

flexible hose fuel line in question) but when the hose was removed, the diagonal split closed up and was almost invisible. This was an indication of some residual torque in the hose. (PAGE 5 LR) E2: Although some failures still occurred with these flexible hoses (no specific examples cited in report), they were adopted as part of the fuel pipework. (PAGE 32) A2: Flexible hoses (Parker Hannifin 206-16) showed 'improved performance'. Problems had been experienced with the pipework of the fuel systems for the main engines - Vibration caused cracking of the rigid pipework. (Rewording PAGE 32) E3: Isolation of noise and vibration from the hull is

• important. Parker Hannifin

206-16 hoses reduced vibration to the hull, as they allowed the generator sets to be flexibly mounted. (PAGE 32) A3: The flexible hoses apparently provided an improved performance, but they were not sheathed and, lacking this fire protection measure, were not ideal. (PAGE 33) E5: The rubber for the inner lining has to be chosen for high resistance to chemical attack by hydrocarbons. Rubbers suitable for this application may be less resistant to simple weathering or fire and flame. (PAGE 10 LR) E4: A list of Lloyd's Register Approved Products for hose assemblies and flexible hoses, dated Feb. 1998 does not list the Parker Hannifin 206 hose as a Lloyd's approved type. (PAGE 40)

Figure 2b. Suitability of Flexible Hoses to their application on the Aurora Australis.

9

• 1. The fire was caused by diesel fuel from a split in the flexible

fuel hose in the spill line from the main engine coming into contact with a component of the port engine turbo-chargers, the temperature of which was inn excess of the auto-ignition temperature of the fuel.

• 2. Failure of the hose was due to its age and "wear and tear".
• 3. Although recommendations relating to fixed pipework on

the engines, contained in Wärtsilä Technical Bulletin issued 3 years earlier, were implemented by the company, the recommendations in the same bulletin relating to the fitting, care and maintenance of sheathed hoses in the low pressure fuel system, were not followed.

• 4. When fitting the flexible fuel hoses at some time between

1991 and 1992, the ship's drawings were not altered to show the modification to the system.

• 6. No approval was sought from the Australian Maritime

Safety Authority for the fitting of flexible hoses.

• 7. Knowledge that the flexible hoses had been fitted under the

floor plates was lost with the turn-over of engineers.

• 8. The fact that other flexible hoses were fitted to the engines

was well evident, but this did not alert either class or AMSA surveyors to the fact that the modifications were not approved.

• 9. In general the response to the fire by the ship's crew and

the expeditioners on board was measured, effective, demonstrated intitiative and reflects great credit to all on

• board. Entry into any area adjacent to fire, however, alone

and without breathing apparatus or backup, is extremely hazardous and could compromise the entire firefighting effort.

• 10. The poor design of the electrical operating system for the

Halon 1301 fixed smothering system led to its unreliable

• peration and to the partial discharge, only, of the halon.
• 12. Those involved in restoring propulsion to the ship

showed considerable ingenuity, skill and initiative.

• 11. The maintenance of the halon system involved at least

three contractors and ship's staff, leading to a lack of continuity in maintenance and probably to the fitting of inappropriate fuses in the 24 volt supplies to the main control units. Hoses in fuel systems should be examined and pressure tested on a regular basis. Evans recommends six months as a suitable interval between tests. In fuel systems for marine diesel engines, spill pulses from the injection pumps are likely to be a significant factor in the pressure loading of hoses. This is not widely -understood phenomenon. Engine manufacturers should be encouraged to measure these and state their magnitude when supplying

• engines. Where these dynamic pressures

have not been determined at the time of construction, it would be prudent at the entry to on-engine pipework. Hoses should not be used in fuel systems unless relative motion of components makes them necessary. If the relative motion is due to vibration, other solutions should be sought in preference to the use of flexible hose. Where hoses are used to connect an engine's fuel system to ship's pipework, safety could be improved by the use of externally sleeved hoses. Any leak from the hose is contained and should be drained to a catch tank fitted with an alarm to indicate the presence of a significant leak. Hoses should be replaced at intervals recommended by the hose manufacturer, determined with knowledge of the working conditions and environment. The locations of all flexible hoses carrying any hazardous fluid should recorded, and mandatory scheduled maintenance applied as above. Hoses must be installed without excessive bending, and without any residual torsion, i. e. with careful attention to avoid any twist in the installed hose. It is important that when maintenance is carried out on pipework with flexible hoses attached to it, the hoses are loosened or removed. Quite small amounts of twist can significantly weaken hoses of this type.

• 5. Consultations between the company and Lloyd's Register

and the company and Wärtsilä, on the use of flexible hoses were "ad hoc" and no record of consultation or approval concerning their fitting was made by any party.

Figure 3. Diagram to show the correspondence between report conclusions and feedback given. The conclusions are on the left hand side (Double line = Technical factor, Thick line = Organisational factor) and the recommendations on the right.

10 The previous CAE diagrams show how an approach that was originally deve within the field of HCI can also be applied to represent and reason about that are presented in accident reports. This extension of design rationa analyse the causes of technical or systems failures. The intention was the wider applicability of this approach. In contrast, the following pa this approach can also be used to analyse arguments about the interactio failures in groupwork and organisational weaknesses that led to the fire.

CONCLUSION 1: Consultations between the Company and Lloyd's Register, and the Company and Wärtsilä,

• n the use of flexible

hoses were "ad hoc" and no record of consultation

• r approval concerning

their fitting was made by any party. (The Company refers to the P+O Company, and so describes the representatives of the Aurora Australis). E2: A substantial number of contemporary notes made by the Chief Engineers on Aurora Australis and the P+O Technical Superintendent support the contention that the problem of engine vibration was raised repeatedly with W ärtsil ä over the

• years. : In particular, P+O

submitted one note made by a chief engineer in July or August of 1991 in the engineer's work book: "LP Fuel Rail - This does not now affect us as we have installed the Enzed flexible type hoses which [Wärtsilä Service Engineer] recommends. (PAGE 35) A1: The Inspector is satisfied that some consultation took place on the fitting of the hoses, however the evidence is that this was at a local, shipboard level and was 'ad hoc' in nature. (PAGE 36) Documentary evidence that the Company consulted Lloyd's Register and Wärtsilä was sought from the 3 parties. (PAGE 35) E5: Both Lloyd's Register and Wärtsilä stated categorically, however that they could find no record of consultation or approval, either written or verbal, on the issue of the fitting of flexible hoses. (PAGE 35) A2: The Inspector found it hard to accept that no consultation took place between P+O and Lloyd's Register and P+O and Wärtsilä, on the issue of

• vercoming the vibration

problem with the flexible

• hoses. Given the constant

problems with vibration and the unsuccessful fitting of "omega pipes" it is logical that some remedy would have been discussed. (PAGE 36) E4: The inspector notes that neither of these two documents (presented in E2 and E3 above) are specific as to the location of the hoses or their construction. (PAGE 35) E6: Since the first voyage of the Aurora Australis in 1990, problems had been experienced with the pipework of the fuel systems for the main engines. Vibration caused cracking of the rigid pipework where it connected to the engines. At an early stage of the ship's like Wärtsilä Australia provided "omega" pipes to connect to the engines in an attempt to overcome the failures in the fuel oil

• pipework. This however did

not solve the problem. Stainless steel convoluted pipes with a stainless steel braided sheath, to form flexible sections where the fixed pipework connected to the engines, were also tried. These too were unsuccessful. (PAGE 32) E3: P+O also submitted a facsimile of 26 March 1993, sent to Wärtsilä, concerning vibration of the engines. "The vibrations that are experienced at the front of the engine has resulted in the fuel lines to the main engines being replaced with flexible lines and in service it has been found that close inspections are required to be carried out at regular intervals as failures through vibration have been experienced with the flexible lines." (PAGE 35) E1: The Company, in submission, stated that they consulted representatives of both class and the engine builders between 1990 and 1993 in relation to the vibration at the forward end

• f the engine and the

resultant cracking of rigid fuel pipes (PAGE 35).

Figure 4. Organisational Issue 1: Communication Deficiencies.

11

• 3. Improving Feedback

Figure 3 reveals the bias towards technical causes of our case study. This diagram shows the correspondence between the conclusions that are listed in the MIIU report,

• n the left of the diagram, and the detailed feedback that was provided to avoid future

failures, on the right of the diagram. The conclusions listed in this diagram were directly taken from the CAE analysis of the full MIIU findings and so can be linked back to their supporting arguments and evidence. It is, therefore, also possible to trace those items of evidence and lines of argument that were not acted upon in the report's

• recommendations. This is shown in Figure 4 which takes Conclusion 5 from Figure 3

and presents the analysis that is constructed in the MIIU report and subsequently neglected in the recommendations. The lack of consultation between the company and regulators can arguably, therefore, be seen as a continuing source of future problems. A latent cause of future failures. Interaction problems are represented in the accident report. For instance, the crew faced considerable difficulties in operating the control system that was used to deploy the Halon 1301 extinguishers. There were also deeper managerial precursors that placed the crew in a situation where they had to deploy this system in the first place. Neither the present crew nor the maritime regulators knew about the installation of the

• hoses. These problems represent over half conclusions (7 out of total of 12). But

despite this recognition of their contribution as root causes, they are not adequately represented in the consequent recommendations. From reading these conclusions, it can be seen that they represent serious

• rganisational deficiencies. These deficiencies placed the crew in a situation where

they were forced to operate the Halon extinguishers using a control system that was poorly understood and difficult to use. CAE diagrams help to visualise the extent to which organisational deficiencies contributed greatly to the fire on the Aurora

• Australis. The fire represents an organisational accident. The report provides evidence

for these organisational failures but the general investigation concentrates on the physical condition of the hoses in the fuel system. Previous sections have argued that the lack of consultation between the operators and the regulators was not explicitly considered in the recommendations that were published with the accident report. Figures 5 and 6 return to this issue and show a further CAE analysis of the organisational factors that ultimately forced the users to deploy the Halon system. Figure 5a reveals the "ad hoc" consultation between the relevant bodies and that contemporary notes by the Engineer's were the only documents regarding the problem of vibration with previous pipes and the fitting of flexible hoses.

12

CONCLUSION 1:

P+O did not follow proper procedure and regulations when fitting the flexible fuel hoses.

A1: P+O did not go through the proper channels, in seeking approval for the flexible hoses initial fitting between January 1991 and December 1992. There are a number of class and flag State requirements relating to the fitting of flexible fuel hoses. (PAGE 35) E1: Lloyd's Rules and Regulations for the Classification of Ships also provides for the use of such hoses but require that the Society's approval must be

• btained before they are fitted.

(PAGE 35) "For the purpose of approval...details of the materials and construction of the hoses, and the method of attaching the end fittings, are to be submitted for consideration." (Annex: PAGE 60) A2: In addition to the fact that P+O did not get approval for the hoses, they did not follow important Lloyd's Regulations and recommendations when deciding which hoses to fit and during the hoses life in service. There are a number of requirements relating to oil fuel pipework and flexible hoses testing and maintenance under the provisions of Lloyd's Classification Society Rules. (Rewording PAGE 39) E5: Lloyd's Register rules provides that any flexible hoses on the fuel system should be tested and approved by the Society. The test requires that t he hoses must withstand pressure at least five times the maximum working pressure in service. Research has shown that pressure pulses in the supply and spill pipework... can reach levels up to ten times the system working pressure. These pressure pulses are the direct result of spilling fuel at 800 to 1500 bar into the low pressure pipework...the notation of "low pressure" having lulled shipyards...and operators into a false sense of security. In June 1994, the IMO issued a circular 647 - "Guidelines to Minimise Leakages from Flammable Liquid Systems." The circular reinforced the SOLAS requirements, noting that flexible hoses may need to be replaced several times in the life of the ship and that manufacturers' recommendations should be followed. (PAGE 40) E6: Parker Hannifin (PH) 206 hoses were fitted on the Aurora, which was not a Lloyd's approved type on a list of all Lloyd's Register Type Approved Products for hose assemblies and flexible hoses, dated 11 Feb. 1998. However, the list does include the PH type 221 FR, as an approved type. (NB: 221 is listed as having a

• max. working pressure of 500psi,

and is temp. rated to 100˚C when used with "gasoline fuel, diesel

• il..." 206, however, has max.

working pressure of 800psi (for 7/8 in internal diameter) and

• temp. rating of 150˚C -

Specifications which exceed those of the approved hose). (PAGE 40/41) E3: SOLAS 1974 permits the fitting of flexible fuel hoses. However it provides that they should be of short a length as

• possible. (PAGE 34)

"Oil pipes and their valves and fittings shall be of steel or

• ther approved material,

except that restricted use of flexible pipes may be permissible in positions where the Administration is satisfied that they are

• necessary. Such flexible pipes

and end attachments shall be

• f approved fire-resisting

materials of adequate strength and shall be constructed to the satisfaction

• f the Administration.

(Annex: PAGE 60) E4: Marine Orders 31 requires that, after any survey, no modification should be made to the ship's machinery or equipment without obtaining the approval of the Chief Marine

• Surveyor. (PAGE 34/35)

E2: In addition, Lloyd's Register stated in submission that "...such hoses are to be installed under the survey and in accordance with the hose and engine manufacturer's recommendation." (PAGE 35)

Figure 5a. Organisational Issue 2: CAE diagram showing Specific Deficiencies of the P+O company in regard to fitting of hoses.

13

CONCLUSION 1:

Lloyd's Register and Wärtsilä Australia contributed to the length of time that the flexible fuel hoses were illegally in service.

E1: The highly visible position

• f the flexible hoses on the fuel

system, at the connections to each main engine, was obvious. They were readily apparent to any surveyor or contractor. Their construction and presence would have been emphasised by the rope lashings used to restrict their movement and chafing. (PAGE 36) A1: The hoses had been in position for at least six years and their fitting had not been queried, despite inspections/surveys of the engine room, carried out by class or the engine builders. (Rewording of PAGE 36) E3: The Inspector asserts: Whether the representatives of class or the engine builders made a mental connection between the hoses and a requirement to gain approval for their use, is a different issue. It is also unclear whether the classification society

• r Wärtsilä would have known

about the two flexible hoses on the port main engine fuel filters, beneath the floor plates. (PAGE 36) A2: On 1 August 1995, Wärtsilä Diesel (as they were then) published a Technical Bulletin entitled "Safety aspects on the maintenance of fuel supply system of VASA 32." It included measures to prevent flexible hose leakage's, being it modification of design, welding method or maintenance routines. This Bulletin did not specifically address its recommendations for the application of the hoses on ships. (Rewording PAGE 37/38) E4: At section 6 of the bulletin, under "Special recommendations concerning power plants" it states: "For ships there are certain safety rules concerning fuel systems and fire protection stipulated by classification societies and authorities, and therefore safety matters for ships will not be dealt with here." "For power plants, however, the safety and fire protection rules

• vary. In addition to the rules

stipulated by authorities we have found reasons to give following the recommendations: (Examples:)

• F

l exible hoses should be inspected visually after every 2000 hours.

• R eplaced after max. two

years service. (PAGE 38) E2: These flexible hoses apparently triggered no concerns at any of the periodic surveys conducted after 1991. Aurora Australis had received ISM accreditation

• n 23 September 1996 and

had its Safety Management Certificate issued. A document of compliance was issued to P+O in Hobart at about the same time. The P+O Maritime Services office in Melbourne was accredited to ISO9002 in 1993. (PAGE 36) E5: P+O submitted that the reference to "power plants" at Section 6 excluded the application of the recommendations to ships. Therefore, no action was taken by P+O to implement the recommendations regarding flexible hoses. (PAGE 38/39) E6: The Inspector agreed that the wording of Section 6 in the Bulletin was ambiguous in its applicability to ships. Clarification was sought from Wärtsilä. They replied: "...the recommendations given under Section 6 could apply to the use of flexible hoses

• nboard provided that the

classification societies and the local laws and regulations allow the use of flexible hoses

• nboard." So, Wärtsilä are

reinforcing that their recommendations were not specified for the hoses on the

• Aurora. (PAGE 39)

E7: The Inspector personally states that: "As the flexible hoses had already been fitted without reference to class or statutory requirements, application of the bulletin's recommendations for their safe use would seem to encompass common sense..." (PAGE 39)

Figure 5b. Organisational Issue 3: CAE diagram showing Specific Deficiencies of

• ther bodies in regard to fitting of hoses.

Specifically, Figures 5a and 5b reveal inadequacies on the part of both the P+O Company, for not following proper procedures and regulations when fitting the flexible fuel hoses and Lloyd's Register and Wärtsilä for contributing to the length of time that the flexible fuel hoses were illegally in service. P+O installed the flexible hoses to the fuel system despite:

14 "Marine Orders 31 requires that, after any survey, no modification should be made to the ship's machinery or equipment without obtaining the approval of the Chief Marine Surveyor." (Fig. 5a E4) On 23rd Sep.1996, Lloyd's Register awarded ISM accreditation to the Aurora: "These flexible hoses apparently triggered no concerns at any of the periodic surveys conducted after 1991." (Fig. 5b E2)

CONCLUSION 1: When fitting the flexible fuel hoses at some time between 1991 and 1992, the ship's drawings were not altered to show the modification to the system. And knowledge of that the flexible hoses had been fitted under the floor plates was lost with the turn-over of engineers. These two factors , added to the

• bscure location of the hoses,

contributed to the hoses being in service, without proper maintenance and checks, long after their useful life. E2: One reason is that, unlike the hoses connected directly to the engine, the flexible hoses between the port engine filters and the rigid pipework were hidden from view by the floor plates. (PAGE 34) A1: It was forgotten that the hoses had been fitted and the short lengths of flexible hose were simply

• verlooked.

(Rewording Page 44) E5: There was no evidence that the hoses had been inspected or tested during their 7 years of service. Had such tests been carried

• ut, the hoses would probably

have failed such tests long before the accident of 22 July 1998. (Rewording of PAGE 10 LR) A2: Continuity of knowledge between the engine staff of the existence of the flexible hoses beneath the floor plates was lost in time, due to the large turn-over in staff in the 8 year period that the ship had sailed. The majority of current Engineers had joined the ship many years after the hoses were fitted. (Rewording of PAGE 44) E7: Of the four engineers on board for voyage No.1 of 1998/1999, the Chief Engineer had joined as 1st Engineer on 26 Jan. 1997 and was promoted to Chief Engineer in about Aug. 1997. The 1st Engineer had joined in early 1998. The 2nd Engineer had been with the ship since Feb. 1995 and the 3rd Engineer since Nov. 1995. It is quite probable that none of them realised that, beneath the floor plates, flexible fuel hoses were connected between the rigid pipework and the fuel

• filters. (PAGE 44)

E3: Nothing, other than an inspection below the floor plates adjacent to the fuel filters would have revealed the condition of the hoses fitted in the supply and return lines, both of which were less than

• ne metre in length.

(PAGE 34) E4: The main safeguards against flexible hose failure is frequent inspection and regular

• replacement. There was no

procedure in place for the regular inspection or replacement of these two short lengths of hose, as they were 'forgotten.' (PAGE 34) E6: Since Aurora Australis entered service in 1990, 26 engineer officers have sailed on the ship. There have been eight different Chief Engineers. (PAGE 44) E1: It seems that in 1998 the ship and relevant shore staff either did not know of the flexible pipes below the plates,

• r had forgotten that they were
• fitted. (PAGE 37)

A3: Adding to the problem of knowledge lost in staff-turnover, was the fact that the ship's drawings had not been amended to document the fitting of the flexible hoses, showing their exact location and positioning. (Rewording) E9: The evidence is that electrical drawings were amended in 1991 and 1992 to show configuration changes in electrical system. However no fuel arrangement drawings were amended at this time, or since 1996, to show the fitting and location of the flexible hoses. (PAGE 37) E8: With the 1996 ISM accreditation, came the requirement to for document and data control, to ensure that up-to- date instructions and references are made available to employees. Specifically, it ISM code states: "The Chief Engineer is reponsible for...updating electrical, mechanical, pipework, and structural drawings held on board and ashore to reflect the 'as fitted' condition of the vessel." However, the ISM accreditation post-dates the fitting of the flexible hoses and the plans should have been amended retrospectively. (PAGE 36/37)

Figure 6. Organisational Issue 4: CAE diagram involving Crew Awareness of flexible fuel hoses location.

15 Figure 6 reveals that the extent of the organisational failures that contributed to the

• accident. The engineers did not know that the hoses had been fitted to the fuel system.

This in turn helps to explain why those engineers had not performed a proper maintenance cycle on those hoses: "It is quite probable that none of them realised that, beneath the floor plates, flexible fuel hoses were connected between the rigid pipework and the fuel filters." (E7, pg.44) "There was no evidence that the hoses had been inspected or tested during their seven years of service. Had such tests been carried out, the hoses would probably have failed such tests long before the accident of 22 July 1998." (E5, pg.10 Lab report) The key point here is that it is important not to focus on the interaction problems that arose during this accident without considering the organisational and managerial failures that forced the crew to rely on emergency systems, such as the Halon

• extinguishers. Our analysis of this and similar accidents reveals that the MIIU case

study is typical of the ways in which these "contextual" managerial weaknesses are central to any understanding of why human computer interaction problems cause or exacerbate major accidents (Johnson, 1999a).

• 4. Conclusions

Accident investigation aims to identify the root causes that lead to major incidents. These factors include problems in the design and operation of human computer

• interfaces. They also include the organisational factors that are increasingly

considered as important in accident causation. They are also critical to our understanding of human computer interaction in complex working environments. However, the organisational “causes” of major accidents are still poorly investigated in comparison to technical failures. This results in disproportionate feedback, and a lack

• f improvement in organisational functioning. It also hides many of the contextual

factors that jeopardise successful human computer interaction. This paper, therefore, has shown how a design rationale notation that was developed within HCI can also be applied to represent and reason about these wider systemic causes of failure. This semi-formal notation is useful because it can be difficult for readers to trace the complex arguments about human ‘error’ and system ‘failure’ that are scattered throughout the body of a lengthy text-based documents. In particular, Conclusion- Analysis-Evidence (CAE) diagrams were used to provide a graphical overview of evidence and lines of argument about human and organisational failure. A Marine Incident Investigation Unit's (MIIU) report into a fire on the Aurora Australis was used as a case study to illustrate the argument in this paper. This paper presents the findings of an initial investigation into the organisational precursors that lead to breakdown in human computer interaction during major

• accidents. Our motivation is to move beyond Norman's focus on "inappropriate

16 feedback" (1990) to look at the reasons why users may be required to operate inadequate systems in the first place. The main contribution of this work has been to extend the application of tools from more "conventional" forms of human computer interaction, and design rationale in particular, to represent and reason about these

• rganisational precursors to interaction problems. As such, our work represents a first

step to provide the analytical tools that are necessary if we are to better understand the causes of "organisational accidents" (Reason, 1998). Acknowledgements Thanks are due to the members of the Glasgow Interactive Systems group (GIST) and the members of the Glasgow Accident Analysis Group. This work is supported by the UK Engineering and Physical Sciences Research council. References

• H. Beyer and K. Holtzblatt, Contextual Design: Defining Customer-Centred Systems,

Morgan Kaufmann, San Francisco, USA, 1998. J.S. Brown, From cognitive ergonomics to social ergonomics and beyond. In Norman D.A. and Draper, S.W. (Eds.) User Centred Systems Design, 457-486. Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1986.

• J. Conklin, J. and M.L. Begeman, gIBIS: A Tool For All Reasons. Journal Of The

American Society For Information Science, 200-213, 1989. C.W. Johnson, Using CAE Diagrams to Visualise the Arguments in Accident Reports, 1999. URL page: http://www.dcs.gla.ac.uk/~johnson/papers/cae_99/ C.W. Johnson, A First Step Towards the Integration of Accident Reports and Constructive Design Documents. In M. Felici, K. Kanoun and A. Pasquini (eds.), Computer Safety, Reliability and Security: Proceedings of 18th International Conference SAFECOMP'99, 286-296, Springer Verlag, 1999a C.W. Johnson, Visualising the Relationship between Human Error and Organisational

• Failure. In J. Dixon (ed.) Proceedings of the 17th International Systems Safety

Conference, The Systems Safety Society. Unionville, Virginia, USA, 101-110, 1999b

• A. Maclean, R. Young, V. Bellotti, & T. Moran "Questions, Options and Criteria:

Elements of Design Space Analysis," Human-Computer Interaction, 6(3&4), 201-250, 1991

17 Maritime Incident Investigation Unit, Investigation into a Fire in the Engine Room Aboard the Antarctic Research and Supply Vessel Aurora Australis at the Antarctic ice edge, on 22 July 1998. Report 135, ISBN 0 642 20019 X, Department of Transport and Regional Development, Canberra, Australia, 1999. T.P. Moran, and J.M. Carroll, Design Rationale Concepts, Techniques And Use. Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1995.

• D. Norman, The Problem with Automation: Inappropriate Feedback and Interaction not

Over-Automation. In D. Broadbent, J. Reason and A. Baddeley (eds.), Human Factors in Hazardous Situations, Clarendon Press, Oxford, 137-145, 1990.

• J. Reason, Managing the Risks of Organisational Accidents, Ashgate, Aldershot, 1998.

S.J. Shum, A Cognitive Analysis of Design Rationale Representation. Doctoral Dissertation, Department of Psychology, University of York, UK, 1991.

• P. Smolensky, B. Fox, R. King, &, C. Lewis, Computer-aided reasoned discourse or,

how to argue with a computer. In Guindon, R. (eds.) Cognitive Science and its Application to Human-Computer Interaction, 109-162, Ablex: Norwood, New Jersey, 1998. US Department of Energy, Root cause analysis guidance document. DOE-NE-STD- 1004-92. Office of Nuclear Energy, Washington DC, 1992.