A Study of how to effectively leverage incident investigations to better inform system safety - McCaughan, Cora (Trinity College Dublin. School of Psychology. Discipline of Psychology, 2020)
Purpose: The purpose of this thesis was to explore how to effectively leverage incident investigations to better inform system safety. Method: Four studies were conducted to achieve the purpose of this research. Study 1 used an iterative action research approach to develop and test an Investigation Quality Evaluation Tool (IQET) to rigorously and reliably evaluate the quality of serious incident investigation reports, including evaluating what Dekker (2006) referred to as the analytic trace for investigation findings. Study 2 applied the IQET to evaluate the quality of serious incident investigation reports assigning Investigation Quality Scores (IQS) and identifying what was done well and what needed to improve. Study 3 used data from (i) the evaluation of investigation quality, and (ii) attendance at investigation training - to empirically test hypothesis about the determinants of investigation quality. Study 4 conducted a thematic analysis of further details of contributory factors identified in investigation reports to identify patterns in causal factors. Results: Study 1 showed reasonable IQET inter-rater reliability whereby this was excellent in 55.5% (n=5) of cases (kappa value ranging from .756 - .859), and fair to good in the remaining 44.44% of cases (n=4) (kappa value ranging from .418 - .587). Study 2 found that IQSs ranged from 13.79% - 78.13% with a mean of 45.17%. Elements satisfactorily done most frequently included placing events in chronological order in the chronology, and evidence of review of records. Elements not satisfactorily done related to generalizing from investigations, and not using the hierarchy of controls to develop recommendations. Study 3 revealed that there was a positive statistically significant correlation between IQS and (i) attendance at NIMLT training; (ii) having investigation expertise on the investigation team; (iii) having a team of not less than 2 and not more than 3 investigators; (iv) conducting individual interviews; and (v) adherence to the definition of Key Causal Factor (KCF) in investigation guidelines. Study 4 identified nine main themes in causal factors including (i) Care pathways, PPPGs and other tools that support care delivery (ii) education, training and supervision; and (iii) Governance and risk management. Conclusions: This thesis shows that it is possible to develop a reliable tool to comprehensively evaluate the quality of investigation reports and that data from this can be used to empirically test hypotheses about the determinants of investigation quality. It reflects analysis at a deeper level of causal factors to identify patterns in causal factors in a larger batch of investigations from a wider span of the health system than done in previous research. It reveals that the outcome of this analysis identifies emergent system features which (i) add to information about existing risks, (ii) identify newly emerging risks, and (iii) are important for informing safety improvement across the organisation. It achieves a resolution of the contradiction between the complexity of the system and the need to identify sufficient cause to implement prevention. Above all, the findings of this thesis supports a change in the concept of how to generalize from investigations for system wide learning, and presents a model of how to continuously improve how to do this.
APPROVED
Keywords: System; Safety; System safety; Incident investigations