Incidental?

How can ordinary punters hold public organisations to account? In the (possibly fleeting) age of the Freedom of Information Act, and in the more durable age of the internet, the possibilities for finding out about our public organisations have in some ways radically improved. However, this new age brings at least two problems. The first is knowing where to look – the multiplication and shape-shifting nature of public organisations makes hiding information in plain sight quite easy. The second is trying to understand what the ‘publicly available’ data actually mean – unless you’re an expert in how the data are collected interpreting them can be really difficult.

My reason for mentioning these problems is that I’ve only just remembered about an obscure (to me) national database of information about the number and nature of ‘incidents’ reported by NHS Trusts. All NHS Trusts have a duty to report monthly information on patient safety incidents to ‘Patient Safety’ (now part of NHS England, up until 2012 the DH-funded National Patient Safety Agency). Their website publishes six-monthly summaries of incidents reported by NHS Trusts in England, with useful comparative information by Trust type (e.g. Mental Health NHS Trusts, of which Southern Health is one of 55-57, depending on the time period) – see here http://www.nrls.npsa.nhs.uk/patient-safety-data/organisation-patient-safety-incident-reports/

If Southern Health are going to make any claims about ‘lessons learned’, then presumably this should show up in their incident reporting. This is where the second problem comes up – interpreting incident data is really difficult. This blogpost briefly summarises the information collected by Patient Safety in six month chunks from October 2011 (when Southern Health Trust officially came into existence) to September 2014. Interpretation, especially when it comes to any ‘lesson learned’, is another matter.

How many incidents?

First, Patient Safety report the total number of incidents reported by the Trust in each six month period, and the number of incidents per 1,000 bed days (to take into account the size of the Trust) with a comparator of information from across all mental health trusts. Surely having fewer incidents is better, right? Well, the National Patient Safety Agency summaries have this standard line “Organisations that report more incidents usually have a better and more effective safety culture. You can't learn and improve if you don't know what the problems are.”

The two graphs below show that the total number of incidents reported by Southern Health Trust vary widely over the 3½ years, with numbers reducing to Apr-Sept 2012 and rapidly increasing after that. Given the absorption of Ridgeway in late 2012, an increase in the number of incidents would be expected in the Oct 2012-March 2013 data, but it is not clear why there are further increases after that.

Does this count as ‘learning lessons’ in terms of incident reporting? Well, the second graph shows that the number of reported incidents per 1,000 bed days was also dropping to Apr-Sept 2012, to a much lower level than comparator mental health trusts. A temporary bump in Oct 2012-March 2013 was followed by a further reduction below comparator trusts throughout April 2013 to March 2014, with a sudden, dramatic increase in Apr-Sept 2014. If lessons are being learned, they seem only to been learned relatively recently, and the big fluctuations in incident reporting rates over time do not suggest steady improvements in incident reporting.

What types of incident?

Second, Patient Safety reports the types of incident recorded by each NHS Trust. In interpreting this information, Patient Safety state “If your reporting profile looks different from similar organisations, this could reflect differences in reporting culture, the type of services provided or patients cared for. It could also be pointing you to high risk areas. The response system is more important than the reporting system.”

The three graphs below report the three most common types of incident for Southern Health; patient accidents; disruptive, aggressive behaviour; and self-harming behaviour; with comparative information from all mental health trusts combined.

Compared to all mental health trusts combined, from April 2012 through to March 2014 Southern Health consistently reported a much higher proportion of patient accidents, with a sharp drop in April-September 2014.

The opposite is true for disruptive, aggressive behaviour, where Southern Health consistently reported much lower proportions than comparator mental health trusts until March 2014, with a sudden sharp increase to comparative levels in April-September 2014.

With some fluctuations, Southern Health reported higher proportions of self-harming behaviour than comparator mental health trusts up until September 2013 – after this levels are similar to comparator trusts.

As with total incident rates, the proportions of these types of incident are consistently out of line compared to other mental health trusts until April-September 2014, with wide fluctuations over time.

How much harm do incidents cause?

Third, Patient Safety reports information on the level of harm reported for each incident, from ‘None’ through to ‘Death’. Deaths of patients do not necessarily have to be recorded as incidents. On harm, Patient Safety states “Nationally, 68 per cent of incidents are reported as no harm, and just under 1 per cent as severe harm or death. However, not all organisations apply the national coding of degree of harm in a consistent way, which can make comparison of harm profiles of organisations difficult. Organisations should record actual harm to patients rather than potential degree of harm.”

The graph below shows the percentage of incidents reported in each category of harm, for mental health trusts as a whole and for Southern Health. The graph looks a bit complicated, but the main difference is that incidents in Southern Health are consistently more likely to be reported as causing ‘Moderate’ harm compared to other mental health trusts. There seem to be no obvious differences over time.

The number of deaths reported as incidents by Southern Health are in the graph below. A very low number of deaths reported as incidents in April-September 2011 is followed by a huge increase in the number of deaths reported as incidents in October 2011-March 2012, then the number of deaths reported as incidents successively drops to a very low level again by October 2013-March 2014. I do not know the total number of deaths occurring across Southern Health services, so I cannot say whether there are fewer deaths overall or whether smaller proportions of patient deaths are being reported as incidents over time.

How quickly are incidents reported?

Finally, Patient Safety set great store by the fast reporting of incidents: “Report serious incidents quickly: It is vital that staff report serious safety risks promptly both locally and to the NRLS, so that lessons can be learned and action taken to prevent harm to others.” Patient Safety report the median length of time in days that it has taken each organisation to report an incident.

The graph below shows the median number of days it took Southern Health and comparator mental health trusts to report incidents. Compared to other mental health trusts, from April 2011 to September 2012 Southern Health were much quicker to report incidents than mental health trusts generally, but from April 2013 onwards they have been much slower.

What does it all mean?

I’m not an expert in patient safety incident reporting, so I can’t produce an informed interpretation of this information. For rates and types of incident reported, it looks like there are pretty consistent differences between Southern Health’s patterns of incident reporting and those of other mental health trusts up to April 2014 – rather a time lag to be claiming to have ‘learned lessons’ from Connor’s preventable death (originally categorised by the trust as due to ‘natural causes’, let us remember). Given recent increases in the total number and rates of incidents reported, it’s unclear to me why the number of deaths reported as incidents should have been decreasing over the same time period. If lessons have been learned about incident reporting, it is also unclear to me why it should still be taking so long to report incidents. Another case of publicly available statistics concealing as much as they reveal?