How can ordinary punters hold public organisations to
account? In the (possibly fleeting) age of the Freedom of Information Act, and
in the more durable age of the internet, the possibilities for finding out
about our public organisations have in some ways radically improved. However, this
new age brings at least two problems. The first is knowing where to look – the multiplication
and shape-shifting nature of public organisations makes hiding information in
plain sight quite easy. The second is trying to understand what the ‘publicly
available’ data actually mean – unless you’re an expert in how the data are
collected interpreting them can be really difficult.
My reason for mentioning these problems is that I’ve only
just remembered about an obscure (to me) national database of information about
the number and nature of ‘incidents’ reported by NHS Trusts. All NHS Trusts
have a duty to report monthly information on patient safety incidents to ‘Patient
Safety’ (now part of NHS England, up until 2012 the DH-funded National Patient
Safety Agency). Their website publishes six-monthly summaries of incidents
reported by NHS Trusts in England, with useful comparative information by Trust
type (e.g. Mental Health NHS Trusts, of which Southern Health is one of 55-57,
depending on the time period) – see here http://www.nrls.npsa.nhs.uk/patient-safety-data/organisation-patient-safety-incident-reports/
If Southern Health are going to make any claims about ‘lessons
learned’, then presumably this should show up in their incident reporting. This
is where the second problem comes up – interpreting incident data is really
difficult. This blogpost briefly summarises the information collected by
Patient Safety in six month chunks from October 2011 (when Southern Health
Trust officially came into existence) to September 2014. Interpretation,
especially when it comes to any ‘lesson learned’, is another matter.
How many incidents?
First, Patient Safety report the total number of incidents
reported by the Trust in each six month period, and the number of incidents per
1,000 bed days (to take into account the size of the Trust) with a comparator
of information from across all mental health trusts. Surely having fewer incidents
is better, right? Well, the National Patient Safety Agency summaries have this
standard line “Organisations that report more incidents usually have a better
and more effective safety culture. You can't learn and improve if you don't
know what the problems are.”
The two graphs below show that the total number of incidents
reported by Southern Health Trust vary widely over the 3½ years, with numbers
reducing to Apr-Sept 2012 and rapidly increasing after that. Given the absorption
of Ridgeway in late 2012, an increase in the number of incidents would be
expected in the Oct 2012-March 2013 data, but it is not clear why there are
further increases after that.
Does this count as ‘learning lessons’ in terms of incident
reporting? Well, the second graph shows that the number of reported incidents
per 1,000 bed days was also dropping to Apr-Sept 2012, to a much lower level
than comparator mental health trusts. A temporary bump in Oct 2012-March 2013
was followed by a further reduction below comparator trusts throughout April
2013 to March 2014, with a sudden, dramatic increase in Apr-Sept 2014. If
lessons are being learned, they seem only to been learned relatively recently,
and the big fluctuations in incident reporting rates over time do not suggest
steady improvements in incident reporting.
What types of
incident?
Second, Patient Safety reports the types of incident
recorded by each NHS Trust. In interpreting this information, Patient Safety
state “If your reporting profile looks different from similar organisations,
this could reflect differences in reporting culture, the type of services
provided or patients cared for. It could also be pointing you to high risk
areas. The response system is more important than the reporting system.”
The three graphs below report the three most common types of
incident for Southern Health; patient accidents; disruptive, aggressive
behaviour; and self-harming behaviour; with comparative information from all
mental health trusts combined.
Compared to all mental health trusts combined, from April
2012 through to March 2014 Southern Health consistently reported a much higher
proportion of patient accidents, with a sharp drop in April-September 2014.
The opposite is true for disruptive, aggressive behaviour, where
Southern Health consistently reported much lower proportions than comparator
mental health trusts until March 2014, with a sudden sharp increase to
comparative levels in April-September 2014.
With some fluctuations, Southern Health reported higher
proportions of self-harming behaviour than comparator mental health trusts up
until September 2013 – after this levels are similar to comparator trusts.
As with total incident rates, the proportions of these types
of incident are consistently out of line compared to other mental health trusts
until April-September 2014, with wide fluctuations over time.
How much harm do
incidents cause?
Third, Patient Safety reports information on the level of
harm reported for each incident, from ‘None’ through to ‘Death’. Deaths of
patients do not necessarily have to be recorded as incidents. On harm, Patient
Safety states “Nationally, 68 per cent of incidents are reported as no harm,
and just under 1 per cent as severe harm or death. However, not all
organisations apply the national coding of degree of harm in a consistent way,
which can make comparison of harm profiles of organisations difficult.
Organisations should record actual harm to patients rather than potential
degree of harm.”
The graph below shows the percentage of incidents reported
in each category of harm, for mental health trusts as a whole and for Southern
Health. The graph looks a bit complicated, but the main difference is that
incidents in Southern Health are consistently more likely to be reported as
causing ‘Moderate’ harm compared to other mental health trusts. There seem to be
no obvious differences over time.
The number of deaths reported as incidents by Southern
Health are in the graph below. A very low number of deaths reported as
incidents in April-September 2011 is followed by a huge increase in the number
of deaths reported as incidents in October 2011-March 2012, then the number of
deaths reported as incidents successively drops to a very low level again by
October 2013-March 2014. I do not know the total number of deaths occurring
across Southern Health services, so I cannot say whether there are fewer deaths
overall or whether smaller proportions of patient deaths are being reported as
incidents over time.
How quickly are
incidents reported?
Finally, Patient Safety set great store by the fast
reporting of incidents: “Report serious incidents quickly: It is vital that
staff report serious safety risks promptly both locally and to the NRLS, so
that lessons can be learned and action taken to prevent harm to others.”
Patient Safety report the median length of time in days that it has taken each
organisation to report an incident.
The graph below shows the median number of days it took
Southern Health and comparator mental health trusts to report incidents.
Compared to other mental health trusts, from April 2011 to September 2012
Southern Health were much quicker to report incidents than mental health trusts
generally, but from April 2013 onwards they have been much slower.
What does it all
mean?
I’m not an expert in patient safety incident reporting, so I
can’t produce an informed interpretation of this information. For rates and
types of incident reported, it looks like there are pretty consistent
differences between Southern Health’s patterns of incident reporting and those
of other mental health trusts up to April 2014 – rather a time lag to be
claiming to have ‘learned lessons’ from Connor’s preventable death (originally
categorised by the trust as due to ‘natural causes’, let us remember). Given
recent increases in the total number and rates of incidents reported, it’s unclear
to me why the number of deaths reported as incidents should have been
decreasing over the same time period. If lessons have been learned about
incident reporting, it is also unclear to me why it should still be taking so
long to report incidents. Another case of publicly available statistics concealing
as much as they reveal?
