Measuring for improvement, not for judgement: How NHS surgeons are leading the world in publishing outcomes data

Moira Livingston

Moira Livingston

Professor Moira Livingston, clinical director of improvement capability, NHS Improving Quality.

This blog post is a shortened version of an article published in the January issue of The Consultant

Last summer, consultant level outcomes data across nine surgical specialities were made publically available for the first time. The English NHS is the first national healthcare system in the world to do this. The consultants who have agreed to their data being published are at the forefront of the move to increase transparency and improve choice and outcomes for patients; which was highlighted as a recommendation in Don Berwick’s patient safety review. However, the move to publish outcomes data was met with concern by some healthcare professionals, who fear that the information could be misinterpreted. There is a perceived risk that surgeons with seemingly ‘poor’ scores might be vilified in the media or find that it has a negative consequence on their careers; and this could result in a situation where some surgeons avoid carrying out high risk procedures.

Here at NHS Improving Quality, we believe that much good can come from publishing the data. The remit of our organisation is to help drive up the quality of outcomes and experience for patients across the health and care system in England by providing improvement and change expertise. The publication of outcomes data presents us with a powerful opportunity to help improve the quality of care for patients. Outcomes data from heart bypass surgery have been published since 2005, and mortality rates have dropped by 50% since then (Tavare 2012). Rather than using data to benchmark performance or as a judgement of compliance against a target, the same data can be used to drive improvement – so increasing transparency in this way can provide a strong impetus to improve the quality of services throughout the health and care system.

Changing the mindset

The move towards focusing on outcomes rather than targets is a positive one, but the target culture has left unease amongst some in the NHS clinical community. Targets are evidenced with measurable data, so any system which measures and publishes data can bring to mind the negative connotations of judgement, and ‘pointing the finger’ at professionals who may be deemed to be not performing as well as they should. At NHS IQ, we want to help change people’s mindsets and to help colleagues understand how data can be used much more positively.

According to Professor Robert Lloyd, Director of Performance Improvement at the Institute of Healthcare Improvement (USA):

“The issue of transparency for judgement or performance management is in sharp contrast to transparency that is driven by intrinsic motivation and the desire for improvement. 

“In fact, the same data can be used for either judgement or improvement. The difference comes in the mindset of the people releasing the data.”

But changing a mindset and a culture is easier said than done. We need to trust the accuracy of the data, that it tells us what we think it tells us, and will be used in a responsible way. We also need to fully understand the purpose behind collecting data. We need to move away from believing that collecting data is simply to identify surgeons who deviate from minimally acceptable patient outcomes, but instead have trust and confidence that the data will be used to help clinicians and specialist societies identify areas for improvement. Above all, we need to help make data available and accessible so that patients and their families can make informed choices about their care, and be involved in decision-making.

Putting patients in the driving seat

Patient experience is central to this issue. The NHS reforms seek to put patients in the driving seat of their care, and ensure they are equipped with appropriate information to make decisions about their treatment. However, this begs the question: if we’re focusing on patients, why have we started this process of transparency with mortality data? There are many other metrics that demonstrate whether a patient has received good quality care. Some of these are easy to measure, such as the length of hospital stay or avoidance of infection; others concerning quality of life are more complicated to capture, but could include whether the patient has returned to work, or if they can run around the park with their children. We hope that publishing mortality data is just the starting point and that, in time, all of these factors can be routinely measured and published to help us drive up the quality of all aspects of care, beyond the surgical specialties that have taken this initial courageous step.

Seeking commitment, not compliance

In order to successfully use outcomes data for improvement, clinical engagement is key.  Edgar Schein, Professor Emeritus at MIT Sloan School, summed things up nicely with his explanation that:You can’t impose anything on anyone and expect them to be committed to it”. Of course, when public money is involved, there must be some element of compliance, but only genuine commitment from the clinical community can lead to greater and faster improvement. In her work to describe how to achieve transformational change, Helen Bevan points out that we need to move away from a system where there is a specified minimum performance standard that everyone must achieve; where people are held to account within a hierarchy through performance management; and where delivery is driven through the fear of penalties, sanctions or professional shame. Instead, we must work towards a collective goal for better outcomes that everyone can aspire to; holding our peers to account through collective sign up to action, with delivery driven through commitment to a common improvement.

Learn from the positive deviants

“Measuring for improvement” rather than “measuring for judgement” raises the standard of care by default. If our aim is to ensure that mortality outcomes from surgical procedures do not rise above an accepted rate, it’s likely that the national average mortality rate will hover around that agreed marker. If our aim is to improve future outcomes, this creates momentum to constantly seek to lower the mortality rate ever further.

Otherwise, there is a risk that those who are “high performers”, and even those in the middle of the pack, might breathe a sigh of relief and carry on with business as usual. We need to understand in what ways those surgeons who show the best outcomes are doing things differently, and spread this as practical advice and guidance for everyone. Rather than focus on the outliers who are not performing as well as can be expected, we should look at those who are performing excellently, the ‘positive deviants’,  and seek to learn from them. Improvement science shows us that the higher the bar is set by the strongest performers continuing to improve their performance, the more the whole system is pulled forward into better performance and better outcomes for patients (Institute for Healthcare Improvement 2002). We will get faster, more sustainable change if we focus on strengths rather than deficiencies in our approach to surgical outcomes data.

Measurement masterclass

At NHS Improving Quality, we recently delivered an innovative and engaging measurement masterclass series for senior clinical leaders. The series was designed to strengthen understanding amongst clinicians of the principles of measurement for improvement, and equip them with the confidence to hold influential discussions with policy makers, data collectors, and other clinicians across the health sector.

This approach to using data for improvement is not new, and our ambition now is to drive a significant increase throughout the NHS. We wanted to offer senior leaders the opportunity to really explore with national and international experts why data measurement is critical to successful transformation, and help them to ensure greater certainty about the data upon which much of their decision-making is based.

The masterclass series has not been a means to an end – for those who attended, it is the beginning of a journey in which they will continue to build their skills and understanding of measurement, and use the techniques they have learned to help inform decisions they make every day.  I want to see others benefitting from the programme too – there is a wealth of resources available on the NHS IQ website which I hope you will look at and share with your colleagues.  Understanding measurement and using it effectively is a vital tool in all of our endeavours to drive system change and to improve quality of care for patients throughout the NHS.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s