Leadership of change (process)

Using evidence to support change

The current social care climate is heavily shaped by the need for modernisation and delivering a high level of change in order to achieve better outcomes and improve practice, all whilst demonstrating value for money. Such increasing expectations mean that leaders and managers must place an increasing emphasis on research and evidence at all levels of policy, commissioning and practice. Increasingly, leaders are required to produce robust evidence of the impact of their work, or develop strong business cases to support investment in new models or approaches to health and care. This means they need to understand how to use and interpret different forms of evidence, including cost-benefit and quantitative data.

A commitment to using evidence to support change means that leaders and managers need to:

• support a whole system approach that promotes a culture of research mindedness and evidence-informed practice
• ensure research and evidence is embedded into and aligned with accountability processes related to planning, decision-making, governance and supervision
• foster a continuous learning culture by encouraging staff to seek out, critically appraise and apply the best available evidence as an integral part of their role
• provide a continuum of training including critical appraisal skills and research methodology and processes
• use evidence to support decision-making and make cases for investment in new ways of working or the commissioning or investment in new interventions or services.

For more information

School for Social Care Research

Research in Practice

Theory of Change model

The Theory of Change model, which was initially developed as an evaluative methodology, is increasingly used to support the development of change interventions. The focus is on facilitating stakeholders to identify long-term programme objectives and then working back through the sequence of events (‘causal pathway’) required to reach the goals, and planning changes accordingly. These sequences are mapped diagrammatically to show the logical relationships between each step. The distinctive elements of the approach lie in the requirement of participants to explicitly model the desired outcomes of an intervention, and enabling explicit comparisons between planned effects and those actually achieved following implementation.

When to use it?

The technique is used in many settings and sectors. The approach is very flexible and may be used at any point during the implementation of a programme or initiative. From the outset, it may be used to inform the development of interventions consistent with planned outcomes. Alternatively, it may be used following implementation to assess the extent to which changes were implemented as planned and the degree to which anticipated benefits were realised. Implementation and evaluation are kept transparent to stakeholders, so that participants are always aware of what is happening and why.

In relation to social care, the explicitly participatory planning phase is helpful in involving a wide range of partners in reaching agreement over anticipated outcomes, revealing the logic of how a planned change will yield anticipated benefits, and planning the stages required to achieve these outcomes.

Stages of the Theory of Change approach

1. Define the problem to be tackled and identify long-term goals/outcomes to be achieved (and the assumptions behind them)
2. Identify the preconditions/change mechanisms required for the goals to be realised and explain the rationale linking the preconditions to outcomes
3. Map backwards to define shorter-term outcomes that will make sustained impact more likely
4. Consider and identify the optimal interventions required to bring about the required changes: activities, processes, initiatives
5. Identify performance indicators to measure required outcomes
6. Write a narrative which summarises the logic of the initiative

Case study: Think Local Act Personal

Think Local Act Personal (TLAP) is a national partnership of more than 50 organisations committed to transforming health and care through personalisation and community-based support. The partnership spans central and local government, the NHS, the provider sector, and people with care and support needs, carers and family members. The partnership’s unique strength is bringing together people who use services and carers with national organisations. Its work involves securing specific agreements from partners on implementing and improving the delivery of personalisation and community-based support and shaping and influencing national policy.

As part of an independent evaluation of its activities, TLAP developed a theory of change model, which identifies the reach and role of the TLAP partnership, underpinned by a shared vision of personalisation and co-production. The model outlines the impact of TLAP on the people and organisations that directly access TLAP’s support as resources – whether as a partner or through the wider network – to understand the extent to which this engagement has changed their knowledge, awareness and understanding of personalised care, and put this into practice.

Figure 8: TLAP Theory of Change Model

For more information

View the Theory of Change template from Nesta

View this helpful guide from NPC on creating a theory of change in 10 steps: Theory of Change in 10 steps (NPC)

Quality improvement and change

Quality improvement has been a popular concept in organisational and industrial settings and has also been adapted for use in the clinical world to make healthcare safe, effective, patient-centred, timely and efficient. Similarly, it has more recently been employed in social care to bring about change in frontline practice, make service user experiences better and to identify whether change in practice has made a difference. Quality improvement is underpinned by a desire or an attitude to want to make things better for people, and thus fits well with social work and social care values.

There is no single definition of quality improvement, but various definitions include improvement or change combined with a method or approach in order to achieve better outcomes and a better service user experience. For example, it has been described as ‘a systematic approach using specific methods to bring about measurement and sustained improvements to health and social care’. It is therefore an important concept and tool for leaders considering and/or implementing changes, no matter how big or small.

Quality Improvement model – PDSA

There are several quality improvement models and approaches that organisations can consider. One of the most widely used models is the Plan-Do-Study-Act (PDSA) Cycle – a four-stage process improvement model used to rapidly trial planned changes to practice, incorporate the lessons learned from the trial into the change process, and integrate continuous improvement into the new service.

The main strengths of PDSA cycles are derived from the rapid testing of proposed changes on a small scale. This allows changes to be achieved relatively quickly and cheaply, with good early indication of the strengths and weaknesses of the proposed changes. Data from the pilot intervention enables the early identification of potential problems, and the involvement of staff in the implementation and improvement process means that they are less likely to resist the changes – they are part of the solution and their input is central to the process.

In relation to social care change, PDSA provides a means to test out improvements in service delivery that have been suggested by people accessing services, their carers, staff or other stakeholders and to learn what works and what does not. It requires senior management to allow frontline staff to test out new ideas and share this learning across the organisation.

Stages of a PDSA

1. Planning the change – this initial stage requires a thorough diagnostic evaluation of the problems/issues that need to be addressed by the change – firstly to identify their probable causes and secondly to design possible solutions
2. Doing the change – a small-scale (pilot) project is undertaken as a limited implementation of the proposed change(s). Data is collected and observations made to identify the impact of the change and any unintended consequences
3. Studying the change – results of the pilot are compared with expectations to identify unintended consequences and shortfalls in improvements
4. Acting on what you have learned – improvements to the proposed intervention are made, incorporating lessons learned from the pilot phase. The changes may be tested again in another pilot, or the decision taken to proceed to full implementation. Ongoing quality improvement activity (continuous cycles) may also be adopted.

Model for Improvement – incorporating PDSA cycles

The Model for Improvement is a simple yet powerful tool for accelerating improvement which asks three critical questions, as set out in Figure 9. It has been used successfully by hundreds of health care organisations around the world to improve many different health care processes and outcomes. The Model for Improvement asks three questions:

• What are we trying to accomplish? Agreeing and setting SMART aims for the improvement project.
• How will we know change is an improvement? Successful measurement is a cornerstone of successful improvement. Clear measures need to be identified at the outset.
• What changes can we make that will result in improvement? Generating ideas for changes that will result in improvement.

Having answered the three questions in the Model for Improvement, the next step is using the PDSA cycle or cycles to test whether the change results in improvement.

Figure 9: Model for Improvement (incorporating PDSA cycles)

Evaluating changes and interventions

Introduction

Evaluation, or measuring success, is a key part of any quality improvement initiative, change or intervention and when done well it can help solve problems, inform decision-making and build knowledge. In fact, evaluation in social care can serve a number of purposes:

• For social care teams and frontline staff, an evaluation is a means of learning whether a desired change has been achieved, whether it represents an improvement, whether it caused problems, how the intervention worked and whether it is sustainable.
• For people who use services, carers and their representatives, it can be an opportunity to have their perspectives heard and taken into account.
• For funders or budget holders, it can provide feedback on whether the provision of financial and human resources needed for the intervention was justifiable, thus ensuring transparency and accountability. It can help them to decide whether or not to fund future work and gain ‘buy-in’ from others.
• For those undertaking improvement work, evaluation can be a resource for learning and sharing knowledge about what others have found to work, and what they have found to be difficult or unsuccessful. This can include whether and how learning within a particular context might be applied more generally.

Moreover, evaluation when done well should be built into the way people and organisations work and be supported in that organisations should have access to training and support to build their evaluation skills.

Types of evaluation

Most evaluations seek to find out if an intervention has had an effect on some observable outcomes, such as improved wellbeing, reduced social isolation or increased independence. These are called ‘impact’ or ‘summative’ evaluations as it can be seen as ‘summing up’ the overall effect of the intervention. This type of evaluation might show whether the intervention worked and met its objectives, what improvements, if any, have been made. It is often carried out at the end, when all the data is available to help the evaluators to determine whether it has been a success or not, often against stated goals.

However, robust evaluation tells us not only whether an intervention worked, but also why and how. This form of evaluation is usually called a ‘process’ or ‘formative’ evaluation and is designed to help form or shape an intervention. It is used as the intervention evolves and can provide information about how best to revise and modify the work taking place. It can help people to explore not only whether improvement has been achieved, but also how it has occurred in their particular environment. Most evaluations combine elements of formative and summative evaluations.

Stages of an evaluation

Identifying outcomes and measures

Outcomes are the changes or benefits that are delivered as a result of an improvement, intervention or service. They should be linked to improved experiences of care as defined by people and capture expected changes, for example to people’s wellbeing, independence and/or choice and control. They should also consider the use of resources as value for money is critical in health and social care. Having clear, focused outcomes is important as this helps identify what to measure in order to find out whether the intervention is making a difference, and to what extent.

Measures define the data that should be collected to develop an understanding about progress and impact. Good measures enable judgement on progress towards outcomes and help collect evidence in a systematic way. A mix of measures is usually needed to ensure that evidence gathered is sufficiently detailed and meaningful for the range of stakeholders involved in, or affected by, the intervention. In addition to the typical measures of personal outcomes (improved quality of life, greater independence) and service proxy measures for improved outcomes (delayed transfers of care, avoidable admission to hospital), there are also measures for user and carer experiences, organisational processes and characteristics and resource utilisation to consider.

Developing a logic model

Before thinking about how to evaluate an intervention, it is important to understand the nature of the intervention by developing a logic model. A logic model is a diagram that describes how an intervention will bring about the desired outcomes. It usually describes:

• Context – what needs to be in place locally to support a successful intervention.
• Inputs – the things you put into the intervention, such as people’s time, money, infrastructure to make it work.
• Activities – what you actually do to make the intervention bring about change.
• Outputs – the immediate results of the intervention, e.g. number of people receiving a single care plan, number of staff trained.
• Outcomes – the short, medium and long term.

A logic model should be developed up front, as part of the intervention design phase, to help clarify assumptions about how inputs and activities are likely to deliver intended outputs, outcomes and impacts. A well thought-out logic model will guide effective and systematic data collection and form the evidence base for assessment of progress and impact over time.

It is important to work in partnership with stakeholders and people who use services and carers to develop the logic model. Co-designing the logic model identifies the different views, values and priorities of each stakeholder and what they would like to gain from both the intervention and the measurement or evaluation of it.

A logic model is very similar to the Theory of Change model described earlier. The latter is a narrative approach that works backwards from the desired outcomes, whereas a logic model works forward from the actions that need to take place to drive the change that is likely to result in the desired improvement. Whichever approach is used the most important thing is to work with all stakeholders and people who use services and carers to develop the model. Co-designing the model ensures that the different views, values and priorities of each stakeholder and what they would like to gain from the intervention are taken into account.

Evaluation approach and data collection methods

There are many approaches to evaluation and selecting an approach depends on a number of considerations, including but not limited to:

• types of research questions or evaluation objectives
• type of intervention that is being evaluated
• resources available for the evaluation
• timescales for the evaluation
• range of stakeholders and their priorities.

Some different approaches to evaluation include:

• Randomised control trial – where people participating in the trial are randomly allocated to either the group receiving the treatment under investigation or to a group receiving standard treatment as the control
• Matched control comparison – when the population receiving the intervention is compared against an appropriately matched comparator population
• Before and after comparison – data is collected for a sample of people who take part in an intervention to see what changes have happened
• Case study – in depth, usually retrospective, write-up of people’s experience of an intervention
• Interviews, observations and focus groups – provide detailed information about how a programme is working in practice, by giving you insight into the attitudes, opinions and experiences of affected groups.

Most evaluations use a mixed-methods approach, which adds to the robustness and credibility of the findings. Specific issues to consider when planning an evaluation include equal opportunities, ethical approval, information sharing and informed consent.

Communicating findings

Communicating the findings of an evaluation should be driven by a clear understanding of who or what is to be influenced in terms of stakeholders involved and decisions to be made. All those involved in the intervention should get feedback based on the findings. There are many creative and helpful ways to communicate findings alongside a formal report. These include high-level briefings or slide decks of key findings, academic papers and conference presentations, press releases or news stories, YouTube videos or e-newsletter and intranet and staff meetings.

For people concerned with the practical running of an intervention, feedback may be needed regularly to adapt and improve the intervention, making interim reports and less formal feedback mechanisms vital in informing key decision points with a summary of progress so far, as well as initial findings. Finally, it is important to be transparent about the findings, including those that are less positive and about any potential limitations of the evaluation methods employed.

This animation highlights the benefits of evaluating an improvement project (West of England Academic Health Science Network, 2016). The website has many other useful resources to support improvement in health and social care.

For more information

This is a helpful guide by the Health Foundation on evaluation in health and social care (Health Foundation, 2015)

This is an interactive logic model template developed by Evaluation Support Scotland (Evaluation Support Scotland, 2019)

This is a detailed web resource from SCIE on evaluating personalised care (SCIE, 2020)