Developing, implementing and evaluating the Yorkshire Patient Experience Toolkit (PET): How process evaluation can strengthen action research

Conference presentation


Mills, T., Lawton, R. and Sheard, L. (2019). Developing, implementing and evaluating the Yorkshire Patient Experience Toolkit (PET): How process evaluation can strengthen action research. Health Services Research UK.
AuthorsMills, T., Lawton, R. and Sheard, L.
TypeConference presentation
Abstract

Background:

Formative and participatory research and improvement methodologies are increasingly advocated in the field of intervention research, in recognition of the complexity of both healthcare problems and the interventions required to solve them (Andersson, 2018). One such approach is action research, which is said to be uniquely capable of developing solutions for complex problems through relational working and the utilisation of locally-situated knowledge (Moch et al, 2016). However, this participatory shift poses the question of how research findings can be scaled-up for transfer into actual healthcare practice. In the case of action research, for example, successful “action” within a healthcare setting can come at the expense of rigorous research (Hammersley, 2004). Here, we report on an innovative study which sought to grapple with this problem of generalisation, combining action research and process evaluation methods to develop and evaluate a toolkit for improving patient experience in the secondary care sector.

Methods:

An independent process evaluation was carried out of a HS&DR funded action research project in which a Patient Experience Toolkit (PET) was developed and implemented across six hospital wards in 3 NHS Trusts. Initially developed through co-design, the action research aimed to implement and refine the process contained within PET through collaborative working between the action researchers (2x) and the six ward teams involved in the study. The aim of the process evaluation was to generate generalisable research findings about the PET’s effectiveness that would aid its future scale-up. To that end, process evaluation methods (Moore et al, 2015) were adapted to make them suitable for interventions being developed on implementation through participatory research. A primarily inductive approach to data collection and analysis was adopted. A large qualitative dataset consisting of multiple sources was thematically analysed alongside the iterative development of a logic model, as is common in process evaluations. Research findings were fed back to the action researchers at the halfway stage and at the end-point to optimise the development and delivery of the intervention.

Results:

A process evaluation is a useful complement to action research, yielding important insights that may go ignored without one. In the study, PET’s role in the intervention was revised after it was found that the ward teams did not use the toolkit documentation and progress was only possible because of the facilitation provided by the action researchers. PET was therefore redesigned at the end of the project to serve as a guide for facilitators, who will use it while working with frontline staff on patient experience. While the action researchers realised this through their practical experience of working with the ward teams, the insights gleaned from the process evaluation were deemed useful because it provided ward teams with an independent channel to voice their opinions of the intervention. Furthermore, the process evaluation was able to track the facilitation in a way that the action researchers could not.

The process evaluation paid careful attention to interactions between the facilitation, each ward setting and study outcomes. The action researchers were found to adopt a range of strategies beyond the core activities contained within PET, in response to barriers and enablers present in each ward setting. These included, at the micro-level, flexible meetings dubbed “pop ins” for particularly busy environments and coaching strategies if staff lacked the belief that positive changes could be enacted. At the meso-level, escalation strategies could be pursued if improvements required buy-in from a different organisational actor or department. However, the effects of pressures on staff perceived as emanating from the macro-level could not always be modified on the wards, implying certain limits to the facilitation role and thus that wards may have to be pre-selected to provide a receptive context for the intervention. The final version of the logic model sought to capture these findings, displaying the fundamental importance of an adaptive facilitation role to the intervention and a list of barriers and enablers that have to be taken into account to successfully deliver PET. It is envisioned that the logic model will be used by future users of the PET to help them develop context-sensitive facilitation strategies or in the selection of wards for its delivery.

Implications:

While formative and participatory methodologies may be increasingly suitable as the complexity of healthcare problems and solutions increases, there remains a role for process evaluation in this emerging research paradigm. In particular, logic models can be used to display the insights of inductive, qualitative research in a way that may enhance the scale-up of interventions developed through participatory research. Process evaluators should use their independence and detachment to explore the interaction between participatory researchers (and the facilitation they provide) with delivery settings and study outcomes.

KeywordsProcess evaluation, logic models, action research, Quality Improvement
Year2019
Accepted author manuscript
License
File Access Level
Open
Publication dates
Print02 Jul 2019
Publication process dates
Deposited05 Dec 2022
Web address (URL) of conference proceedingshttps://hsruk.org/hsruk/viewpoints/hsruk-conference-2019-resources
Permalink -

https://openresearch.lsbu.ac.uk/item/92vw5

Download files


Accepted author manuscript
Mills HSRUK 2019 - YPET.pptx
License: CC BY 4.0
File access level: Open

  • 117
    total views
  • 20
    total downloads
  • 6
    views this month
  • 0
    downloads this month

Export as

Related outputs

Challenges to the real-world delivery of brief alcohol interventions in the custody suite: qualitative study
Jarrett, M., Mills, T., Mallion, J., Sykes, S., Wills, J. and Chaplin, E. (2024). Challenges to the real-world delivery of brief alcohol interventions in the custody suite: qualitative study. BJPsych bulletin. https://doi.org/doi:10.1192/bjb.2024.48
Involving Lived Experience in regional efforts to address gambling-related harms: going beyond ‘window dressing’ and ‘tick box exercises’
Jenkins, C., Mills, T., Grimes J, Bland C, Reavey, P., Wills, J. and Sykes, S. (2024). Involving Lived Experience in regional efforts to address gambling-related harms: going beyond ‘window dressing’ and ‘tick box exercises’. BMC Public Health. 24, p. 384. https://doi.org/10.21203/rs.3.rs-3658745/v1
Evaluation of a city-region initiative to galvanise a community response to gambling-related harms
Jenkins, C and Mills, T (2023). Evaluation of a city-region initiative to galvanise a community response to gambling-related harms. European Journal of Public Health. 33 (Supplement_2). https://doi.org/10.1093/eurpub/ckad160.676
Harnessing lived experience in a community-based intervention to address gambling-related harms
Jenkins, C, Mills, T, Reavey, P, Moss, A, Sykes, S, Wills, J and Grimes, J (2023). Harnessing lived experience in a community-based intervention to address gambling-related harms. European Journal of Public Health. 33 (Supplement_2). https://doi.org/10.1093/eurpub/ckad160.1654
‘Odds Are: They Win’: a disruptive messaging innovation for challenging harmful products and practices of the gambling industry
T. Mills, J. Grimes, E. Caddick, C.L. Jenkins, J. Evans, A. Moss, J. Wills and S. Sykes (2023). ‘Odds Are: They Win’: a disruptive messaging innovation for challenging harmful products and practices of the gambling industry. Public Health. 224, pp. 41-44. https://doi.org/10.1016/j.puhe.2023.08.009
Consultation skills development in general practice: findings from a qualitative study of newly recruited and more experienced clinical pharmacists during the COVID-19 pandemic.
Madden, M., Stewart, D., Mills, T. and McCambridge, J. (2023). Consultation skills development in general practice: findings from a qualitative study of newly recruited and more experienced clinical pharmacists during the COVID-19 pandemic. BMJ Open. 13 (4), p. e069017. https://doi.org/10.1136/bmjopen-2022-069017
Alcohol, the overlooked drug: clinical pharmacist perspectives on addressing alcohol in primary care.
Madden, M., Stewart, D., Mills, T. and McCambridge, J. (2023). Alcohol, the overlooked drug: clinical pharmacist perspectives on addressing alcohol in primary care. Addiction science & clinical practice. 18 (1), p. 22. https://doi.org/10.1186/s13722-023-00378-x
Development of a ‘real-world’ logic model through testing the feasibility of a complex healthcare intervention: the challenge of reconciling scalability and context-sensitivity
Mills, T., Shannon, R., O'Hara, J., Lawton, R. and Sheard, L. Development of a ‘real-world’ logic model through testing the feasibility of a complex healthcare intervention: the challenge of reconciling scalability and context-sensitivity. Evaluation. https://doi.org/10.1177/13563890211068869
Improving patient experience and safety at transitions of care through the Your Care Needs You (YCNY) intervention: a study protocol for a cluster randomised controlled feasibility trial
Baxter, R., Murray, J., O'Hara, J., Hewitt, C., Richardson, G., Cockayne, S., Sheard, L., Mills, T. and Lawton, R. (2022). Improving patient experience and safety at transitions of care through the Your Care Needs You (YCNY) intervention: a study protocol for a cluster randomised controlled feasibility trial. Pilot and Feasibility Studies. 6 (123). https://doi.org/10.1186/s40814-020-00655-5
Improving the safety and experience of transitions from hospital to home: a cluster randomised controlled feasibility trial of the ’Your Care Needs You’ intervention versus usual care
Baxter, R., Murray, J., Cockayne, S., Baird, K., Mandefield, L., Mills, T., Lawton, R., Hewitt, C., Richardson, G., Sheard, L. and O.Hara, J. (2022). Improving the safety and experience of transitions from hospital to home: a cluster randomised controlled feasibility trial of the ’Your Care Needs You’ intervention versus usual care. Pilot and Feasibility Studies. 8 (222). https://doi.org/10.1186/s40814-022-01180-3
A qualitative formative evaluation of a patient facing intervention to improve care transitions for older people moving from hospital to home
Shannon, R., Baxter, R., Hardicre, N., Mills, T., Murray, J., Lawton, R. and O'Hara, J. (2022). A qualitative formative evaluation of a patient facing intervention to improve care transitions for older people moving from hospital to home. Health Expectations. https://doi.org/10.1111/hex.13560
Integration of a clinical pharmacist workforce into newly forming primary care networks: a qualitatively driven, complex systems analysis
Mills, T., Madden, M., Stewart, D., Gough, B. and McCambridge, J. (2022). Integration of a clinical pharmacist workforce into newly forming primary care networks: a qualitatively driven, complex systems analysis. BMJ Open. 12 (11). https://doi.org/10.1136/bmjopen-2022-066025
Early implementation of the structured medication review in England: a qualitative study.
Madden, M., Mills, T., Atkin, K., Stewart, D. and McCambridge, J. (2022). Early implementation of the structured medication review in England: a qualitative study. The British journal of general practice : the journal of the Royal College of General Practitioners. https://doi.org/BJGP.2022.0014
Multiple meanings of resilience: Health professionals' experiences of a dual element training intervention designed to help them prepare for coping with error
Janes, G., Harrison, R., Johnson, J., Simms-Ellis, R., Mills, T. and Lawton, R. (2021). Multiple meanings of resilience: Health professionals' experiences of a dual element training intervention designed to help them prepare for coping with error. Journal of Evaluation in Clinical Practice. 28. https://doi.org/10.1111/jep.13555
Using qualitative process evaluation in the development of a complex intervention to advance person-centred practice by pharmacists: The Medicines and Alcohol Consultation (MAC)
Madden, M., Morris, S., Stewart, D., Atkin, K., Gough, B., Mills, T. and McCambridge, J. (2021). Using qualitative process evaluation in the development of a complex intervention to advance person-centred practice by pharmacists: The Medicines and Alcohol Consultation (MAC). Social Science and Medicine - Qualitative Research in Health. 1. https://doi.org/10.1016/j.ssmqr.2021.100012
The Association Between Health Care Staff Engagement and Patient Safety Outcomes: A Systematic Review and Meta-Analysis
Janes, G., Mills, T., Budworth, L., Johnson, J. and Lawton, R. (2021). The Association Between Health Care Staff Engagement and Patient Safety Outcomes: A Systematic Review and Meta-Analysis. Journal of Patient Safety. 17 (3), pp. 207-216. https://doi.org/10.1097/PTS.0000000000000807
Can we prepare healthcare professionals and students for involvement in stressful healthcare events? A mixed-methods evaluation of a resilience training intervention
Johnson, J., Simms-Ellis, R., Janes, G., Mills, T., Budworth, L., Atkinson, L. and Reema, H. (2020). Can we prepare healthcare professionals and students for involvement in stressful healthcare events? A mixed-methods evaluation of a resilience training intervention. BMC Health Services Research. 20. https://doi.org/10.1186/s12913-020-05948-2
Improving transitions from hospital to home: Using ‘real world’ logic models to derive learning from a feasibility trial of a ‘hybrid intervention’
Mills, T., Baxter, R., Shannon, R., Murray, J., Lawton, R. and O'Hara, J. (2020). Improving transitions from hospital to home: Using ‘real world’ logic models to derive learning from a feasibility trial of a ‘hybrid intervention’. Health Services Research UK.
Between interventions and systems: A proposal for ‘real world’ logic models
Mills, T. (2019). Between interventions and systems: A proposal for ‘real world’ logic models. Improving Patient Safety Conference. Oct 2019
Dialectical thinking and the 'economic determinants' of performance: What Marxism can contribute to health services research in an age of complexity
Mills, T. (2019). Dialectical thinking and the 'economic determinants' of performance: What Marxism can contribute to health services research in an age of complexity. British Sociological Association: Medical Sociology.
National Patient Safety Alerts: An Exploration of How Trusts Coordinate their Responses
Mills, T., Atkinson, L., Janes, G. and Lawton, R. (2019). National Patient Safety Alerts: An Exploration of How Trusts Coordinate their Responses. NHS.
Using patient experience data to develop a patient experience toolkit to improve hospital care: a mixed-methods study
Sheard, L., Marsh, C., Mills, T., Peacock, R., Langley, J., Partridge, R., Gwilt, I. and Lawton, R. (2019). Using patient experience data to develop a patient experience toolkit to improve hospital care: a mixed-methods study. NIHR, Health Services and Delivery Research . https://doi.org/10.3310/hsdr07360
Steering, Knowledge and the Challenge of Governance Evaluation: The Case of National Health Service Governance and Reform in England
Greenwood, D and Mills, T. (2019). Steering, Knowledge and the Challenge of Governance Evaluation: The Case of National Health Service Governance and Reform in England. Administration and Society. 52 (7). https://doi.org/10.1177/0095399719883564
Improving Patient Experience in Hospital Settings: Assessing the Role of Toolkits and Action Research through a Process Evaluation of a Complex Intervention
Mills, T., Lawton, R. and Sheard, L. (2019). Improving Patient Experience in Hospital Settings: Assessing the Role of Toolkits and Action Research through a Process Evaluation of a Complex Intervention. Qualitative Health Research. 29 (14), pp. 2108 - 2118. https://doi.org/10.1177/1049732319855960
Advancing Complexity Theory in Health Services Research: The Logic of Logic Models
Mills, T., Lawton, R. and Sheard, L. (2019). Advancing Complexity Theory in Health Services Research: The Logic of Logic Models. BMC Medical Research Methodology. 19 (55). https://doi.org/10.1186/s12874-019-0701-4
Advancing Complexity Science in Healthcare Research: The Logic of Logic Models
Mills, T., Lawton, R. and Sheard, L. (2018). Advancing Complexity Science in Healthcare Research: The Logic of Logic Models. Health Services Research UK.
Exploring the Economic Dimensions of Complexity: A Post-Positivist Evaluation of Diabetes Policy and Governance
Mills, T. (2017). Exploring the Economic Dimensions of Complexity: A Post-Positivist Evaluation of Diabetes Policy and Governance. "We Need to Talk About Complexity": Oxford University Talks. Green Templeton College, University of Oxford 13 - 14 Jun 2017
Losing Control Again? Power and the quality of working life
Mills, T. and Overell, S. (2010). Losing Control Again? Power and the quality of working life. The Work Foundation.