Developing, implementing and evaluating the Yorkshire Patient Experience Toolkit (PET): How process evaluation can strengthen action research

Background:

Formative and participatory research and improvement methodologies are increasingly advocated in the field of intervention research, in recognition of the complexity of both healthcare problems and the interventions required to solve them (Andersson, 2018). One such approach is action research, which is said to be uniquely capable of developing solutions for complex problems through relational working and the utilisation of locally-situated knowledge (Moch et al, 2016). However, this participatory shift poses the question of how research findings can be scaled-up for transfer into actual healthcare practice. In the case of action research, for example, successful “action” within a healthcare setting can come at the expense of rigorous research (Hammersley, 2004). Here, we report on an innovative study which sought to grapple with this problem of generalisation, combining action research and process evaluation methods to develop and evaluate a toolkit for improving patient experience in the secondary care sector.

Methods:

An independent process evaluation was carried out of a HS&DR funded action research project in which a Patient Experience Toolkit (PET) was developed and implemented across six hospital wards in 3 NHS Trusts. Initially developed through co-design, the action research aimed to implement and refine the process contained within PET through collaborative working between the action researchers (2x) and the six ward teams involved in the study. The aim of the process evaluation was to generate generalisable research findings about the PET’s effectiveness that would aid its future scale-up. To that end, process evaluation methods (Moore et al, 2015) were adapted to make them suitable for interventions being developed on implementation through participatory research. A primarily inductive approach to data collection and analysis was adopted. A large qualitative dataset consisting of multiple sources was thematically analysed alongside the iterative development of a logic model, as is common in process evaluations. Research findings were fed back to the action researchers at the halfway stage and at the end-point to optimise the development and delivery of the intervention.

Results:

A process evaluation is a useful complement to action research, yielding important insights that may go ignored without one. In the study, PET’s role in the intervention was revised after it was found that the ward teams did not use the toolkit documentation and progress was only possible because of the facilitation provided by the action researchers. PET was therefore redesigned at the end of the project to serve as a guide for facilitators, who will use it while working with frontline staff on patient experience. While the action researchers realised this through their practical experience of working with the ward teams, the insights gleaned from the process evaluation were deemed useful because it provided ward teams with an independent channel to voice their opinions of the intervention. Furthermore, the process evaluation was able to track the facilitation in a way that the action researchers could not.

The process evaluation paid careful attention to interactions between the facilitation, each ward setting and study outcomes. The action researchers were found to adopt a range of strategies beyond the core activities contained within PET, in response to barriers and enablers present in each ward setting. These included, at the micro-level, flexible meetings dubbed “pop ins” for particularly busy environments and coaching strategies if staff lacked the belief that positive changes could be enacted. At the meso-level, escalation strategies could be pursued if improvements required buy-in from a different organisational actor or department. However, the effects of pressures on staff perceived as emanating from the macro-level could not always be modified on the wards, implying certain limits to the facilitation role and thus that wards may have to be pre-selected to provide a receptive context for the intervention. The final version of the logic model sought to capture these findings, displaying the fundamental importance of an adaptive facilitation role to the intervention and a list of barriers and enablers that have to be taken into account to successfully deliver PET. It is envisioned that the logic model will be used by future users of the PET to help them develop context-sensitive facilitation strategies or in the selection of wards for its delivery.

Implications:

While formative and participatory methodologies may be increasingly suitable as the complexity of healthcare problems and solutions increases, there remains a role for process evaluation in this emerging research paradigm. In particular, logic models can be used to display the insights of inductive, qualitative research in a way that may enhance the scale-up of interventions developed through participatory research. Process evaluators should use their independence and detachment to explore the interaction between participatory researchers (and the facilitation they provide) with delivery settings and study outcomes.