Evaluation of the PPI process
Strengthening the science underpinning PPI is key to ensuring that it becomes a robustly conducted and properly resourced aspect of research. Greater transparency, reflection, and scrutiny of the involvement process is essential for continued learning and development of PPI.
There are many tools for the evaluation of PPI. However, many of these tools are not co-designed, nor are they tested for reliability ((opens in a new window)learn more).
Evaluation need not be complicated to be useful. Simple reflection and feedback between PPI partners and researchers can improve the involvement process and influence future practice.
Evidencing Impact and Evaluation
Watch back this seminar on evaluating PPI and evidencing impact.
This seminar was held as part of the UCD-RCSI Seminar Series on PPI in preclinical research. In this seminar, the focus is on evaluation, assessment and evidencing impact of PPI. There is a general overview of evaluation and impact for PPI generally, before a specific presentation on the PPI assessment survey (see below).
What is being evaluated?
In any evaluation, you need to define what you are evaluating. Are you interested in the PPI process, or in the effect of PPI on the research? These are different questions that require different approaches.
If you are in evaluating the effect of PPI then you need to define:
- Effect on what- research quality? relevance? research outcome?
- Effect for whom- the researcher? the PPI partner? the research institution? wider society?
Who is evaluating?
- Is your evaluation co-designed and/or co-conducted?
- Is everyone clear on the goal(s) of evaluation?
Key principles for PPI evaluation
PPI evaluation should be grounded in key principles. (opens in a new window)Boivin's work argues for six key evaluation principles.
- Clarity of goals and definitions
- Reflective about unspoken values and power imbalance underpinning PPI and its evaluation
- Rigor in defining and measuring high quality PPI, how it is reported and assessed
- Transparency to enable distinguishing between ineffective PPI (theory failure) & inappropriate resourcing or implementation (practice failure)
- Pragmatism to ensure that evaluation efforts are not paralysed by a misguided perception that PPI is too complex to be evaluated
- Reciprocity for PPI to become a truly joint enterprise where PPI partner have an equal footing with other stakeholders in research and its evaluation
PPI Assessment Survey
The PPI Assessment Survey (PAS) is a (opens in a new window)validated, co-designed and easy to use questionaire. It was developed as a simple way to help researchers minimise or avoid risk throughout the PPI process.
The goal of using the PAS is that unsatisfactory processes (from the PPI partner point of view) can be identified and addressed resulting in a more robust and meaningful PPI process.
The survey is designed to be completed by PPI partners at multiple stages in the lifecycle of the research project.
This iterative approach has a number of benefits including:
- identifying risks to the PPI relationship
- identification and accommodation of changing requirements throughout the research lifecycle
- Helps identify the need for refinement and improvement of PPI strategies
- Helps identifies if changes or adaptions to the process were successful
The PAS also has an (opens in a new window)associated tool that will automatically analysis and report the survey results over the project lifecycle.
Capturing Impact
Broadly speaking, when we talk about research impact we mean the effect that research has byond academia. Often, this is societal and economic impact and is the demonstrable contribution research makes to society and the economy, of benefit to individuals, organisations and nations.
How to you capture evidence of research impact? UCD has a dedicated Impact Toolkit that will take you through the entire process. Explore how you can plan, capture, communicate and monitor the impact of research here.