Measuring or evaluating the impact of involving public or citizens throughout a research study, should involve asking two questions. How did the public involvement impact the research? How did it impact the people involved. This not only includes the members of the public, but also the researchers. Evaluating impact is often seen as the most contested and complex aspects of public involvement. As a result, is often ignored or poorly done, but it doesn't have to be this way. The most important thing for evaluating impact is being clear from the beginning about why the public being involved and what outcome everyone involved hopes to see. With this in mind, you can co-produce a personalized approach to measure the impact of your chosen activities together with your stakeholders. You may choose a metric approach, a narrative approach or a mix of the two. Really there is no right or wrong way of measuring impact, it's what's most appropriate for your project. The people doing the evaluation and the impact you're looking for. However, there are some existing tools that can be adopted for your own tailored approach. Don't worry, I'll expand on some of these in this talk. Before expanding on approaches measuring the impact of public or citizen involvement, I first want to explain some reasons why it's important. If public contributors have given their time and energy to supporting a project, it's important to acknowledge this. Demonstrating how they've made a difference will make them feel valued and motivated to continue. A second reason, it is important to demonstrate to funders that you've achieved the outcomes you said you would. Reflecting critically on your public involvement approach helps you understand what works and what doesn't. This means you can improve the quality of your public involvement work over time. Finally, by sharing and reporting your learnings, you may encourage others to follow your lead and enable them to build on your experience. So those are some reasons as to why you evaluate the impact of public involvement, but what do we mean by impact? The National Institute of Health Research defines impact as the changes, benefits and learning gained from insights and experience of patients, carers and the public when working in partnership with researchers and others involved in NIHR initiatives. But really, impact is for you to define ideally before the involvement begins. The changes, benefits and learnings that you choose to measure should relate to both the aims of the research project and what's important to all those involved. In other words, not only the researchers, but the public partners too. In fact, some of the objectives could be led by the public. You should also consider who is setting the agenda or who is the evaluation for. Consider whether you're evaluating the process, the outcomes or both. Of course, sometimes it's hard to know exactly what made a difference throughout the life of a project. That's why it's good practice to keep track of the comments and suggestions that are provided over time, along with a record of what happened, what was or wasn't acted upon and why. Some people call this a public involvement look, such as this one by the University of the West of England. They encourage citizens, researchers and facilitators to complete throughout the project. When such logs are brought together at the end, this approach can help everyone reflect on the involvement that has taken place. They can also identify the positive and negative impacts and whether they were expected or unexpected. Another well-known and comprehensive tool for evaluating public involvement is PiiAF, or the Public Involvement Impact Assessment Framework. PiiAF is a two-part practical and visual guide outlining the principles of impact evaluation, along with guidance for assessment, plan, development. It was published in 2014 to help researchers in health or social care. It may seem intimidating, but in fact it's purely meant to act as a framework allowing you to pick and choose the elements that are most appropriate to your project. You will find a practical example in the course resources. Finally, while not an evaluation tool, the UKs national standards for public involvement aims to provide people with clear, concise benchmarks for effective public involvement. It was developed through multi-stakeholder consultation and sets out six core quality standards, one of which is impact. I highly recommend you refer to this useful framework to ensure that your own approach is effective. Whichever method you adopt, make sure that you take the time to reflect on and learn from the findings of your evaluation, and even better, share your experience and any impactful findings with both those involved and others in the field. Many journals now expect people to report within their manuscripts, how the public were involved in the research and the impact it had. One tool that you may find useful for this is called GRIPP2, which you can read more about in the course resources. So to conclude, there is no right or wrong way of measuring impact. It's what's most appropriate for your project, the people involved in the project and the impact that you're looking for. Hopefully you've gained insight into some of the ways to plan for and capture the impact of people's involvement, for example through an involvement log. You could also use feedback forms after individual activities or run interim workshops to reflect on how well you're meeting aims and what could be improved. This can all feed into a final impact assessment. But the key thing to remember is you need to map out your evaluation plans and approach at the start of the project. These should be carried out as a joint exercise with all key stakeholders involved. The earlier you plan, the easier it will be and the more likely you are to achieve your public involvement objectives. Once finished, share what you've learned and use that to improve future public involvement activities.
