Getting the best from a public advisory panel

Recently I’ve had the privilege of convening a couple of public advisory panels for large-scale research and evaluation programmes, including the Connected by Data Public Voice in AI (PVAI) People’s Advisory Panel. In both cases the panels led to tangible improvements in the programmes they were advising, but running these sessions was not without its challenges. Here I share some reflections on how to work well with a public advisory panel.

1. Choose the right participants

Public advisory panels work best when they include people with lived experience relevant to the project being advised. For the Public Voices in AI Public Advisory Panel we recruited members of the public who had already participated in at least one public engagement activity related to AI. As such, they had already spent some time discussing AI and establishing their own views. These formed the foundation of their contributions and gave them confidence to participate. They were also well-networked and able to engage with family, friends, co-workers and customers to bring a wider range of voices into our discussions.

2. Allow time for settling in

For people who do not work on research projects day to day, the idea of participating in research can be somewhat novel - the idea of taking part in a panel about research (one step removed) can be even more mind-blowing. ‘We want your opinions on what we should ask other people, but don’t need to hear your actual opinions on the topic itself’ can be a difficult one to get your head around. It takes time. It’s also a good investment to allow time for panel members to get to know each other and their positions on key topics, even though that’s not necessarily a required output, as it helps the group to build bonds and work better together. Ideally this would be face to face.

3. Start as early as possible and ensure strong links to project delivery

Easier said than done, but often advisory groups can find themselves up and running too late to impact early design decisions that shape the programme they are advising. This can be a cause for frustration and can undermine confidence that the group will be able to impact. The PVAI programme did well to adapt to feedback where they could, but the Panel felt they could have added more value if they had been involved in designing the work packages themselves. Similarly, it can be challenging to ensure the Panel were advising at the best time – early enough to have impact, late enough to have something concrete to respond to. This is especially relevant when a project is fast-moving, but can potentially be mitigated by including the Panel convener in scheduled project management meetings.

4. Ensure the questions are the right ones

It can be really easy to slip into asking questions such as ‘how should we engage the public’ but that isn’t necessarily suited to the panel’s expertise – they are not social researchers. Just as we wouldn’t necessarily expect them to be experts in AI, they are also not experts in public engagement. But what they can share is where you might find them or people they know, what makes something appealing to be a part of, and what to avoid. Similarly, they can be helpful for identifying gaps or issues with a sampling strategy or discussion guide, ensuring that you’re asking relevant and meaningful questions.

5. Find innovative ways to engage

We used the PVAI Public Advisory Panel to innovate with the use of AI during our sessions. This supported Panel member’s learning and enabled us to learn about the strengths and weaknesses of AI during delivery.

Using AI avatars as speakers

We created videos of lifelike AI avatars to explain the different roles involved in developing AI: this helped us share information in an engaging format while not needing to secure 6 speakers for our session at short notice. We could also manage exactly what they said and how long they spoke for. It was a bit glitchy but the Panel enjoyed it.

Using AI for analysis

We also used AI to do a write-up of one of our sessions, based on an AI generated transcript. Although it was broadly ok, there appeared to be some hallucinations and unnecessary deviations away from the language used in the sessions. It also struggled to write a detailed summary, preferring to write just one or two pages which did not have the level of detail required to evidence the points made.

Despite the challenges, overall if there is a commitment to genuinely working together with a public advisory group then it can be very worthwhile. Keeping a record of ‘you said, we did’ can help create a sense of trust and momentum, keeping panel members engaged over months or years. From identifying missing voices, to uncovering research team’s unconscious bias, a public or lived experience advisory group can help rebalance power and ensure that research programmes address the issues that people care about.