Discovering what works - thinking about effective insight infrastructures

This week we had the first of our workshop sessions to explore the potential ecosystem around, and stakeholder engagement in, Joseph Rowntree Foundation (JRF) plans to develop an Insight Infrastructure on poverty in the UK.

Below I’ve written up an initial synthesis of the session.

Taking a long view

As we started introductions, one participant reflected on how good it was to see JRF looking at renewed investment in this area, particularly given the long history of the Foundation in sparking and supporting efforts to improve data and evidence on poverty, going back to the 1990s. This comment led me to look more at the history of JRF, nicely captured in this archive record, which reports how, when the modern Foundation adopted a new approach in the late 1980s the new Director:

“saw an immediate need to make the results of the JRF’s broad range of research more readily available to policy makers, the press and the public, and in 1989 Roland Hurst, the Trust’s first Director of Information Services, was appointed to manage the dissemination of the Trust’s research output through new publications, specialist briefings and a closer relationship with the media. As part of his work he launched the ‘Findings’, as well as ‘Search’, a more detailed quarterly magazine.”

This provides a useful context to this latest iteration of JRF action, which is looking not only to provide better access to JRF generated insight, but also to explore how JRF can contribute to better access to insight from, and to, a wide range of stakeholders.

Insight and Infrastructure

We started by thinking about the kinds of insights participants use everyday: from statistical reports and public health profiles of local areas, to web analytics indicating trends, composite indices and ‘risk maps’, crowdsourced community data, and lived experience shared through regular calls. We asked what makes these insights useful, surfacing issues around the granularity of insight, having information at your fingertips, and having conversations with colleagues to draw out the narratives from within available evidence, and to contextualise data with knowledge and experience.

This provided a key starting point for discussion. Insight is more than data. And an insight infrastructure is something that enables greater insight to be generated, discovered and shared to the right people, at the right time.

What works?

Because the concept of an insight infrastructure is (intentionally at this stage of JRF thinking) underdefined, we turned to three examples (surfaced through our earlier stakeholder mapping) of platforms that might be thought of as kinds of insight infrastructure, and used these to spark discussion about the features and aspects that make for effective insight sharing.

In the examples given we found cases of overview content that offered an introduction to complex topics, and provided ways for those interested to dig deeper through data and visualisations. We found platforms providing access to a broad range of data, and offering contact details of the data stewards responsible for it. And we found a range of effective interfaces that combined punchy narratives with interactive charts, and platforms that incorporated video presentations of lived experience alongside statistical tables and charts.

However, we also found a lot of questions and issues thrown up by the examples we looked at, including concerns that platforms were driven by the data available, rather than by the data needed, or that platforms were presenting estimated or inaccurate data without clear caveats or guidance to users. Workshop participants observed that while insight need and impact often comes from hyperlocal interests, the barriers to entry for accessing relevant insight can be too high, and presentation of data at too high a level, with the granularity too low.

The different needs of different users came out strongly in our discussions. While some users want to go straight to raw data, others want easy-to-use visualisations which generate graphics to drop into reports, and others are looking for pre-curated key insights. In the examples we explored, it was not always clear the kind of user the platform intended to serve.

One of the key take-aways from this part of the discussion can be summed up as a call for a ‘questions first’ approach. As one participant put it:

“With all these tools, it feels like the real challenge comes before you even get to the infrastructure. How do you translate your challenge into a question? Many organisations we talk to don’t engage with these kinds of tools much, if at all, because they get overwhelmed. There’s 15 datasets that could be relevant. How do they choose the right one? Yet, no one is providing support with problem framing or filtering datasets because it’s insanely resource intensive and requires a deep knowledge of the organisation’s mission and challenges.”

Who needs what?

Key to JRF’s plans is a recognition that an insight infrastructure will have many different stakeholders. Perhaps unlike work to “make the results of the JRF’s broad range of research more readily available to policy makers, the press and the public” back in the early 90s, which may have adopted a focus on these as ‘audiences’, the current work needs to consider how different stakeholders are more active participants in the creation, maintenance and use of the infrastructure, and how they will have different interests and needs at different points.

Using a set of themes drawn from the What Works activity (data & evidence, interfaces, analysis, governance & trust and purpose), we looked at what users, contributors, affected communities, and stewards of an infrastructure might want from each.

For example, users may need evidence not only about the scale of poverty challenges, but also information about who else is working on a particular issue, and where? And when it comes to the analysis of data, beyond provision of tables or charts, users may be seeking to answer the question “So what? What should I do (or not do) as a result of this evidence?”. For some, this is about access to storytelling, and for others, it might involve being able to extract analysis easily, or integrate live analysis into other systems.

Support for clear data licensing and attribution was raised as a key theme for contributors to an infrastructure, as were questions of incentives for participating in shared platforms, and the need for clarity, as a contributor, on the kinds of decisions that data or evidence might impact, so contributors can think about how to present it effectively. This points to a potentially important role of an infrastructure in supporting a feedback loop between insight ‘producers’ and ‘consumers’.

Comments against the ‘affected communities’ stakeholder category in our canvas highlight that such a feedback loop should also include those described within, or affected by, particular data and insights, as a number of workshop contributors called for affected communities to have ability to shape both data, and how it is analysed. One contribution also suggested the possibility of a ‘code of conduct’ or agreement that would bind users and stakeholders of the infrastructure to consider affected communities: much as review processes in some academic disciplines ask for consideration of issues of diversity and gender.

A theme that emerged in discussion of both direct platform users, and affected communities, was the education role of an insight infrastructure: helping stakeholders consider the kinds of evidence needed to create certain kinds of change, and building literacy on how to engage with, question, and take a role in the governance of, key data.

Those stewarding an infrastructure also have their own needs: from knowing that the work they are doing is not duplicative, to understanding the gaps that need to be filled, and better tracing the connection between data and decisions. Notes on this track also suggest value if an infrastructure can build on and promote shared data standards, and provide shared information architectures.

Advice and insight

The session closed with a round of advice for JRF on things to do, things to avoid, and things to consider, in the next stages of the Insight Infrastructure project.

Fortunately, the most common suggestion, speak to affected communities, is core on the agenda for upcoming stages of our work, as we move from conversations with ‘infrastructure experts’, to talking with more frontline organisations, and towards engaging with communities directly affected by the use of poverty-related insight.

There was also welcome feedback on the concept of infrastructure, highlighting the need to find more accessible language for public communication about the project, at the same time as diving more into the infrastructure analogy, with the question “Do we need more roads or better drivers?” hinting at a need to think about data and insight literacies and capabilities within the project’s development.

Next steps

Soon after the last participant left the workshop zoom room, we started on preparation for Workshop 2 and 3, the ‘Dream’ sessions to follow on from this first ‘Discovery’ meeting. In these sessions we’ll be looking ahead and thinking in more detail about what JRF’s insight interventions might be, and how different groups would want to engage with this.

These sessions are taking place on 6th March (in London), and 7th March (Online), and we’re looking for organisations involved in poverty action to take part.

If you are interested in coming along, please register your interst here get in touch with obioma@connectedbydata.org for more details and to secure a place.