What is insight infrastructure?

We are working on a project with the Joseph Rowntree Foundation to help them understand the ecosystem around their work to develop an insight infrastructure for social and economic inequalities, and how to engage with it.

One of the things that’s been a bit challenging to pin down is how to think about what an “insight infrastructure” actually is or does. Is it just a fancy name for what’s essentially a data portal? Or the long term development of system-wide change? Is it intended to be a set of services that JRF will provide? Or a shared movement they’re building? I’ve been drawing on previous work to provide a couple of ways of thinking about what insight and insight infrastructures actually are.

Call for input: workshops

Does your work address poverty, economic or social inequality in the UK? What difference would it make if you had more granular or real-time data? What lived experience insights could help your planning or practice? Which communities might be affected most?

Register here to take part in our upcoming workshops in March and April.

What is insight?

The first question I had was what is meant by “insight” and how this differs from data, or analytics. I’ve recently been involved with a project with Public Digital around “data as a service”, where we’re drawing the distinction between data services (that provide computer-consumable data) and analytics services (which provide human-consumable information).

Within this framework (based on the classic DIKW pyramid), I’d characterise an insight service as going beyond providing information and towards providing knowledge. I’d interpret that as information oriented towards action: perhaps there’d be a bit of automated prediction and recommendations built in, and perhaps more qualitative information, such as accounts of lived experience, to make it come alive (and be persuasive).

So, for example, a data service might provide nicely structured machine readable data about taxes and benefits; an analytics service might turn that raw data into human-readable tables, graphs or textual summaries; but an insight service would bring this together to rapidly show the impact of a new budget on typical or target households, alongside a few real-world examples and stories of people and families affected by the changes.

Plainly this data/information/knowledge distinction is fuzzy around the edges, and more of a spectrum than distinct levels, but it’s useful to be explicit about what the target is. There are different kinds of audiences for data, information and knowledge. And at each level, the organisation providing the service is layering more interpretation (and therefore more opinion and bias) into the mix. People receiving insight may want to question it, and dig back through the layers of analytics to understand the spin that’s been put on the figures, and perhaps to create their own alternative interpretations using the same raw data.

What is insight infrastructure?

When we developed the idea of data infrastructure at the Open Data Institute, we thought a lot about what that infrastructure consists of. Leigh and Peter wrote it all up back in 2019, in the State of Open Data, where they described data infrastructure as comprising:

• Data assets, such as datasets, identifiers, and registers.
• Standards and technologies used to curate and provide access to data assets.
• Guidance and policies that inform the use and management of data assets and the data infrastructure itself.
• Organisations that govern the data infrastructure [what we came to call data institutions]
• The communities involved in contributing to or maintaining it, and those who are impacted by decisions that are made using it.

Taking this breakdown as inspiration, an insight infrastructure might be described as a similar mix of things:

• A data infrastructure on which insights can be built.
• Models and other technologies that provide ways of understanding and interpreting that data.
• Guidance, policies and governance that inform the use of those models and the insight they generate.
• Organisations that govern, steward and provide the insight infrastructure.
• The people and communities involved in contributing to and maintaining the insight infrastructure, and those who are impacted by decisions that are made using it.

These infrastructures are socio-technical systems. When we talked about data infrastructure at ODI, we wanted to shift a bit of the focus away from the technical aspects of data infrastructure (the data itself), and towards the social parts (the standards, governance, organisations, and communities). (I’m classing standards as social because they involve coming to agreements, an essentially social process.)

We frequently used the analogy of road infrastructure. Road infrastructure isn’t just about the roads themselves, it’s also about the rules of the road in the Highway Code and the whole system of driving tests and licensing. It’s about Highways England and local authorities having appropriate powers, responsibilities and resources to keep them maintained. And it’s about working with the cities, towns, villages and neighbourhoods that the roads go through (think about the role of communities in creating Low Traffic Neighbourhoods, for example).

Our focus on the social side of infrastructure was partly to try to redirect investment and attention. We had seen many examples of data platforms being built that had minimal use, not because the platform wasn’t designed well, but because the social parts of the infrastructure weren’t there, so the data wasn’t trusted, or people lacked motivation or capability to use it, or the data they provided fell into disrepair as organisational priorities changed. It’s interesting reading Tim’s reflections on the journey that Land Portal has been on, as an infrastructure provider, as they seem to have gradually shifted more and more towards a focus on this social side of the equation.

The Joseph Rowntree Foundation’s work on an insight infrastructure for poverty has a number of strands to it, including those that are focused on the technical side – creating new data sources and insight services or platforms, for example. Our part of the work is really on developing the social side. Who is / should be / will be involved in this infrastructure? What roles do they play? How do they come to agreements?

It’s really important to us at CONNECTED BY DATA, but also to JRF, that the people and communities who are actually going to be affected by it have a powerful say in shaping it. We don’t just care about user needs for insight services, but about the needs and expectations of the people who have data gathered about them, and those who are affected by the decisions that the data, analytics and insight informs.

To find out more about JRF’s ambitions around analysis and insight infrastructure, read Rosario’s piece on missing data and Graeme’s piece on enhancing insight into poverty. Do also email Rosario if you’d like to learn more.

For our part of the work, we’re looking for other organisations with experience developing, using, and being affected by both insight services and insight infrastructures. We’re also looking for people and organisations who already work to tackle social and economic injustices, whether or not you currently use data or insights. Please do get in touch if you have any thoughts you’d like to share.