Why does so much good analysis change so little? 

In October, we launched our first piece of original research into UK higher education. As is fairly standard, we arranged a webinar to reveal some of our findings and I opened it with the obvious question: does the sector really needed another report. 

The honest answer is probably not. 

Higher education does not have an evidence shortage. The Office for Students publishes the financial picture. Jisc does genuinely good work on digital maturity and technology legacy. UCISA maps the systems landscape. Uniforum gives institutions deep operatioenal benchmarking. Universities UK pushes the sector on efficiency and collaboration. Gartner and a long list of others add more. The evidence is there, it is largely good, and more of it arrives every quarter. 

But, using higher education as one example, what the sector has is not an evidence problem. It is an evidence-to-action problem, and I think our profession is part of the reason it persists. 

The deliverable is not the document 

Evidence is only useful when it changes something. A decision, an investment, a priority, a process, a forecast, a service that gets better for the people using it. Something has to be different afterwards. 

When it does not, what we have produced is another PowerPoint. Another PDF. Another well-argued document that gets read, agreed with, and filed. 

I say this as someone who has written a great many of them. That is the uncomfortable part. Our industry is extremely good at the analysis step and much less reliably good at the step where analysis becomes a decision somebody is accountable for. We have professionalised the production of insight. We have not professionalised what happens to it on the Monday after the steering group. 

I do not think the answer is better reports. I think it is a harder question asked earlier. 

Stop asking what people believe 

When we surveyed 250 senior higher education leaders, via Censuswide, we asked them to assess their own institutions across 27 statements. The findings gave us a lot to think about.

Ask a leadership team whether they track benefits and they will say yes. Ask whether they make evidence-led decisions and they will say yes. Ask whether their programmes are properly governed, whether they have trusted data, whether they are good at transformation, and the answer is yes every time. Not because anybody is being dishonest, but because that is what the question invites. 

You learn almost nothing from a question somebody can agree with comfortably. 

The alternative is not a cleverer question. It is a different verb. Do not ask whether benefits are tracked; ask to be shown a benefit twelve months after go-live. Do not ask whether decisions are evidence-led; ask to be shown something that was stopped because the evidence changed. Do not ask whether the data is trusted; ask where finance, technology and operations disagree about what it says. Do not ask whether an organisation is transforming; ask what is actually different. 

Show me is a more useful instrument than tell me, and it is available to every consultant in every engagement without a survey, a benchmark or a maturity model. 

Restructuring changes the cost base. Transformation changes the economics. 

This matters because of what the word transformation has come to cover. 

Headcount reductions, course closures, recruitment freezes, portfolio reviews, estate rationalisation, delayed investment, structural change. In a financially pressured organisation much of that is necessary, and I am not arguing against any of it. I am arguing against calling it something it is not. 

The distinction is simple. When you restructure you ask where cost can be taken out. When you transform you ask why that cost exists in the first place. Those are different questions and they produce different results. 

If a hundred people become eighty, and the eighty are still re-keying the same data, reconciling the same spreadsheets, working around the same systems and handling the same exceptions, then a saving has been delivered and nothing has been transformed. There are simply fewer people doing the same badly designed work under more pressure. 

There is a nastier version of this. When money is tight, the things that get cut are often training, change capability, data capability, programme capacity and specialist skills. In other words, the capability required to transform. Six months later the organisation finds it cannot do the work it needs to do, and buys that capability back at a premium. That is not a saving. That is cost moving house. 

Technology behaves the same way. Defer the platform replacement and the old platform is not free: the integrations still need maintaining, the specialists still need keeping, the manual work continues. Technology debt becomes financial debt, just later and in a different budget line. 

Disagreement is data 

The most useful finding in our research was not a number. It was a divergence. 

Asked whether benefits continue to be tracked after technology goes live, 93% of chief information officers said yes. Among chief financial officers and finance directors, 57% did. It is a small base and I would not over-read it. But it raises a question worth sitting with: what if the people delivering transformation and the people signing off its economics are working from different versions of reality? 

I have no problem with leadership teams disagreeing. If everyone around an executive table agrees about everything, I would be worried, and probably bored. The problem is not disagreement. It is hidden disagreement. 

Picture the familiar version. Technology reports the programme delivered. Finance says the benefit has not landed. Operations says the process is still painful. The programme team declares go-live and moves on. Four functions report separately, the papers get consolidated, and what reaches the board is a consensus that nobody in the building actually holds. We take the most useful information in the organisation and average it away. 

The opposite is better. Finance and technology disagree about the benefit? Good — show me. Operations think the process got worse? Show me. The executive and the board see the risk differently? Show me, and let us understand why rather than smoothing it over. 

Alignment does not mean everyone agrees. It means everyone is looking at the same evidence, knows where they differ, understands why, and has decided what to do about it. 

What this looks like in a board paper 

If I were sitting on a governing body with a forecast in front of me, I would ask five things. 

  1. What happens if the plan is missed, and what decision is triggered when it is? A downside scenario without a trigger is a spreadsheet, not a plan. 
  1. Which commitments become irreversible in the next twelve months? Resilience is not only about money. It is about options, and the biggest risk is often not the wrong decision but waiting until there are no good ones left. 
  1. What has structurally changed, rather than simply reduced? Which processes disappeared, which systems were switched off, which service improved. Transformation leaves evidence behind. 
  1. Where do finance, technology and operations disagree about our resilience? Not the average. The disagreement. 
  1. And what evidence would prove this plan wrong? Most governance is designed to reassure. Genuinely resilient organisations go looking for evidence that they might be wrong. 

Don’t approve the confidence 

None of this is specific to higher education, of course. I used that sector because it is where we did the work and because the financial pressure there is acute enough to make the pattern visible. We’re running the same process with the NHS sector. But every organisation we work in has the same gap between what it believes about itself and what it can demonstrate, and our profession is often the last group of people in the room with the standing to say so. 

Our role is not to add another layer of evidence to a pile that is already deep. It is to connect the evidence that exists, find the disagreements inside it, challenge the assumptions underneath it, and then prove that something changed. 

So the question I would take into your next engagement is not what score a client would get. It is this: what do they believe about themselves that they cannot currently prove? 

Confidence without challenge becomes reassurance. Reassurance delays difficult decisions. Delayed decisions reduce options. Everyone has a plan — the question is whether they are prepared to change it when reality tells them to. 

Don’t approve the confidence. Approve the evidence. 

Dave King
Dave King
Articles: 16