Numbers speak for themselves. Or do they?
A conversation with Berend van der Kolk on the darker side of measurement
We measure everything with gusto: performance, customer satisfaction, publications, even our fitness via a running app. Almost nothing escapes being reduced to a number these days. Berend van der Kolk, associate professor specialising in performance measurement, described in The Quantified Society what happens when people and organisations try to capture everything in figures. 'An indicator shows green. But dig a little deeper, and it turns out to be bright red underneath.' What starts as an innocent figure, says Van der Kolk, can quickly take on a life of its own.
Author: Edo Groot
Why is measuring so tempting? According to Van der Kolk, simply because it offers a sense of grip. 'You can compare, spot patterns, track progress. That's true privately -think of a running app like Strava - but certainly also within organisations. The higher up you go, the less you can know about every detail, so the more important figures become for maintaining a sense of control.' Our rector, he illustrates, simply cannot know what's happening in every lecture hall. That's not necessarily a problem, sometimes it's even necessary. The trouble starts when you no longer realise how many assumptions are hidden within such a figure. 'You think you're dealing with objective, hard data, while the subjective choices that precede the production of that data often remain invisible. And that can lead to poor decisions.'
7
If you unquestioningly assume that quality equates to a place on a league table, you no longer really need to think about it yourself, you simply copy someone else's definition. But the moment you let go of that, you suddenly have to have the conversation together: what actually is quality?
If you unquestioningly assume that quality equates to a place on a league table, you no longer really need to think about it yourself, you simply copy someone else's definition. But the moment you let go of that, you suddenly have to have the conversation together: what actually is quality?
Still, he also sees growing awareness in many places that figures don't tell the whole story. Take the consultancy firm where he conducted research, which decided to change how it measured performance. 'Employees no longer felt seen as individuals but reduced to a handful of flat figures, and left the organisation in droves. Based on those signals, the firm adjusted its approach so that employees once again felt more seen.' Or take Utrecht University, which withdrew from international league tables such as the Times Higher Education ranking. 'That's quite a brave step,' says Van der Kolk, 'because a league table like that is precisely the kind of measurement instrument that embeds itself within an organisation. It quickly starts to determine how you manage your own staff: how often do you publish in Science or Nature, how many international students do you attract' By withdrawing, Utrecht University is effectively saying: we will not let an external league table dictate how we define quality, we will determine that ourselves again, with attention to our own local context.
'Figures, after all, carry an appearance of objectivity and neutrality. They're still too often used to claim authority: I know what I'm talking about, because I know the percentages.'
What is quality? Letting go of external yardsticks sounds simpler than it is, Van der Kolk admits. 'If you unquestioningly assume that quality equates to a place on a league table, you no longer really need to think about it yourself. You simply copy someone else's definition. But the moment you let go of that, you suddenly have to have the conversation together: what actually is quality? What do we consider a good employee, a good colleague?' Thinking of his own colleagues, he lands on things that no performance measurement ever captures. 'They need to be capable researchers, of course, but I find it at least as important that someone can listen well, ask the right question, offer constructive criticism, or step in when needed. That's rarely measured systematically, yet these are all essential qualities, and they all sit on the process side, not the output side.'
There are also examples that end less well, and Van der Kolk finds these at least as instructive. A local council he studied once began automatically measuring how long it took staff to issue a passport: around ten minutes on average. At a certain point, that time was used to assess whether people were productive enough. The consequence was stark: if someone came in whom the employee, based on their surname, assumed would need a bit more explanation, then some employees would literally slip away to the toilet, leaving a colleague to take over. Purely to avoid ruining the average. 'What began as an innocent bit of record-keeping thus became an instrument of exclusion.'
Lessons for professionals
In one of Van der Kolk's courses for public-sector professionals, a participant working at a ministry described this as 'watermelon-green'—an indicator that looks green at first glance but turns deep red the moment you cut into it. In VU for Professionals' Certified Public Controller programme, he devotes several sessions to how to set up KPIs responsibly, in line with an organisation's strategy. According to Van der Kolk, there's often a gap between a manager's intention and how an employee experiences it. You might think you're helping someone by gathering feedback points, but the other person may experience it as a burden. 'So it starts with an honest conversation: how do you actually experience this instrument? Design a measurement together with employees.' He also advises against becoming too rigid once a dashboard or measurement system is in place. Ask together what you value in each other's work, and how you can make that visible. 'That works very differently from a manager stating directively, without further dialogue: from now on I'm going to periodically measure four things to determine whether you're doing good work. The first approach builds trust; the second quickly leads to cynicism.'
The rise of AI adds a new layer to this, according to Van der Kolk. Anyone can now produce a huge amount of output in just a few clicks. The question is whether we should still attach so much weight to output, or whether we should instead look more closely at the process itself—within the university, but also beyond it. Does someone work thoroughly and with integrity, in line with the principles a team values? According to him, that says more than the simple fact that someone has appeared in a journal three times.
Looking five years ahead, Van der Kolk hopes above all for greater awareness: the realisation that every measurement is, by definition, incomplete and built on assumptions. 'Figures carry an appearance of objectivity and neutrality. They're still too often used to claim authority: I know what I'm talking about, because I know the percentages. The key lies in better understanding of how figures are actually produced, and greater humility about what you can and can't do with them afterwards.'
Further reading
Read The Quantified Society by Berend van der Kolk, published by Business Contact (2021)
VU for Professionals
With a programme from the School of Business and Economics for Professionals, you choose high-quality, innovative, and ambitious education. With a meaningful impact on the economic, social, and global challenges of our time. Always part-time, so it can be easily combined with work and personal life. From long-term programmes like MBAs and master's degrees in various disciplines to short-term courses focused on specific developments such as sustainability or data-driven management, as well as more general topics like leadership and change management. Exclusive for you as an alumnus Did you know that as a VU alumnus, you can continue developing your career through SBE for Professionals – with exclusive alumni benefits, including discounts on our programmes for professionals? Keep growing and stay connected.
DEEL DEZE PAGINA
