The Importance of Reflexive Practice in Evaluating Data

This is a replay of a 2018 conversation with Professor James Wilsdon when he was a Professor of Research Policy in the Department of Politics at the University of Sheffield in the UK. He is now at UCL as a Professor of Research Policy in the Dept for Science, Technology, Engineering and Public Policy (STEaPP). James has been involved in many policy and think tank initiatives. Of particular interest here, he chaired an independent review of the role of metrics in the management of the UK’s research system, publishing a final report in 2015 called The Metric Tide. Connecting to our recent episodes on review practices, we start here at about 19mins into the original recording exploring what is the purpose of research evaluation.

“Metrics are a technology and there is nothing intrinsically good or evil in them, it’s all about how they are used.”

“It is incumbent on us not to indulge processes of evaluation that we know empirically are bad science.”

Full Transcript available here.

Related links:

Original Feb 2018 podcast with James Wilsdon on impacts, responsible metric & evaluation practices

Aricles about James’ move to UCL:

UCL https://www.ucl.ac.uk/steapp/news/2022/dec/metrics-metascience-and-now-ucl-steapp-welcomes-james-wilsdon

Research on Research Institute (RoRI): https://researchonresearch.org/tpost/6283xjy2y1-roris-director-moves-to-ucl - includes links to more recent responsible research evaluation reports James has been involved in.

UK research funding councils – Higher Education Funding Council - http://www.hefce.ac.uk

Research Excellence Framework (REF) - http://www.ref.ac.uk/2014/

The Metric Tide report – https://responsiblemetrics.org/the-metric-tide/

San Francisco Declaration on Research Assessment - http://www.ascb.org/dora/

Leiden Manifesto for Research Metrics - http://www.leidenmanifesto.org