Changing Academic Life

We need to reform research evaluation because "It's not just which people are we excluding, which profiles are we excluding, which ideas are we excluding, but also whose problems are we not solving" says Dr Karen Stroobants.Karen is a researcher, policy adviser and consultant on research policy and strategy, with a focus on research culture. Her research background and PhD is in chemistry. She now manages a portfolio of activities, combining roles as (part-time) lead policy advisor on research landscape & economy for the Royal Society of Chemistry in the UK and as freelance consultant, focusing most recently on contributing to the drafting of a European agreement on research assessment reform. This agreement on reforming research assessment from CoARA, the Coalition for Advancing Research Assessment, is the reason I wanted to speak with Karen. We cover: concerns of current research assessment approaches; the need for both top-down and bottom up buy in to create research culture change and what that change might look like; how we can navigate career choices while the system is still in transition; the importance of reflection for research culture change and embracing a diversity of people ideas and research problems; the core commitments of the CoARA agreement; and the move to more qualitative assessments at both individual and institutional and national levels. We finish with Karen reflecting on her own career choices driven by values and what is important. [00:00:29] Episode introduction[00:02:39] Welcome & Introduction[00:04:39] Concerns around research system[00:13:05] Research culture change needs top-down and bottom-up buy-in[00:20:12] Negotiating choices while the system is undergoing transition[00:23:25] Importance of reflection for research culture change[00:30:21] Diversity of people, ideas and research problems[00:34:17] CoARA Agreement on reforming research assessment[00:40:04] Signing up to the CoARA agreement[00:50:10] Narrative CVs - for inividuals, organisations and national level[00:54:02] Other ways of brining a qualitative lens[00:56:15] Karen's career path - setting boundaries, choosing valuesRelated links:CoARA: Coalition for Advancing Research Assessment https://coara.euProf Frank Miedema, UMC Utrecht, https://www.umcutrecht.nl/en/research/researchers/miedema-frank-f#Marie Sklodowska-Curie Actions MSCA https://marie-sklodowska-curie-action; San Francisco Declaration on Research Assessment s.ec.europa.euINORMS: International Network of Research Management Societies https://inorms.netDORA: San Francisco Declaration on Research Assessment https://sfdora.org/dora-community-engagement-grants-supporting-academic-assessment-reform/The Metric Tide: Review of metrics in research assessment https://www.ukri.org/publications/review-of-metrics-in-research-assessment-and-management/Book:Caroline Criado Perez, Invisible Women, Vintage Books, 2019https://carolinecriadoperez.com/book/invisible-women/ Related podcasts:Sarah Davies: Part 1 on mobility, precarity and notions of excellence https://www.changingacademiclife.com/blog/2023/cal81-sarah-davies-part-1 and part 2 on luck, disrupting excellence, and cultures of care https://www.changingacademiclife.com/blog/2023/cal82-sarah-davies-part-2Tanita Casci and Elizabeth Adams on supporting, rewarding and celebrating a positive collegial research culture https://www.changingacademiclife.com/blog/2021/5/21/tanita-casci-elizabeth-adamsJames Wilsden on metrics and responsible research evaluation https://www.changingacademiclife.com/blog/2023/cal77-james-wilsdon-replay

This is a short episode to kick off the new Season of the Changing Academic Life podcast series.Looking forward to bringing some great conversations coming on a range of topics fro reforming research assessment, burnout and neurodiversity.A new website will be coming very soon!You can access all the latest episodes via this link that will connect you to your favourite podcast player. And don’t forget to subscribe if you haven’t already to be sure you don’t miss any episodes!

This is the end of the current season of episodes. I take the opportunity to shortly reflect on the themes we’ve heard - around listening, leadership, wellbeing, peer review, and research excellence and evaluation. I also invite you to share your feedback and suggestions, and I share some of the various transitions happening over the break, all leading up to the next season starting Sept/Oct 2023.If you missed any episodes from this season, you can find them all on your favourite podcast app where you subscribe to this or go to the Changing Academic Life podcast page. I’d also really love to hear from you if you have any feedback generally about what works well, what doesn’t work so well with the podcasts for you, any suggestions for future themes or people you’d like me to talk with, or if you’d like to volunteer to help out with a transcription. You can contact me via the contact form here or via my uni account (email on this page).Full transcript available to download here.Overview:00:27 Introduction and request for feedback and suggestions.02:03 Walking through common themes of the last 23 episodes of this season 09:26 The transition to a new podcasting server platform and new webpage12:59 The personal transition post-Sept 202314:51 Wrapping up, reminder re feedback, and final thanks18:59 EndSpecial Thanks:Thanks to Sabrina Burtscher for all the help with transcriptions and wishing her all the best for her PhD. Thanks too to Rafael Vrecar, Katharina Werner and Silke Buchberger who are helping with transcriptions.

Sarah Davies is a Professor of Technosciences, Materiality, & Digital Cultures at the Department of Science and Technology Studies at University of Vienna. Overall her work explores how science and society are co-produced, with the digital and digitisation being key aspects. Of particular interest for our discussions is her current research on the conditions of academic work and knowledge production. In Part 2, Sarah Davies shares from both her research and lived experiences on topics like equity and valuing diverse work, care work in academia and who does that work, creating collegial research cultures, and about luck - the trigger for why I wanted to talk to Sarah. She discusses a recent paper she and co-authors published on luck and the situations of research, and how accounting for luck might just be one way of disrupting problematic rhetorics of excellence. This is a continuation of Part 1 of our conversation where she talked about her own career path, touching on issues of mobility, precarity and projectification of research notions of excellence. “Care for people is very integrated with care for their science…You [can’t] separate the epistemic production, the knowledge production, from caring for people”“It throws into contrast, this narrative of the heroic, excellent individual researcher, and the complex contexts in which good research is done.”“You can't do science alone, you can't be this isolated person, you actually need quite some social skills” “When we talk about excellence, that is somehow really seen as not including luck […] It's important to take luck seriously.”Overview (times approximate) [Transcript coming soon]:00:29 Episode introduction03:26 Sarah’s inaugural lecture06:06 Negotiating contemporary academia, related equity issues & how we can see and value diverse work08:17 Acknowledging care work and who does that work11:09 Increasing interest in statements on research cultures12:38 Having explicit discussions about how we engage together17:19 The way people speak about luck in academic trajectories & the disruption of the excellence rhetoric22:15 Toxic behaviors and the care work it requires25:14 Taking luck seriously & her won experiences of luck30:00 The importance of people and social skills31:55 Sarah’s experiences in setting up her own group and learning through doing37:05 EndRelated links:Sarah’s Inaugural lectureText: https://phaidra.univie.ac.at/o:1622120Video: https://vimeo.com/780321785/0ef9327e6b Papers:Davies, Sarah R., and Maja Horst. 2015. ‘Crafting the Group: Care in Research Management’. Social Studies of Science 45 (3): 371–93. On ‘research culture' (56 min) - Wellcome seems to be taking the lead here:Davies, S. R., & Pham, B.-C. (2023). Luck and the ‘situations’ of research. Social Studies of Science, 53(2), 287–299. Davies, S.R. &  B-C Pham, Why don't we account for luck in research careers? LSE Blog, 18 April 2023.Liboiron, Max, Justine Ammendolia, Katharine Winsor, Alex Zahara, Hillary Bradshaw, Jessica Melvin, Charles Mather, et al. 2017. ‘Equity in Author Order: A Feminist Laboratory’s Approach’. Catalyst: Feminism, Theory, Technoscience 3 (2): 1–17. Loveday, Vik. 2017. ‘“Luck, Chance, and Happenstance? Perceptions of Success and Failure amongst Fixed-Term Academic Staff in UK Higher Education”: Luck, Chance, and Happenstance?’ The British Journal of Sociology, September.  Lund, Rebecca W. B. 2015. Doing the Ideal Academic - Gender, Excellence and Changing Academia. Aalto University.Tanita Casci and Elizabeth Glasgow CAL podcast conversation: ‘Tanita Casci and Elizabeth Adams on supporting, rewarding and celebrating a positive collegial research culture’ Sarah’s collaborators/team membersAriadne AvkıranEsther DessewffyFredy Mora GámezBao-Chau PhamAndrea SchikowitzKathleen GregoryNora EdererElaine Goldberg CC BY-SA 4.0 by Geraldine Fitzpatrick and Sarah DaviesThanks to Jana Herwig and Mario Seidl from the Vienna University Center for Teaching and Learning (CTL) Video production suite for use of their podcast recording facilities.

Sarah Davies is a Professor of Technosciences, Materiality, & Digital Cultures at the Department of Science and Technology Studies at University of Vienna. Overall her work explores how science and society are co-produced, with the digital and digitisation being key aspects. Of particular interest is her current research on the conditions of academic work and knowledge production. In Part 1 here, we discuss her experiences with academic mobility, touching on issues of cultural differences, precarity, and reflecting on who is able to be mobile or not and with what consequences. This leads to discussions about how we interpret CVs, and she challenges us to re-think internationalization, projectification of research, and notions of excellence. Part 2 is out now too.“Research always unfolds…also …finding things that I thought were interesting, or people that I wanted to work with, and that was driving some of the choices I made around this mobility.”[Projectification]: “the shift to a project logic, where you have to design a piece of research that fits into a certain timeframe… it is actually a totally different logic to older imaginations of scholarship … something that unfolded over many years. It was not oriented to packages of funding”Overview (times approximate) [Transcript coming soon]:0:05 Welcome to Changing Academic Life & episode introduction 02:58 Sarah introduces her background and her many geographical and disciplinary moves08:30 The passive voice removing the messiness of the lab11:02 Move from science communication to science and technology studies16:30 The red threads through her work19:00 Coping with the uncertainty of shorter term contracts23:00 The cultural challenges of the different academic contexts33:15 Re-thinking how we imagine internationalization and mobility to explore other ways of creating networks37:38 How we interpret CVs to recognize diverse situations40:56 Notions of excellence and projectification, re-thinking new funding models and more long-term positions51:47 EndRelated links:Sarah’s web pageExample ‘Projectification’ references:Felt, Ulrike. 2017. ‘Under the Shadow of Time: Where Indicators and Academic Values Meet’. Engaging Science, Technology, and Society 3 (February): 53. https://doi.org/10.17351/ests2017.109.Ylijoki, Oili-Helena. 2014. ‘Conquered by Project Time? Conflicting Temporalities in University Research’. In Universities in the Flux of Time, 108–21. Routledge.Stuart Reeves’ podcast conversation CC BY-SA 4.0 by Geraldine Fitzpatrick and Sarah DaviesThanks to Jana Herwig and Mario Seidl from the Vienna University Center for Teaching and Learning (CTL) Video production suite for use of their podcast recording facilities.

Dr Elizabeth Churchill is a Senior Director at Google. We recorded this interview while we were both at a conference, where she was awarded a SIGCHI Lifetime Service Award. In this conversation she shares insights and experiences around building good team cultures, managing diversity, onboarding for global teams, and some management frameworks that she has found useful. She also shares her journey from a psychology background to working in big tech and from research to now delivering technical infrastructures, what she was looking for in moving between companies, and the red threads of her love of people, of being challenged and continually learning.“In a team, you don't need to know everything. Yes, it's actually a collective.”Download a full transcript of the conversation here.Overview (times approximate):0:05 Welcome to Changing Academic Life. 0:30 Intro to the episode02:18 The joy of in-person conferences04:09 Elizabeth introduces herself and her current operating systems work09:12 Team culture and diversity15:18 Negotiating tensions and conflicts23:15 Culture of software engineering environments27:48 Onboarding for a global team31:37 Frameworks for management of teams39:35 Her fascination with people that took her from psychology to large scale tech companies 43:08 The pragmatics that led her to industry positions and from industry research to platform47:39 The motivations around the moves to different companies53:02 Love of learning and taking on new challenges56:02 Her Interactions magazine article on imposterism & being comfortable not knowing everything01:02:11 Final exhortation find your community01:06:05 EndRelated links:Elizabeth at LinkedIn, Wikipedia, TwitterSIGCHI Lifetime service award CHI2023 conference [Book] Lee Vinsel & Andrew Russell, 2020, The innovation delusion, Penguin Random House. [People] Steve Benford, Nottingham Uni [Team management frameworks:]Cynefin Framework (Dave Snowden, 1999) Polarity Management (Barry Johnson, 1996)[Article] Elizabeth F. Churchill. 2019. Impostor syndrome and burnout: some reflections. interactions 26, 3 (May - June 2019), 20–21.

This is part 2 of my conversation with Dr Marta Cecchinato (Part 1 is here). Marta is an Assistant Professor (Senior Lecturer) in the Computer and Information Sciences Department at Northumbria University. Her research focuses on understanding the complexities of multiple technologies in everyday life and how digital experiences can be shaped to support wellbeing at work and in our personal lives.  In part 2, we go on to discuss a range of issues from how her wellbeing stance has influenced her leadership of a new group and the Athena Swan EDI initiative in her department, how managers can better support people on parental leave, her own experiences being a first-time mother, and the digital wellbeing strategies that have been most helpful to her. “I love my job, but it’s a job at the end of the day, and just because I stopped sooner, and I don't work longer hours every day doesn't mean that I love my job less. I'm probably setting a better example. For others as well.”“There's so much more to life than than just work”Overview (times approximate): [Full TRANSCRIPT available here]00:30 Preamble03:00 Setting up her own group and building trust in your relationships 9:11 The transition from a self-acknowledged control freak to a leader. 13:46 Leading the Athena Swan exercise. 17:53 Digital wellbeing, flexible working and EDU issues 20:03 How managers can better support people going on parental leave25:30 Having her own first child during the pandemic 31:12 The importance of being out in nature. 35:02 The micro-boundary strategies with the biggest impact for her 38:01 Sabbatical plans41:31 How Marta creates micro moments of rest in her day44:29 Key strengths47:11 EndRelated links:Digital Wellbeing Strategies Handbook (updated Sept 2023)Marta’s personal website & Northumbria profileMarta on social media: LinkedIn and TwitterAurora leadership programmeAisling O’Kane CAL podcast Part 1 and Part 2 Advance HE Athena Swan Charter Oscar Trimboli CAL podcast Part 1 and Part 2

Dr Marta Cecchinato is an Assistant Professor (Senior Lecturer) in the Computer and Information Sciences Department at Northumbria University. Her research focuses on understanding the complexities of dealing with multiple technologies in everyday life and how digital experiences can be shaped to support wellbeing at work and in our personal lives. In part 1 of our conversation, she talks about her digital wellbeing research, the concept of microboundaries, and strategies for taking more control of these microboundaries. You can also download her Digital Wellbeing Strategies Handbook to find more practical strategies.“It's not about having that equal balance between work and life. As life changes, your responsibilities change, you need to revise things.”“The way we use technology is different depending on the roles that we are embodying in that specific moment.”“Micro boundary strategies help limit the negative impact that might arise from constant micro role transitions throughout the day.”“Empower people…to make the technology to work for them, rather than working for the technology.”Overview (times approximate): [Full transcript available here]00:30 Introductions02:22 Marta’s psychology background and path to her PhD at UCLIC around work life balance in the digital age05:37 Language and culture differences in terminology eg no WLB term in Italian09:47 What made her fall in love with research13:15 Her PhD research journey to a focus on digital wellbeing 17:10 Her issues with the term WLB19:20 A digital wellbeing approach, empowering people bottom-up, microboundary strategies & making these accessible via a downloadable booklet23:20 Red thread of improving life, also through impacting EDI issues28:15 The definition of microboundary supporting micro-role transitions & practical examples from her own life32:05 The importance of reflection and colleagues to reflect with38:05 Broad types of microboundary strategies38:53 EndRelated linksFind Marta at:Personal website: https://www.cecchinato.me/ Northumbria profile: https://www.northumbria.ac.uk/about-us/our-staff/c/marta-cecchinato/ LinkedIn: https://www.linkedin.com/in/martaececchinato/ Twitter: https://twitter.com/martacecchinato Digital Wellbeing Strategies Handbook (updated Sept 2023)2019 Digital Wellbeing workshop: https://digitalwellbeingworkshop.wordpress.com/ Giulio Jacucci, Uni of Helsinki and Eve Hoggan Abi Sellen, Microsoft ResearchUCLIC UCL Interaction Centre, Anna Cox, UCL and Anna’s CAL podcastEU Erasmus programme

This is a replay of a 2018 conversation with Professor James Wilsdon when he was a Professor of Research Policy in the Department of Politics at the University of Sheffield in the UK. He is now at UCL as a Professor of Research Policy in the Dept for Science, Technology, Engineering and Public Policy (STEaPP). James has been involved in many policy and think tank initiatives. Of particular interest here, he chaired an independent review of the role of metrics in the management of the UK’s research system, publishing a final report in 2015 called The Metric Tide. Connecting to our recent episodes on review practices, we start here at about 19mins into the original recording exploring what is the purpose of research evaluation.“Metrics are a technology and there is nothing intrinsically good or evil in them, it’s all about how they are used.”“It is incumbent on us not to indulge processes of evaluation that we know empirically are bad science.”Full Transcript available here.Related links:Original Feb 2018 podcast with James Wilsdon on impacts, responsible metric & evaluation practicesAricles about James’ move to UCL:UCL https://www.ucl.ac.uk/steapp/news/2022/dec/metrics-metascience-and-now-ucl-steapp-welcomes-james-wilsdon Research on Research Institute (RoRI): https://researchonresearch.org/tpost/6283xjy2y1-roris-director-moves-to-ucl - includes links to more recent responsible research evaluation reports James has been involved in.UK research funding councils – Higher Education Funding Council - http://www.hefce.ac.ukResearch Excellence Framework (REF) - http://www.ref.ac.uk/2014/The Metric Tide report – https://responsiblemetrics.org/the-metric-tide/San Francisco Declaration on Research Assessment - http://www.ascb.org/dora/Leiden Manifesto for Research Metrics - http://www.leidenmanifesto.org

This conversation continues the reflections on our peer reviewing practices, this time bringing in the experiences of someone from the Global South. Here I am speaking with Prof Heike Winschiers-Theophilus. Heike is a Professor in the Faculty of Computing & Informatics, at Namibia University of Science & Technology. Heike shares the many challenges she and her colleagues have faced in the form of reviewer assumptions and biases, often informed from a Global North mindset as the default ‘mainstream’ gold standard.  She also talks about the challenges in getting their research to be equally valued in making contributions to knowledge creation, and not just accepting their projected positions as exotic others and mere users of knowledge. This is also exacerbated by the exclusionary implications of open access for them, and the difficulties in getting their work more internationally visible and cited. “At times I don’t understand what is happening in the Global North. I don’t understand the issues and conversations.”“How much context must we give so we can override assumptions?”“You are exotic therefore your value comes through being different. But that’s not the case.”“It’s still keeping the ‘mainstream’ as the goalposts but that’s not what diversity is supposed to be like. We are equivalent to any other”“Reviewers explicitly write that actually: ‘we understand there should be diversity but you don’t compare to mainstream’”“I don’t think people can imagine we do serious research here.”“Do they [reviewers] really read our study, do they even look at the academic value of it. Or is it becoming a political discussion now.”“We should remember our own western bias. But reviewers keep on forgetting that. Because now they are in this power position.”“Context and assumptions … when I write I have to think, who can be the reviewer and how much must I write to convince the reviewer…to override whatever assumptions the person has. And it becomes really complicated.”“We are no longer part of the whole knowledge creation process. [… Open access] is totally counterproductive for us. And that’s scary.”Overview (times approximate): [Full TRANSCRIPT coming soon]0:05 Welcome to Changing Academic Life. 0:30 Intro to the episode03:17 Welcome Heike introduces herself, how she came to Namibia, the challenges doing a PhD, and now working as a professor in Namibia and integrating issues around linguistics, AI and culture 09:40  Ways she experiences some of the cultural tensions giving example of developing software and the participatory design techniques that didn’t work in this context, leading to shift in research focus onto methodologies and indigenous knowledge.…The importance of asking questions14:42 Being fully immersed in Namibia and now not understand the issues and conversations in the Global North and how this makes communication about what she does here harder and harder.15:57 Example of these issues are important in the Global North but not important for her – the difference between learned or abstract empathy but not lived. Tolerance as an example. Diversity and inclusion also. The dissonance between behaviour and abstract discussions.18:27 How this plays out in the review process. Heike goes through some of their recent review experiences – illustrating a lot of assumptions e.g., issues around diversity, different expectations about the value they ‘should’ deliver being ‘exotic’22:27 Keeping the [WEIRD] ‘mainstream’ as the goalposts and reviewers expecting them to compare to the ‘mainstream’ despite espousing diversity as a principle. Yet WEIRD is not even the majority on the planet.25:07 Anchored in views, eg from media, that Africa cannot be progressive or come up with innovations. 26:25 And opinions being even worse than biased assumptions, being called colonisers which is really insulting, and the new form of apartheid ‘you are not allowed to use VR’Telling the story of one particular paper and conference review experience32:38 The power of the reviewers and in particular the 2AC, and the gamble of whether the 2AC likes it or not34:15 The impact of these sorts of reviews on the people involved36:49 The question of who can be their audience and who not, trying to anticipate and mitigate reviewer assumptions and bias, and the problems of anonymity for them41”10 The challenges of visibility and getting cited, and the exclusive consequences of open access publishing when they don’t have national agreements or access to funds to pay to publish a paper, and are relegated to users not creators of knowledge.45:09 Discussing different open access or publication models, and also the similar challenges of costs for conferences and what is missed there by not going50:14 Pluraversality as an alternative for looking at different perspectives and that all fit… and also complicated to operationalize as an abstract concept54:00 How to change our fundamental underlying perspectives – encouraging people to experience, come to Africa, see that there is real research happening, and more collaborations58:06 It’s about individuals, the power is with the 1AC and 2AC, that we carefully choose our ACs and needing mentorship for ACs1:01:49 And the extra challenges for people in the Global South who don’t have Heike’s experience; and the importance of national gatherings and the challenges getting these recognised1:06:14 Wrapping up1:07:36 My final reflections1:09:53 EndRelated links:[Abbreviations; Acronyms]PD: Participatory DesignCHI: Computer Human Interaction1AC/2AC: Associate Chair roles in our SIGHI conference program committees, responsible for managing the reviews for a set of papersVR: Virtual RealityWEIRD people: Western, Educated, Industrialized, Rich and DemocraticACM: Association of Computing Machinery, a professional organisationACM SIGCHI: Special Interest Group for Computer Human Interaction https://sigchi.org GIZ: the main German development agency, "Deutsche Gesellschaft für Internationale Zusammenarbeit"[People]Prof Christiane Floyd: https://en.wikipedia.org/wiki/Christiane_Floyd Stanley[Misc]SIGCHI Ethics Committee: https://sigchi.org/ethics/ SIGCHI Gary Marsden Travel Awards: https://sigchi.org/awards/gary-marsden-travel-awards/ [Book]Arturo Escobar, Designs for the Pluriverse: Radical Interdependence, Autonomy, and the Making of Worlds, 2018, Duke University Press[Heike’s publications if you want to check them out]