The Cognitive Crucible #174 Kara Masick on Assessment Insights from Program Evaluation
The Cognitive Crucible is a forum that presents different perspectives and emerging thought leadership related to the information environment. The opinions expressed by guests are their own, and do not necessarily reflect the views of or endorsement by the Information Professionals Association.
During this episode, US Air Force Major Kara Masick discusses her thought-provoking article: 12 (Mis)Steps from Sober Assessments: Confessions of a Failed OIE Assessor. Our discussion covers her OIE assessment recovery journey. Like an alcoholic seeking sobriety by first admitting they have a problem and acknowledging where they’ve gone wrong, this essay is 12 confessions of her OIE assessment failures. During the discussion, she presents a new assessment metric that she calls: Most Likely Cause (MLC). Assessors should learn to think like an IT Helpdesk professional; like detectives discovering clues to find the most likely culprit, we can weigh relative likelihoods that our OIE had the effect compared to other potential causes.
Resources:
- Cognitive Crucible Podcast Episodes Mentioned
- IPA Blog Article: 12 (Mis)Steps from Sober Assessments: Confessions of a Failed OIE Assessor by Kara Masick
- The Iron Law Of Evaluation And Other Metallic Rules by Peter H. Rossi
- Evaluation: A Systematic Approach by Peter H. Rossi, Mark W. Lipsey, Gary T. Henry
- Foundations of Program Evaluation: Theories of Practice By Shadish, Cook, Levitan
- Scriven, M. (1974). Maximizing the power of causal investigations: The modus operandi method. In Evaluation in education (pp. 68–84). McCutchan Publishing
- Scriven, M. (2005). The Logic and Methodology of Checklists.
- McGuire’s Classic Input–Output Framework for Constructing Persuasive Messages. In Public Communication Campaigns (Fourth Edition, pp. 133–145). SAGE Publications, Inc.
- Decoding Crimea. Pinpointing the Influence Strategies of Modern Information Warfare by Alan Kelly and Christopher Paul
Link to full show notes and resources
Guest Bio:
Kara Masick is an Air Force 14F (Information Operations officer) with a passion for MISO/PSYOP who's worked mostly within Intel and Cyber organizations. She was the first 14F officially assigned to the J39 of the Information Warfare Numbered Air Force (16AF). She was sponsored by USSOCOM to study Psychology and is doing that within the Measurement Research methodology Evaluation and Statistics Lab at George Mason University (GMU) with the goal of improving her MISO operations and assessments contributions. Her dissertation research on persuasion uses Large Language Models to analyze text. Previously, she studied theology at the University of Oxford (certificate), Behavioral Science (BS) with Religious Studies and Arabic minors at the Air Force Academy, and Sociology (MA) at GMU.
About: The Information Professionals Association (IPA) is a non-profit organization dedicated to exploring the role of information activities, such as influence and cognitive security, within the national security sector and helping to bridge the divide between operations and research. Its goal is to increase interdisciplinary collaboration between scholars and practitioners and policymakers with an interest in this domain.
For more information, please contact us at communications@information-professionals.org.
Or, connect directly with The Cognitive Crucible podcast host, John Bicknell, on LinkedIn.
Disclosure: As an Amazon Associate, 1) IPA earns from qualifying purchases, 2) IPA gets commissions for purchases made through links in this post.