Naturalistic Decision Making

Date: 9/11/2020 Show Description: Dr. Stephen M. Fiore is Director, Cognitive Sciences Laboratory, and Professor with the University of Central Florida's Cognitive Sciences Program in the Department of Philosophy and Institute for Simulation & Training. He maintains a multidisciplinary research interest that incorporates aspects of the cognitive, social, organizational, and computational sciences in the investigation of learning and performance in individuals and teams. His primary area of research is the interdisciplinary study of complex collaborative cognition and the understanding of how humans interact socially and with technology. Where to find Steve: ResearchGate Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: Highlights of work at Cognitive Sciences Laboratory [2:00} Where NDM fits into interdisciplinary research and how it relates to anthropology [3:50] How Steve became familiar with NDM [5:25] “Aha” moments that led to interest in understanding the psychology of groups and teams {9:10] Current CSL projects, including new applications for Artificial Intelligence {12:10} Potential breakthroughs in Theory of Mind project [17:10] How leadership in NDM spaces has shaped Steve’s approach to the work [22:37] The idea os “stewardship” to help direct where the field of NDM should go [25:35] Trends in NDM that warrant debate [27:30] Developing “problem spaces” for better identify and understand complex issues [33:55] Building relationships to raise funding and support around complex problems [35:45] Early career in marketing for the Computer Learning Center and the transition to cognition [39:00] Major influences that shaped Steve’s thinking [42:30] Others’ reaction to NDM when being introduced to it for the first time [50:42] If you could choose three philosophers to be your student, mentor, and collaborator, who would they be? [54:12]

Date: 8/27/2020 Show Description: Today we welcome Emily Patterson. Emily is an associate professor in the School of Health and Rehabilitation Sciences at the Ohio State University. She is known for her application of naturalistic decision making and cognitive engineering in health care. She did groundbreaking work in bar code medication administration when that technology was new, highlighting unintended consequences and making recommendations to improve the integration of new technologies into workflow. She co-authored the national standard for the summative usability testing methodology for ensuring the safety of electronic health records published by the National Institute of Standards and Technology (NIST). She has conducted applied research on clinical reminders, alarms, and transitions of care with a focus on improving the efficiency, usability, and accuracy of clinical documentation in electronic health records. Where to find Emily: The Ohio State University  Usability and User Experience in Health Care certificate program  NISTIR 7804: Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records Macrocognition Metrics and Scenario: Design and Evaluation for Real-World Teams  Learn more about NDM: NaturalisticDecisionMaking.org Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter

Date: 8/07/2020 Show Description: Today we welcome Lee Beach. Lee is the McClelland Centennial Emeritus Professor of Management at the Eller College of Business, at the University of Arizona. He was a Professor of Public Administration and of Psychology, teaching graduate and executive courses on managerial decision making and organizational change. He served on the editorial boards of several well-known journals including: Organizational Behavior and Human Decision Making, The Journal of Behavior Decision Making, and The Journal of Forecasting. He attended the first conference on Naturalistic Decision Making in 1989 and is known for his work on the theory of the mind, the theory of narrative thought, and image theory. He has authored many influential books and articles, and he is also a professional artist. Where to find Lee: NarrativeThoughtForum.com The psychology of narrative thought: How the stories we tell ourselves shape our lives A new theory of mind: The theory of narrative thought The structure of conscious experience Learn more about NDM: NaturalisticDecisionMaking.org Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter

Date: 7/16/2020 Show Description: David Woods has worked to improve systems safety in high risk, complex settings for over 38 years. These include studies of human coordination with automated and intelligent systems and accident investigations in aviation, nuclear power, critical care medicine, crisis response, military operations, outages of critical digital services, and space operations (he was an advisor to the Columbia Accident Investigation Board). He began developing Resilience Engineering in 2000-2003 as part of the response to several NASA accidents and is Past-President of the Resilience Engineering Association. His research publications on people, safety, and complexity have been cited > 25,000 times (H-index>80). Where to find David: The Ohio State University LinkedIn Learn more about NDM: NaturalisticDecisionMaking.org Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Resources mentioned during the show: · Resilience in a pandemic working paper series · The Ohio State University, Department of Integrated Systems Engineering · Resilience Engineering in Practice

Date: 7/6/2020 Show Description: Gary L Klein received his BA in Psychology and his PhD in Cognitive Social Psychology. He is a Senior Principal Scientist in cognitive science and artificial intelligence at The MITRE Corporation, where he has worked for nearly 30 years, focusing primarily on how people acquire and use information in domains including emergency response, air traffic management, and intelligence analysis. He has led projects about using forecasting models that generate graphical depictions of decision spaces to improve decision makers’ “option awareness” under deep uncertainty, designing a Collaborative Option Awareness for Joint Actions, or COAction, process, to provide emergency responders with something beyond situation awareness, and generating a framework for evaluating collaboration. In 2015, Gary and MITRE served as hosts for the 12th International NDM Conference. Where to find Gary MITRE Learn more about NDM: NaturalisticDecisionMaking.org Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Resources mentioned during the show: Drury, J.      L., Pfaff, M. S., and Klein, G. L. (in press). A Method for Rigorously      Assessing Causal Mental Models to Support Distributed Team Cognition. Pfaff,      M. S., Drury, J. L., and Klein, G. L. (2015). Crowdsourcing Mental Models      using DESIM (Descriptive to Executable Simulation Modeling).   Pfaff,      M. S., Klein, G. L., Drury, J. L, Moon, S. P., Liu, Y., and Entezari, S.      O. (2012). Supporting complex decision making through option awareness.      Journal of Cognitive Engineering and Decision Making Klein,      G. L., Drury, J. L., and Pfaff, M. S. (2011). Providing an option awareness      basis for naturalistic decision making. Journal of Cognitive      Technology, Vol 16(2), pp. 10 – 19.

Date: 6/29/2020 Show Description: We’re pleased to be talking today with Joel Suss. Joel is Assistant Professor of Human Factors in the Department of Psychology at Wichita State University. His research work focus on understanding and improving perceptual-cognitive performance – in particular, anticipation, decision making -- in complex and challenging operational settings, such as law enforcement, security, military command and control, aviation, and emergency medicine. He has examined perceptual and cognitive aspects of CCTV monitoring and how automated, intelligent videos surveillance systems are changing the human operator’s role in security surveillance. Joel is perhaps best known known for his work investigating ways to train police officers to make better decision in stressful situations. Originally from Australia, Joel completed his PhD in Applied Cognitive Science and Human Factors at the Michigan Technological University. Where to find Joel: Wichita State's Human Factor's Ph.D. Program Online cognitive-skills training for improving police decision making: Can it work? Professor investigates what makes a good law enforcement officer through lab research Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: How Joel got involved in NDM and cognitive expertise in law enforcement [1:40] The role of anticipation in police decision-making [12:30] How do you help police be better calibrated in their responses? [15:37] National Institute of Justice project to create tactical decision making exercises for police [17:35] Law enforcement’s response to NDM approaches [21:55] Cultural skepticism toward academics within law enforcement [26:30] People within and outside the NDM community who influenced Joel’s career [30:25] How have you benefited from helping others break into the NDM world? [34:25] The role of mentorship as a supplement to NDM graduate programs [38:10] Ways the NDM community can do a better job of developing young NDM researchers [44:05] Joel’s current and future research focus [49:40] Joel’s connection to Peter Fadde [55:07] If you could instantly achieve expertise in anything what would it be? [56:15]

Date: 6/18/2020 Show Description: Today we welcome Shawna Perry. Shawna is an emergency medicine physician and runs a consulting firm called SJP Consulting. In both her roles as a physician and a consultant, she actively advocates for systems level views of patient safety. She is an active member of the naturalistic decision making community, the human factors and ergonomics society, and the resilience engineering association. She uses her vast and varied clinical experiences to mentor countless resident physicians and also to inform applied research. Shawna very generously serves on panels, gives invited talks, and collaborates on writing projects to share her front-line perspective on the organizational constraints, competing goals, and other challenges emergency department physicians face every day as they care for patients. Dr. Perry spent 6 years as the Director for Patient Safety System Engineering at Virginia Commonwealth University Health Systems and is currently an Associate Professor of Emergency Medicine at the University of Florida Health Science Center in Jacksonville, FL. Where to find Shawna: 1.  Klein, G., Pliske, R., Crandall, B., & Woods, D. D. (2005). Problem detection.  Cognition, Technology & Work ,  7 (1), 14-28. DOI:  10.1007/s10111-004-0166-y 2.  Cook, R., & Rasmussen, J. (2005). “Going solid”: a model of system dynamics and consequences for patient safety.  BMJ Quality & Safety ,  14 (2), 130-134.  doi:  10.1136/qshc.2003.009530 3.  Perry, S., Wears, R. L., Anderson, B., & Booth, A. (2007). Peace and war: Contrasting cases of resilient teamwork in healthcare. In  Eighth International Naturalistic Decision Making Conference, Pacific Grove, CA . Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: How did you become interested in NDM? [1:50] What are different attitudes that you've observed in regard to patient safety in clinical settings? [12:30] What are some times you've gotten pushback on your ideas? [23:35] What's a favorite story where you had to rely on your intuition? [42:40]

Show Description: Peter Fadde is professor and director of the Learning Systems Design and Technology (LSDT) graduate program at Southern Illinois University. Olivia Brown is a Research Associate at the University of Bath, School of Management. Where to find Peter: Accelerating the Acquisition of Intuitive Decision-Making through Expertise-Based Training (XBT) Instructional Design for Accelerated Macrocognitive Expertise in the Baseball Workplace TEDx talk "Frugal Engineering in Baseball and Beyond" Association for Applied Sport Psychology: Virtual Conference on Technology in Applied Sport Psychology Practice Where to find Olivia: University of Bath Teamwork in extreme environments: identifying challenges and generating solutions CREST Research Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: “Extreme team” research premise [2:30] Forming expedition teams for team research [4:45] Team research data and observations [6:15] Prepping research participants on how to share their data [9:05] Peter describes the background of his award-winning research in expertise-based training [10:10] Explaining the video occlusion method [21:30] How good are experts at articulating their expertise? [27:00] Debate between immersive experience training vs. targeted skill training [31:00] How Olivia got into Naturalistic Decision Making [33:00] Justifying NDM’s unconventional research methods [35:10] Advice for those looking to conduct similar NDM studies [36:00] Olivia's background in extreme environments and what drew her to “extreme team” research [38:15] What "extreme team" research can teach us about how people work together generally [40:45] Applying sports-based NDM research to real life [42:15] Peter compares STEM training to sports training [46:45] None of us are immune, and maybe de-stigmatizing racism will lead more people to confront it in ourselves [53:15] "If both of you could instantly become an expert in something, what would it be?" [56:30]

Date: 5/22/2020 Show Description: “Jan Maarten is Principal Scientist at TNO and Professor of Applied Cognitive Psychology at the University of Twente, The Netherlands. His research interests include resilience engineering, team communication processes, and human-machine teaming. He is the lead editor on two influential volumes: Cognitive Task Analysis (2000) and Naturalistic Decision Making and Macrocognition(2008). He is co-editor of the recently released Oxford Handbook of Expertise (2020). He is editor in chief of the Journal of Cognitive Engineering and Decision Making. Dr. Schraagen holds a PhD in Cognitive Psychology from the University of Amsterdam, The Netherlands.” Where to find Jan Maarten: University of Twente TNO LinkedIn JCEDM The Oxford Handbook of Expertise Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: What is the first paper you ever published? [2:00] Experience conducting research with children [4:11] Nature of Jan Maarten's 1993 research study on expertise [6:15] Working with Herb Simon at Carnegie Mellon [10:57] What led Jan Maarten to attend the 1994 NDM conference in Dayton [13:00] Discussing the birth and significance of the book "Cognitive Task Analysis" [17:25] Experience and advice for navigating difficult research interviews [20:35] Interviewing technique advice for students [23:52] The most exciting project Jan Maarten is working on right now [27:47] Goals and directions for future work [36:50] What is a "cyber-physical system"? [40:20] Industries that could benefit from an NDM perspective [42:30] Which project has been the most rewarding for you? [47:50] Evaluating the success of Dutch navy training simulations [54:50] One questions that can reveal if someone is an NDM practitioner [57:06] Two truths and a lie [58:30]

Date: 5/15/2020 Show Description: “Colonel Shattuck is Senior Lecturer and Director of the Human Systems Integration Program at the Naval Postgraduate School in Monterey, California. Dr. Shattuck holds a faculty appointment in the Operations Research Department where he teaches human systems integration and human factors engineering. Colonel Shattuck graduated from the United States Military Academy in 1976, and following distinguished service as a signal officer and receiving his PhD from the Ohio State University in Cognitive Engineering, he returned to his alma mater to teach Engineering Psychology until his retirement from the Army in 2005. He has been at NPS since, he co-directs the Human Systems Integration Program. Larry has been an active researcher in the domain of military command and control for nearly two decades. He is unique among his peers in the NDM community for his ability to draw on deep operational experience when studying processes such as how commanders communicate their intent, the manner in which decision makers fuse data in tactical environments, and the ways in which data and information flow through technological system elements to the human agents in that system.” Where to find Larry: Naval Postgraduate School HSI DL Certificate Program HSI DL Master’s Degree Program Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: How Larry's military operational experience has informed his research findings [2:20] Making the shift to engineering psychology [7:47] Takeaways from the Army research laboratory's advanced decision architecture program [10:12] Examples of the program's contributions to the Army [13:58] Making progress with NDM solutions within a government context [19:10] Explaining the relationship between NDM and Human Systems Integration [29:07] Making the NDM research process more efficient [34:40] How NDM perspectives have influenced Larry's approach to teaching [38:26] The aspect of Larry's work that has been most rewarding [40:10] Lessons learned from students [43:45] Larry's top three mentors [45:46] Influences outside of NDM [49:25] One question that can determine if someone is an NDM practitioner [51:07] Two truths and a lie [52:08]