Naturalistic Decision Making

Date: 5/15/2020 Show Description: “Today we welcome Julie Gore. Julie Gore is a Reader in Organizational Psychology, at the School of Management, University of Bath in the UK. She is a Chartered Psychologist and Fellow of the British Psychological Society. Her research focus is on the psychology of expertise and Naturalistic Decision Making (NDM) across a range of professions working under uncertainty. She is Associate Editor for Journal of Occupational and Organizational Psychology and serves on the boards of the British Journal of Management and Frontiers in Organizational Psychology. Julie is also an academic advisor for Nasdaq’s Behavioural Science Lab. Dr Gore received one of the world’s first NDM PhD degrees in Applied Cognitive Psychology from Oxford Brookes University, UK.” Where to find Julie’s work: The Oxford Handbook of Expertise  The University of Bath  Naturalistic Decision Making and Uncertainty Naturalistic decision making: navigating uncertainty in complex sociotechnical work SPRITE+  Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: Experience as one of the first NDM PhD program participants [1:46] How Julie initially heard about NDM [3:03] Julie's PhD research question [3:55] Discoveries made during PhD research [5:18] Advice for young NDM PhD students [6:15] Common barriers students need help managing [8:02] Surprises and concerns about today's generation of NDM researchers [10:10] Examples of PhD students whose perspectives have been transformed by NDM models [12:05] Making space for NDM research papers in journals and publications [14:52] Nature of Julie's work with NASDAQ [16:32] Exciting research on decision-making ethics and accountability in the digital space [19:23] Organizations' and outsiders’ attitudes toward NDM [23:17] Showcasing the utility of NDM methods [25:18] A particularly rewarding project [27:10] An explanation of "adaptive expertise" [29:12] Opportunities to advance NDM in the context of today's novel circumstances [32:37] Three people who influenced Julie's career [33:47] Influences from outside NDM [34:43] Changes in the NDM community [36:35] One question that can determine if someone is an NDM researcher [37:30] Directions for future research [39:37] Two truths and a lie [41:02]

Robert Hutton, co-founder of Trimetis and a cognitive systems engineering expert, dives into the challenges of establishing his consultancy in England, often working with the Ministry of Defence. He shares fascinating stories from intense field research in places like Iraq and Lebanon, highlighting interactions with local security forces. Emphasizing the importance of bridging theory with practice, Hutton also discusses innovative teaching methods that connect psychology to real-world applications while tackling the complexities of cognitive task analysis.

Date: 5/1/2020 Show Description: “Today we welcome our pal, Jennifer Phillips. Jenni is the CEO of the Cognitive Performance Group, which she cofounded in 2006. Her work focuses expert decision-making, primarily in the military training community. She and her colleagues at CPG have pioneered research into the development of skilled performance, and designed innovation in the areas of decision-centered training and assessment, including the development and application of mastery models. Among her many publications is a chapter in the Oxford Handbook of Expertise exploring mastery models and their value in supporting the acquisition and assessment of expertise.” Learn More About Jenni: Cognitive Performance Group Developing Mastery Models to Support the Acquisition and Assessment of Expertise Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: Jenny describes her focus of expertise, including tactical thinking in the marine corps and mastery models [1:30] Jenni breaks down the term “mastery model” and its stages [3:45] Jenni gives a mastery model example from her work with marine corps instructors [6:05] Work on assessment of decision skills, performance rubrics, and the process of adaptability [9:50] Major influences in Jenni’s career in NDM [14:05] Other people and organizations outside the NDM community who have impacted Jenni’s work [16:30] What makes Jenni’s approach and expertise unique in her field [19:15] Challenges and nuances of working in a military environment [21:30] Difficulty getting buy-in to conduct research in law enforcement [23:45] A favorite anecdote from research in IED defeat work [24:45] Some distinct and difficult nuances of IED defeat work in Afghanistan and Iraq  [29:45] Meaning and application of “cognitive fidelity” [32:50] A surprising characteristic of young researchers [35:20] One single question that can determine if someone is an NDM practitioner [39:05] Two truths and a lie [39:45] Excitement about research on making mastery models work at a large scale [42:10]

Date: 4/24/2020 Show Description: “Today we are talking with our friend and colleague, Emilie Roth. I first met Emilie at the second Naturalistic Decision Making meeting in Dayton, OH in 1994, but she had been studying how people manage complexity long before that meeting. In fact, I remember at that meeting thinking: ‘I want to be like her.’ Emilie is one of the people that inspired me to think I really could be a scientist. And I know Emilie has inspired many others over the course of her career. She is one of the architects of the cognitive systems engineering movement. She had an important role in redesigning nuclear power plant control rooms in the wake of Three Mile Island. And that was just the beginning; her contributions have been felt far and wide. She now lives in Palo Alto, CA where she runs Roth Cognitive Engineering.” Where to find Emily: Roth Cognitive Engineering Resources: Oxford Handbook of Expertise Designing collaborative planning systems: Putting Joint Cognitive Systems Principles to Practice Cognitive work analysis Facets of complexity in situated work Learn more about NDM: NaturalisticDecisionMaking.org Journal of Cognitive Engineering and Decision Making Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Timestamps: Emily's first published paper [1:30] Events that shifted Emilie’s focus from experimental psychology to NDM [7:30] Developing the observation tools and methods for developing systems of expertise [12:25] Challenges in creating expert decision-making models for nuclear operations at Westinghouse [17:15] Experience navigating a world of engineers as a female psychologist [20:00] The benefits of work culture at Westinghouse [21:45] Observations of railroad industry culture, practices, and informal procedures [23:00] Optimistic outlook on specific research and applications of NDM [28:45] Two things about Emilie’s personal life that others might not know [34:20] Three books that influenced Emilie’s work [38:00] Associative thinking exercise [44:15] Elaborating on automation pros and cons [45:25]

Robert Hoffman, a luminary in the NDM community, talks about his early career, naturalistic decision making, troubling experiences in research, the importance of education in psychology and computer science, love for professional football, and applying cognitive task analysis in sports and policing.

Show Description: “Today we are talking with my friend and business partner, Gary Klein. Gary is one of the pioneers of the Naturalistic Decision Making movement. He is known for moving the study of decision making out of the laboratory and into the wild. In 1989, he invited a small group of international researchers to meet in Dayton, OH to talk about decision making in natural environments and how to study this phenomenon. It was that group of 32 researchers who came up with the term Naturalistic Decision Making. Gary's many contributions include the recognition-primed decision model that has greatly influenced how we all think about decision making and expertise. He and his colleagues developed early cognitive task analysis methods that have been a core set of tools for studying decision making in real world environments. He has mentored countless researchers, including both of us. Gary now lives in Boston, MA. He runs a company called ShadowBox LLC, and is also part-owner and advisor to Applied Decision Science, LLC.” Where to find Gary ShadowBox LLC Website LinkedIn Psychology Today Column Learn more about NDM: Website: https://naturalisticdecisionmaking.org/ Where to find hosts Brian Moon and Laura Militello: Brian’s website Brian’s LinkedIn Brian’s Twitter Laura’s website Laura’s LinkedIn Laura’s Twitter Resources mentioned during the show: Sources of Power by Gary Klein Seeing What Others Don't: The Remarkable Ways We Gain Insights by Gary Klein Timestamps: Gary's first published paper [1:36] How Gary got started in Naturalistic Decision Making [4:48] Gary's research on Israeli aircraft maintainers' coping methods during Yom Kippur War [7:55] Current NDM-related research that Gary is most excited about [12:10] Two of the biggest mistakes of Gary’s career and how he recovered from them [14:53] Gary shares his favorite research stores: -A firefighter whose quick thinking saved several of his colleague’s lives [22:53] -A British naval officer whose quick, life-saving decision was at first mistaken for ESP [28:00] -A neonatal nurse's expert instincts helped save a baby's life [33:33] Suggestions for how the NDM community can help during today's coronavirus crisis 37:15] Where Gary would prioritize funding and research investment right now [44:05] Associative thinking exercise [46:18] How the scientific method is “often a trap” that can limit researchers' learning opportunities [48:55]