Primary Submitter:
Julie Gore, j.gore@bbk.ac.uk
Domain:
Employment Support (ES). Employment Support advisors help the unemployed find meaningful work.
Tasked to apply the Naturalistic Decision Making framework to understand how highly effective
employment support advisers operate to identify some of the core mindsets, rules of thumb, and
other ‘work-arounds.’
Generic description of sponsoring organization or customer:
UK government department responsible for administering a range of working age, disability, and ill health benefits to around 20 million claimants and customers.
Cognitive Task Analysis Method(s):
Applied Cognitive Task Analysis (Militello & Hutton, 1998); Adapted ACTA and utilized some of the Critical Decision Method (CDM) (Crandall, Klein, & Hoffman, 2006) structured as an intensive incident-based interview protocol.
The use of the Critical Decision Method (CDM; Klein, Calderwood & Macgregor, 1989) approach was also integrated with the Knowledge Audit stage in order to probe for challenges in the ES task and add a timeline of critical decision points (Borders & Klein, 2017, referred to this adaptation as the Cognitive Audit method).
The data set was read multiple times whilst considering issues that appeared to be relevant to the analysis. This process was iterative and was conducted by both authors of this report. A summary of both the Task Diagram and Cognitive Demands of the task were completed via this process. A Cognitive Demands table is used to combine and amalgamate data from all participants across the two or three stages. The table focuses upon: the difficult cognitive element of the task; why this cognition is difficult; what are the common errors; what cues and strategies are used to accomplish the task. This is intended to focus the analysis and identify common themes in the data, establishing tacit expert knowledge, which is not often shared or revealed, Militello and Hutton (1998), and Crandall, Klein and Hoffman (2006).
Number of participants:
Four (one male, three female) voluntarily participated in this study. (80 – 180-minute CTA interviews). The Employment Support (ES) professionals in this sample were recommended by their line managers as high performers/experts.
Duration:
One calendar month.
Application(s):
Task Diagrams, Cognitive demands tables, insight stories and extracts from the
Knowledge audit interviews were produced and shared via a 60-page report.
Demonstration of value:
Evidence of value
The sponsoring organization utilized the findings of this complex sensitive project to reconsider a larger change to the roles and responsibilities of ES professionals as part of an innovative, transformative re-think. The findings were also commended for providing one of the most helpful insights into ES expertise that existed within the organization at that time. This was communicated to the authors of this project and a wider pool of researchers/consultants (looking at the area from different disciplinary perspectives), and staff within the organization via a confidential report and presentation completed by the sponsor of the research.
Findings were shared internally with leading employment experts throughout the organization.
Citations
Crandall, B., Klein, G., and Hoffman R.R. (2006). Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. Cambridge, MA: MIT Press.
DiBello, L., Lehmann, D., & Missildine, W. (2010). How Do You Find an Expert? Identifying Blind Spots and Complex Mental Models Among Key Organizational Decision Makers Using a Unique Profiling Tool. Informed by Knowledge: Expert Performance in Complex Situations, 261.
Cannon-Bowers, J. A., Salas, E., & Converse, S. A. (Eds.). (1993). Shared mental models in expert decision making. Norwood NJ: Ablex.
Gore. J. & McAndrew, C. (2009) Accessing expertise. The Psychologist. Vol 22, 3: 218-219.
Gore, J. & Ward, P. (2018 online in advance) Naturalistic Decision Making and Macrocognition Under Uncertainty: Theoretical & Methodological Developments. Special section for Journal of Applied Memory & Cognition.
Gore, J. and Ward, P. (2017) Eds. Naturalistic Decision Making and Uncertainty. Proceedings of the 13th Bi-Annual Naturalistic Decision Making Conference University of Bath, UK. ISBN 978-0- 86197-194-7.
Gore, J., Flin, R., Stanton, N., & Wong, W. (Eds) (2015). Applications of Naturalistic Decision Making. Special Issue of the Journal of Occupational & Organizational Psychology. 88, 223-386.
Grossman, R., Spencer, J.M. and Salas, E. (2014). Enhancing Naturalistic decision making and accelerating expertise in the workplace: training strategies that work. In S. Highhouse, R. Dalal,, E. Salas.(Eds.). Judgment and decision-making at work. New York: Routledge.
Hoffman, R.R., & Militello, L.G. (2008). Perspectives on cognitive task analysis: Historical origins and modern communities of practice. New York, NY: Taylor & Francis.
Hoffman, R. R., Ward, P., Feltovich, P. J., DiBello, L., Fiore, S. M., Andrews, D. (2014). Accelerated expertise: Training for high proficiency in a complex world. New York, NY: Psychology Press.
Klein, G. A. (1997). Developing expertise and decision making. Thinking and Reasoning, 3, 337– 352.
Klein, G., & Hoffman, R. (2008). Macrocognition, mental models, and cognitive task analysis methodology. In J. M. Schraagen, L. Militello, T. Ormerod, & R. Lipshitz, (Eds.). Naturalistic decision-making and & macrocognition. (pp. 57-80). Ashgate: Hampshire, U.K.
Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in the workplace: development of a taxonomy of adaptive performance. Journal of applied psychology, 85(4), 612.
Koehler, D.J. and Harvey, N. (2004). Expertise in judgement and decision making: a case for training intuitive decision skills in Editors? Blackwell Handbooks of Experimental Psychology: Blackwell Handbook of Judgement and Decision Making. London, Blackwell.
Militello, L.G. & Hutton, R.J.B. (1998). Applied cognitive task analysis (ACTA): A practitioner’s toolkit for understanding cognitive task demands, Ergonomics, 41, 11, 1618-1641. Doi: 10.1080/001401398186108.
Militello, L.G., Wong, W., Kirsechenbaum, S.K., and Patterson, E. (2010). Systematizing discovery in cognitive task analysis. In Mosier, K.L. & Fischer, U.M. (Eds.) Informed by knowledge: Expert performance in complex situations: 23-40. New York, NY: Taylor and Francis.
Phillips, J. K., Klein, G., & Sieck, W. R. (2004). Expertise in judgment and decision making: A case for training intuitive decision skills. Blackwell Handbook of Judgment and Decision Making, 297, 315.
Salas, E., Rosen, M.A. & Diaz-Granados, D. (2015) Naturalistic decision making in individuals and teams. In S.W.J. Kozlowski (Ed). Oxford Handbook of Industrial and Organizational Psychology. New York: Oxford University Press.
Schraagen, J.M., Chipmam, S.F. & Shalin, V.L. (2000) Cognitive Task Analysis. Psychology Press.
Schraagen, J.M., Militello, L., Ormerod, T., & Lipshitz, R. (2008). (Eds), Naturalistic decision making and macrocognition. Aldershot, UK: Ashgate Publishing Limited.
Tofel-Grehl, C., & Feldon, D.F. (2013). Cognitive task analysis-based training: A meta analysis of studies. Journal of Cognitive Engineering and Decision Making 7 : 293-304.
Ward, P., Gore, J. Hutton, R. J. B., Conway, G. & Hoffman, R. R. (2018) Adaptive Skill as The Sine Qua Non of Expertise. Journal of Applied Memory and Cognition.
