Command Support Research Overview Leonard Adelman C4I Center - PDF document

Command Support Research Overview Leonard Adelman C4I Center Review May 19, 2006 Research Constants • Theory driven, empirical research • Decision processes • Task dependency 1

Agenda • Order Effects: How different ordered sequences of the same information can result in different decisions – With Patriot air defense officers • Time Pressure: How it affects team decision making – With GMU, ROTC cadets • Ongoing research Order Effects • Substantial research investigating order effects • Hogarth & Einhorn (1992): Order effects task dependent • For an air defense task, which (1) is simple for trained personnel, and (2) involves a short series of information, their review & “anchoring & adjustment” (A&A) theory predicted – Recency effect when information was presented sequentially, and a probability estimate was obtained after receiving each piece of information (step-by-step) – No order effect when all information was presented at once (i.e., globally), and the probability estimate was obtained at that time (end-of-sequence) 2

Notional Recency Effect Degree of Belief in Hypothesis 1 0.9 Order 1 (CD) 0.8 Order 2 (DC) (Probabilities) 0.7 Global (EOS) 0.6 0.5 0.4 0.3 0.2 0.1 0 #1 #2 #3 Information Iteam Representative Patriot Track (CD Late Order Sequence) FSCL Mode 3 IFF (F) SPC (C) Jam (D) Stops (C) Leaves SPC & Returns (D) Asset Asset Patriot 3

Results: Recency Effect 0.06 Mean Probability Difference 0.04 0.02 0 -0.02 CD Order -0.04 DC Order -0.06 -0.08 -0.1 -0.12 3 4 5 Information Item Affects Engagements Too • Number of engagements – CD Late Order: 9 – DC Late Order: 5 • But it is not that simple: Different layouts (i.e., contexts) created different effects – If eliminate tight turn and have aircraft leave safe passage corridor farther from Patriot – Get Primacy Effect 4

Primacy Effect: Mean Differences (for friendly aircraft) Using Different Layout 0.1 Mean Probability Difference 0.05 0 CD Order -0.05 DC Order -0.1 -0.15 -0.2 3 4 5 Information Item Primacy Effect Affects Engagements Too • Mean number of engagements (friends) – CD Late Order: 1.25 (previously 9) – DC Late Order: 3.00 (previously 5) • Engagement differences due to primacy effect more extreme for hostiles – CD Late Order: 9.25 mean engagements – DC Late Order: 3.25 mean engagements 5

Order Effects: Conclusion • Order effects can affect engagement decisions • Primacy effect (under-adjustment) depended on explaining away later disconfirming information – Information reinterpretation based on “task” context • Recency effect (over-adjustment) when could not explain away later disconfirming information – Task context where H&E’s model holds • Both depended on human judgment processes and situation-specific, task context – Same results for Patriot teams, not just individuals Team Decision Making Under Time Pressure • Examined how – Increasing levels of time pressure affected the performance of 3-person, ROTC teams – Using different interfaces • Again, theory-driven empirical research examining decision processes as a function of task characteristics 6

“Perceptual-Support” Interface Brehmer & Hagafors (1986) Lens Model Representation of Staff Support LEADER’S ACCURACY STAFF VALIDITY CUE STAFF’S CUE UTILIZATION VALIDITY UTILIZATION OF STAFF d d d Correct d Answer LEADER’S JUDGMENT d d d CUES STAFF JUDGMENTS 7

Len Model Equation r a = GR s R e r a (achievement) Answer (Ye) Judgment (Ys) cues R s R e Predictability Predictability of Ys of Ye G (Knowledge) Hollenbeck et al.’s (1995) Multi-Level Theory of Team Decision Making 8

Path Model Showing Time Pressure’s Effect Total R 2 : 0.988 Adj R 2 : 0.981 -0.521** 0.456* Leader Leader G z Tempo Informity 0.88*** -0.589*** 0.636** -0.495* Decision Accuracy (r az ) Hierarchical (in)Sensitivity 0.700** Staff ( r mz ) (Log) Staff Accuracy -0.510* Informity 0.19*** Cond 2 -0.734** 0.626** Leader R sz Interface Time Pressure: Conclusion • One can map how increasing time pressure causes a decrease in information flow & subsequent team decision processes • The system interface can affect this process, for better or for worse • Again, theory can guide a better understanding of how task characteristics affect individual and team decision processes 9

Ongoing Research • Order effects for a long series of info – Again driven by H&E’s theory of A&A – Affect of task complexity & info grouping • Intuitive versus analytical thought – Driven by “Cognitive Continuum” Theory – Assumption: Individuals’ decision processes are driven by task characteristics Selected References • Adelman, L., Bresnick, T.A., Christian, M. Gualtieri, J., & Minionis, D. (1997). Demonstrating the effect of context on order effects for an Army air defense task using the Patriot simulator. Journal of Behavioral Decision Making , 10, 327-342. • Adelman, L., Bresnick, T.A., Black, P.K., Marvin, F.F., & Sak, S.G. (1996). Research with Patriot air defense officers: Examining information order effects. Human Factors, 38 , 250-261. • Adelman, L., Miller, S.L., & Yeo, C. (2004). Testing the effectiveness of icons in supporting distributed team decision making under time pressure. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans , 34(2) , 179-189. • Adelman, L., Miller, S.L., Henderson, D., & Schoelles, M. (2003). Using Brunswikian theory and a longitudinal design to study how hierarchical teams adapt to increasing levels of time pressure. Acta Psychologica , 112 (2), 181-206. • Adelman, L., Yeo, C., & Miller, S.L. (2006). Understanding the effects of computer displays and time pressure on the performance of distributed teams. In A. Kirlick (Ed.), Adaptive Perspectives of Human-Technology Interaction: NY: Oxford Univ. Press, pp. 43-54. • Brehmer, B., & Hagafors, R. (1986). Use of experts in complex judgment making: A paradigm for the study of staff work. Organizational Behavior and Human Decision Processes, 38 , 181- 195. • Hogarth, R.M., & Einhorn, H.J. (1992). Order effects in belief updating: The belief adjustment model. Cognitive Psychology, 24 , 1-55. • Hollenbeck, J.R., Ilgen, D.R., Sego, D.J., Hedlund, J., Major, D.A., & Phillips, J. (1995). Multilevel theory of team judgment making: Judgment performance in teams incorporating distributed expertise. Journal of Applied Psychology, 80 , 292-316. • Keltz, I., & Adelman, L. (2005). Testing information order effects in a long series of evidence. Proceedings of the 26th National Conference of the American Society of Engineering Management , pp. 483-492. 10