In this paper we introduce a conceptual framework for the design of automated team evaluation processes (FATE), inspired by lessons learned from multiple intelligent team tutoring experiences. The framework consists of five phases. The first, Team Construct, defines the theoretical basis of the evaluation and therefore the end goal of the evaluation process. The second, Behavioral Markers, defines a method for identifying the otherwise unobservable constructs. The third, Raw Data, defines the data to be captured and recorded. The fourth, Enriched State Representation, defines a method for making the data directly relevant for team evaluation. The fifth, Team Metric, is the end goal of the evaluation defined by team constructs and derived from the enriched state representation. The framework is organized in a “V” shape to act both as a hierarchical model relating teaming theory to scenario-specific data and as a sequential process flow diagram representing the steps recommended to design an ideal team evaluation process. Each phase of the framework is described in detail, and its use is demonstrated with a hypothetical emergency response training scenario.
Available at: http://works.bepress.com/stephen_b_gilbert/81/
This proceeding is published as Ostrander, Alec, Stephen Gilbert, and Michael Dorneich. "Team Data Analysis Using FATE: Framework for Automated Team Evaluation." In Workshop Proceedings: Approaches and Challenges in Team Tutoring. Proceedings of the Approaches and Challenges in Team Tutoring Workshop held in conjunction with the 20th Artificial Intelligence in Education Conference (AIED 2019). Chicago, IL, USA, June 29, 2019. (Anne M. Sinatra and Jeanine A. DeFalco, eds.) (2019): 5-14. Posted with permission.