Evaluation of Technosocial Predictive Analytics
Jean Scholtz and Mark Whiting
Executive Summary
Our evaluation efforts focus on evaluating the utility that technosocial predictive analytic environments provide to the end users. User-centered evaluations are iterative in nature, both in the definition of those measures and in their application. While the end goal of our work is to develop metrics and methodologies to measure the utility of the complete system, we will accomplish this in a hierarchical manner. Based on overall utility measures developed through contact with potential end users, we will develop metrics and methodologies for assessing the contribution of individual components to the overall utility of the system. We will conduct these initial evaluations and provide feedback to researchers to use in refining their components. As components are integrated into the gaming environment, we will conduct utility evaluations at the systems level. This approach ensures that the individual modeling components, including the gaming environment and the knowledge encapsulation framework, have produced satisfactory measures to support overall utility measures prior to conducting an overall assessment.
Click on image to enlarge.
The development of user-centered evaluation methodologies and metrics for Technosocial Predictive Analytics Initiative will provide researchers with feedback to improve their work and will provide data for decision makers to use in comparing and adopting these new technologies into their processes.