CogSIMA 2016

Tutorial 1: Semantically Aware Uncertainty Management for Situation Awareness and Decision Support

Instructor: Dr. Kathryn Blackmond Laskey, George Mason University, USA
Date: Monday, March 21, 2015
Time: 9:00 am - 12:30 pm

Techniques for making semantic information explicit and computationally accessible are key enablers for systems to support situation awareness and decision-making. Most ontology languages have their logical basis in some fragment of first-order logic. According to Sir Harold Jeffreys, "traditional or deductive logic admits only three attitudes to any proposition: definite proof, disproof, or blank ignorance." Real-world situations are characterized by noisy, incomplete, and ambiguous evidence that falls far short of logical proof, yet nonetheless provides substantial information on which to base decisions. This tutorial discusses how to combine semantic technology in a principled way with reasoning under uncertainty and learning from observation. A number of highly expressive probabilistic languages have been developed over the past two decades. These languages can be given a semantics that relates in a natural way to the semantics of traditional first-order logic. These probabilistic logics naturally represent degrees of plausibility intermediate between proof and disproof, support evidential reasoning and learning from observation, and thus provide a mathematically rigorous foundation for probabilistic ontologies. The tutorial introduces the PR-OWL probabilistic ontology language, an OWL upper ontology based on the MEBN probabilistic logic. PR-OWL allows an ontology engineer to incorporate probabilistic knowledge into ontologies written in the widespread, W3C recommended OWL ontology language. As an ontology language, PR-OWL supports logical as well as probabilistic reasoning. Thus, PR-OWL ontologies can represent the knowledge needed to represent and reason about complex situations. The tutorial includes a hands-on demonstration of how to construct a probabilistic ontology from scratch and from an existing ontology. The Java-based, open source graphical probabilistic editor UnBBayes-MEBN will be used to illustrate the basic aspects of PR-OWL.

Kathryn Blackmond Laskey, Ph.D., is Associate Professor of Systems Engineering and Operations Research at George Mason University and Associate Director of the Center of Excellence in Command, Control, Communications, Computing and Intelligence (C4I Center). She teaches and performs research on computational decision theory and evidential reasoning. Professor Laskey's research involves methods for representing knowledge in forms that can be processed by computers, extending traditional knowledge representation methods to represent uncertainty, eliciting knowledge from human experts, applying probability theory to draw conclusions from evidence arising from multiple sources. She is a leader in the application of Bayesian methods to automated support for multi-source fusion and situational awareness. She has developed methods for knowledge-based construction of problem-specific Bayesian belief networks, specifying Bayesian belief networks from a combination of expert knowledge and observations, and flexible adaptation of models to anomalous situations. She has applied her research to problems such as modeling the emplacement of improvised explosive devices, predicting aircraft delays, managing terrorist risk at public facilities, judicial reasoning, and planning military engagements. Professor Laskey developed multi-entity Bayesian networks (MEBN), a language and logic that combines the expressive power of classical first-order logic with the uncertainty handling capability of Bayesian inference. As an invited expert to the World Wide Web Consortium (W3C), Dr. Laskey co-chaired the Uncertainty Reasoning for the World Wide Web Experimental Group (URW3-XG), a group devoted to identifying aspects of uncertainty that need to be standardized for web-based applications. She is co-organizer of the Uncertainty Reasoning for the Semantic Web series of workshops at the International Semantic Web Conference, and the Semantic Technology for Defense, Intelligence and Security Conference. Professor Laskey serves on the Board of Directors of the Association for Uncertainty in Artificial Intelligence. She served on several boards and committees of the National Academy of Sciences.

Tutorial 2: Designing to Enhance Situation Awareness

Instructor: Dr. Mica R. Endsley, SA Technologies, Inc., USA
Date: Monday, March 21, 2015
Time: 9:00 am - 12:30 pm

Situation awareness (SA) is a fundamental construct driving human decision making in a complex, dynamic environments. Achieving adequate situation awareness is one of the most central and difficult portions of many decision makers' jobs, indicating that significant assistance and design work is needed. An examination of the underlying factors associated with situation awareness provides an integrated perspective for addressing the system design issue. This perspective focuses on the means by which the human operator maintains an on-going representation of the state of the environment. By creating designs that enhance an operator's awareness of what is happening in a given situation, decision making and performance can improve dramatically.

The objective of this tutorial is to provide attendees with required background knowledge on theoretical aspects of situation awareness, including cognitive constructs that play a role in achieving it in complex domains, and, based on this model, to provide an approach for supporting SA through system design initiatives. This design approach supplements other human factors analysis techniques, design guidelines and measurement approaches to provide a critical linkage to the cognitive foundation of performance in systems operations — the ongoing representation of the current situation that drives decision making and behavior aimed at keeping that environment aligned with operator goals. This approach is aimed not at system design at the component level, so much as system design at an overall level of integration. The SA-Oriented Design process is a means to improve human decision-making and performance by optimizing situation awareness. This method has been used to develop and evaluate system design concepts in aviation, medical and information intelligence operations. It features three main components: SA Requirements Analysis, SA-Oriented Design Principles, and SA Measurement and Validation. The design process is user-centered, and derived from a detailed analysis of the goals, decisions and situation awareness requirements of the operator derived through a Cognitive Task Analysis methodology called Goal-Directed Task Analysis. In this analysis, the major goals of a particular job are identified, along with the major subgoals necessary for meeting each of these goals. Associated with each subgoal, the major decisions that need to be made are then identified. The situation awareness needed for making these decisions and carrying out each sub-goal are then delineated. These SA requirements focus not only what data the operator needs, but also on how they are integrated or combined to address each decision. The higher levels of SA— comprehension (Level 2 SA) and projection (Level 3 SA) — are emphasized in this analysis, each a function of the decision requirements associated with each goal. This is critical for feeding the design effort.

Design principles for enhancing situation awareness through system design will be presented based on theoretical model of human situation awareness. These principles form the foundation linking SA requirements to good system design within a domain. It provides the foundation for dealing with system design on the basis of information content, rather than just at a surface feature level (e.g. colors, symbology type, size of font, etc…). Design guidelines for supporting SA based on a model of how people gain, maintain and in some cases lose SA is tied to this critical foundation, providing the basis for transforming SA requirements into systems that will deliver that SA effectively. Fifty design principles have been developed for addressing the development of systems to support SA in a wide variety of challenging domains. They include principles for dealing with system complexity, conveying levels of confidence and uncertainty associated with information, the design of alarms and advanced automation concepts, and supporting SA in multi-operator and distributed team environments. These SA-oriented design principles are applicable to a wide variety of system designs. The principles have been successfully applied to systems involving remote maintenance operations, medical systems, military command and control, and flexible manufacturing cells.

Finally methods for measuring objectively situation awareness will be presented. Measurement of situation awareness during iterative test and evaluation is critical for insuring that system design concepts are successful in meeting the desired objectives. Many concepts and technologies are currently being developed and touted as enhancing SA. Prototyping and simulation of new technologies, new displays, and new automation concepts is extremely important for determining the actual effects of proposed concepts within the context of the task domain. If SA is a design objective, then it is critical that it be specifically evaluated during the design process. Without this, it will be impossible to tell if a proposed design actually helps SA, does not effect it, or inadvertently compromises it in some way. The tutorial will incorporate examples from a wide variety of domains, including aviation, air traffic control, command and control, driving, and maintenance. The key tenets of the SA-Oriented Design process will be demonstrated through examples from each of these areas. Audience discussion and participation will be emphasized into to insure that attendees are able to address the application of the material presented to their domains of interest.

Mica Endsley is President of SA Technologies, a cognitive engineering firm and is the former Chief Scientist for the US Air Force. She is Past-President of the Human Factors and Ergonomics Society. She received a Ph.D. in Industrial and Systems Engineering from the University of Southern California. Dr. Endsley is a recognized world leader in the design, development and evaluation of systems to support human situation awareness (SA) and decision-making. She has authored over 200 scientific articles and is the co-author of "Designing for Situation Awareness".

Tutorial 3: Context-enhanced Information Fusion

Instructor: Dr. Lauro Snidaro, University of Udine, Italy
Date: Monday, March 21, 2015
Time: 1:30 am - 5:00 pm

Contextual Information (CI) can be understood as the information that "surrounds" an observable of interest. Even if not directly part of the problem variables being estimated by the system, CI can influence their state or even the sensing and estimation processes. Therefore, understanding and exploiting CI can be a key element for improving the performance of Information Fusion algorithms and automatic systems in general that have to deal with varying operating conditions. There is a growing interest for this promising research topic that should be considered for developing next-generation Information Fusion systems. Context can have static or dynamic structure, and be represented in many different ways such as maps, knowledge-bases, ontologies, etc. It can constitute a powerful tool to favour adaptability and boost system performance. Application examples include: context-aided surveillance systems (security/defence), traffic control, autonomous navigation, cyber security, ambient intelligence, ambient assistance, etc. The purpose of this tutorial is to survey existing approaches for context-enhanced information fusion, covering the design and development of information fusion solutions integrating sensory data with contextual knowledge. After discussing CI in other domains, the tutorial will focus on context representation and exploitation aspects for Information Fusion systems. The applicability of the presented approaches will be illustrated with real-world context-aware Information Fusion applications.

- Representation and exploitation of contextual information at different levels of an Information Fusion system - Managing of heterogeneous contextual sources
- Adaptation techniques to have the system respond not only to changing target's state but also to the surrounding environment
- Architectural issues and possible solutions
- Augmentation of tracking, classification, recognition, situation analysis, etc. algorithms with contextual information

- Fundamental concepts of context for Information Fusion
- Design patterns for context-aware fusion systems
- Mathematical aspects and algorithms
- Examples and applications

computer scientists, students, Information Fusion practitioners

Dr. Lauro Snidaro received his MSc and PhD in Computer Science from University of Udine in 2002 and 2006 respectively. Since 2008 he is an Assistant Professor at the Dept. of Mathematics and Computer Science, Univ. of Udine. His main interests include Data/Information Fusion, Computer Vision, Pattern Recognition, Machine Learning, Multimedia, Video understanding and annotation. He actively publishes in international journals and conferences and has co-authored more than 80 papers. He cooperates with international industries and research centres and is involved in several international projects on multi-sensor data fusion. He has been invited speaker at international symposiums and meetings in Armenia, Bulgaria, Spain, and United States. In 2011 he was program co-chair of the 16th International Conference on Image Analysis and Processing (ICIAP). He has co-organized several special sessions on context-enhanced Information Fusion at the International Conference on Information Fusion and the IEEE International Conference. He is special issue organizer and "Fusion for Signal/Image Processing and Understanding" Area Editor of the Information Fusion journal (Elsevier). He is lead editor of the forthcoming Springer book "Context-enhanced Information Fusion - Boosting real world performance with domain knowledge". He serves as IT project reviewer for the Italian Ministry of University and Research and EU research programmes. He regularly serves as reviewer for 25 international journals and 25 conferences.

Tutorial 4: Knowledge Level Models of Situations and Situation Assessment: Theory and Applications

Instructor: Dr. Odd Erik Gundersen, Norwegian University of Science and Technology, Norway
Date: Monday, March 21, 2015
Time: 1:30 am - 5:00 pm

The concept of a situation is central to the situation management community, Still, the concept is not clearly defined and often research on the topic rarely refer to the definition of situation that the research is based on. This is probably because the concept of a situation is elusive and a common definition has not been agreed on. The tutorial will start with an introduction to different definitions of the concept of a situation, so that the audience can get a better grasp of complexities related to defining situations. The definitions will be analyzed and a definition that bring these together will be discussed. A formal framework for describing situations based on the proposed definition will presented. The framework will be used to discuss situations that develop over time, observers and situations, future situations, the concept of history, explanations and justifications of predictions. The three sub-processes of situation assessment is described in Endsley's three situation awareness levels [1]. However, while the levels are given detailed descriptions, they are not related directly to any definition of the concept of a situation. By analyzing the concept of a situation in relation to the situation awareness levels, the situation awareness levels can be decomposed further into more fine grained levels. These fine-grained levels are described as a knowledge level model [2] of situation assessment that includes a task tree, a goal analysis and detailed task description. The knowledge level model is used to enhance and detail Endsley's three levels of situation awareness model. The concept of context is an elusive one as well, which Bazire and Brezillon [3] document in this paper where they analyze 150 definitions of context found on the web. The tutorial will present some definitions of context from the context research community. These definitions will be discussed and analyzed in relation to the concepts situation and situation assessment. Furthermore, the role of context in situation assessment will be analysed, and a context element ontology will be presented. A formal framework for describing situation assessment will be presented. Examples of how the knowledge level model and the formal framework can be used to analyze domains will be given. Two different use cases will analyzed using the methods presented, and these are oil well drilling and emergency services. Both use cases are interesting because the situations are assessed remotely by the observers the analyses focus on. The drilling engineers assess the situation in the wellbore, which can be 10 000 feet below the drilling rig, indirectly through interpreting real-time measurements. Emergency call centre operators assess situations through callers describing emergencies over the phone. The examples include situation descriptions, task and goal analyses, a discussion of context elements and the role of context.

This tutorial has six learning goals for the audience, which are: (i) Get a deeper understanding of the concept situation, (ii) understand how the definition of situation affects and allows a more detailed description of situation assessment, (iii) get a more evolved understanding of the concept context when related to situation assessment, (iv) learn a formal framework for describing situations, (v) learn how to apply the formal framework on real-world examples, and finally, (vi) learn how the framework can be used when developing decision support software.

The intended audience is everyone that is interested in situation management, as the tutorial provides an introduction and overview to central concepts in situation awareness and situation assessment. However, the knowledge level model is most suited for researchers and software developers that are interested in developing decision support systems for situation assessment and need to model situations.

The tutorial is based on experience in developing commercial real-time decision support systems for oil well drilling and emergency call centres. The decision support systems have been developed to enhance the drilling engineers situation awareness of the situation in the wellbore and the emergency call centre operators awareness of emergency situations, and they have been commercially deployed and used operatively. The research related to the decision support system for oil well drilling is documented in the following papers [4], [5], [6], [7] and the emergency call centre decision support system is documented in the following paper [8] that will be submitted to Cogsima 2016. Some of the work presented here is taken from a forthcoming journal paper.

Odd Erik Gundersen is currently the Chief Operating Officer of MazeMap AS and a Scientific Researcher at the Norwegian University of Science and Technology (NTNU). He got his Masters degree in computer science from NTNU in 2004 and his doctorate, also in computer science from NTNU, in 2014. Dr. Gundersen has worked full-time in the software industry since 2007 where he has developed commercial decision support systems utilizing artificial intelligence and machine learning methods for oil and gas, healthcare, financial services and emergency services. Since graduating with his Masters degree, he has held positions in academia ranging from research assistant, university lecturer, associate professor and now scientific researcher. Gundersen is an author of more than 30 scientific peer-reviewed papers, among them seven journal papers, and he has supervised more than 20 master students. His research includes decision support, machine learning methods, temporal reasoning and computer graphics. He is listed as an inventor of three granted patents and five patent applications.

[1] M. R. Endsley, "Toward a Theory of Situation Awareness in Dynamic Systems," Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 37, pp. 32-64(33), 1995. 


[2] A. Newell, "The Knowledge Level," Artificial Intelligence, vol. 18, no. 1, pp. 87-127, 1982. 


[3] M. Bazire and P. Brezillon, "Understanding Context Before Using It," in 
Proceedings of the 5th International Conference on Modeling and Using Context, ser. CONTEXT'05. Berlin, Heidelberg: Springer-Verlag, 2005, pp. 29-40. [Online]. Available: http://dx.doi.org/10.1007/11508373 3 


[4] O. E. Gundersen, "The Role of Context and its Elements in Situation Assessment," in Context in Computing, P. Brezillon and A. J. Gonzalez, 
 Eds. Springer New York, 2014, pp. 343-357. 

[5] O. E. Gundersen, Enhancing the Situation Awareness of Decision Makers by Applying Case-Based Reasoning on Streaming Data, ser. Doctoral theses at NTNU;2014:315. NTNU, 2014. 


[6] O. E. Gundersen, F. Sørmo, A. Aamodt, and P. Skalle, "A Real-Time Decision Support System for High Cost Oil-Well Drilling Operations," AI Magazine, vol. 34, no. 1, pp. 21-32, 2013. 


[7] O. E. Gundersen, "Situational Awareness in Context," in CONTEXT, ser. Lecture Notes in Computer Science, P. Bre ́zillon, P. Blackburn, and R. Dapoigny, Eds., vol. 8175. Springer, 2013, pp. 274-287. 

[8] O. E. Gundersen and F. Øvergård, "Towards Reducing the Reaction Time of Emergency Service through Improved Situation Assessment," submitted to Cogsima 2016.