Agent-based modeling and simulation of emergent behavior in air transportation
© Bouarfa et al.; licensee Springer. 2013
Received: 11 October 2012
Accepted: 5 March 2013
Published: 15 August 2013
Commercial aviation is feasible thanks to the complex socio-technical air transportation system, which involves interactions between human operators, technical systems, and procedures. In view of the expected growth in commercial aviation, significant changes in this socio-technical system are in development both in the USA and Europe. Such a complex socio-technical system may generate various types of emergent behavior, which may range from simple emergence, through weak emergence, up to strong emergence. The purpose of this paper is to demonstrate that agent-based modeling and simulation allows identifying changed and novel rare emergent behavior in this complex socio-technical system.
An agent based model of a specific operation at an airport has been developed. The specific operation considered is the controlled crossing by a taxiing aircraft of a runway that is in use for controlled departures. The agent-based model includes all relevant human and technical agents, such as the aircraft, the pilots, the controllers and the decision support systems involved. This agent-based model is used to conduct rare event Monte Carlo (MC) simulations.
The MC simulation results obtained confirm that agent based modeling and simulation of a socio-technical air transportation system allows to identify rare emergent behavior that was not identified through earlier, non-agent-based simulations, including human-in-the-loop simulations of the same operation. A typical example of such emergent behavior is the finding that alerting systems do not really reduce the safety risk.
Agent based MC simulations of commercial aviation operations has been demonstrated as a viable way to be evaluated regarding rare emergent behaviour. This rare emergent behaviour could not have been found through the more traditional simulation approaches.
KeywordsAgent-based modeling and simulation Complex socio-technical systems Air transportation Airport operations Safety risk analysis
The Air Traffic Management (ATM) community is continuously seeking to improve ATM system performance and identify best practices (PRC & ATOS PBU 2009). Within the Single European Sky ATM Research program (SESAR), a performance framework for the future European ATM system has been proposed (SESAR Consortium 2006). This framework aims at improving ATM system performance, in particular with regard to the Key Performance Areas (KPAs) safety, capacity, cost efficiency, and environment. These KPAs are important to accommodate the expected air traffic growth and respond to user’s concerns (Bouarfa et al. 2012; Fron 2001). As part of this framework, airport performance is considered to have an impact on the entire ATM system performance, as airports make up the fixed nodes on which the ATM system is built (Ball et al. 2007; PRC 2009).
Civil air transportation is an example of a complex socio-technical system. Each airport comprises of interactions between a variety of facilities, users, technical systems, human resources, rules, and procedures, and is embedded in a large network of other airports, multiple airlines, and ATM centers. This type of complex socio-technical systems is characterized by a large number of interconnected parts, the difficulty to predict the behavior, and the existence of many different stakeholders (Forrester 1971;6061). Moreover, in civil air transportation, different parties are involved in the operation of an airport with varying boundaries of responsibility per region or country. At Amsterdam Airport Schiphol for instance, the parties involved in air transport related operations include the airport operator, the airport authority, the ministry of infrastructure and environment, the slot coordinators, the air navigation service provider, the airlines, as well as the general public (Deregee 2006). All these stakeholders have a certain viewpoint on performance which adds to the airport complexity (Bouarfa et al. 2012). In addition, each airport is characterized by a number of runways, taxiways, navigational aids, stop-bars, markings, and so on. Technology plays a central role in the airport system as does the social context within which the system is operating. This makes major airports by their very own nature complex socio-technical systems.
Past experience has shown that improving airport and ATM performance is very difficult to achieve. For example, Bar-Yam (Bar-Yam 2005) discusses the failure of design and implementation of the Advanced Automation Systems (AAS) in the previous century. A centerpiece of AAS was the replacement of the air traffic control system near airports. This process faced so many problems in terms of cost overruns, program delays, and safety issues, that it could only be partially completed after an FAA emergence decree. Bar-Yam argues that due to the level of complexity, different parts of the AAS design are so interdependent that changes in one part may have unforeseen effects on other parts.
Holland (Holland 2006) argues that there are few points through which the behavior of complex socio-technical systems can be changed to a desired state. These points named “lever points” are not where designers typically expect them to be. They are points where an action has an amplifier effect on the entire performance. If these points can be discovered, designers can make best use of them. There is however no theory that tells where or how to look for these points (Holland 2006).
Airport operators are faced with different trade-offs every day. These trade-offs are created by the complexities inherent to the processes managed and the finite resources of operational systems (Hollnagel 2009). Potentially, there are conflicting goals leading to dilemmas and bottlenecks that must be dealt with. An example would be how to make sure that a new environmentally-oriented procedure is safe without putting aircraft and people at risk? The widely established system engineering approach has not been developed to capture the socio part of a socio-technical system well. Then it should not come as a surprise when this creates unforeseen behavior that goes unnoticed during the development and implementation of a complex socio-technical system. Instead, such socio-technical behavior should be identified early on in the development process. This requires an approach to identify emergent behavior early on in the development of changes in civil air transportation operations.
Emergent behavior by an air transportation system results from interactions between the various human operators, technical systems, and procedures. The emergence concept is central to complex socio-technical systems and refers to how collective properties arise from the properties of the parts. Examples in air transportation include the impact on air traffic safety by new air traffic control techniques such as time-based spacing, the impact on airport capacity or airline business by policies restricting airport noise, the propagation of delays through the air transportation network, or the consequences that multiple actors can have when coordinating together. The Tenerife airport disaster for instance resulted from an interruption of routines among aircraft crew and air traffic control, loss of communication accuracy, and low visibility conditions (Weick 1990). At the same time, one should be aware of the fact that current aviation also works thanks to the explicit use of emergent behavior for the better. Examples of this are the various control loops that are working in current ATM, within each aircraft itself and also those formed by the interplay between the aircraft crew and each ATM center on its path. And there is no doubt that the role of control loops will only increase for advanced ATM. Logically, one should expect that emergent behavior that is not well understood is often characterized by poor performance, missed opportunities, and the inability to quickly adapt to disturbances. Only once emergent behavior is well understood, it may be exploited for the better.
In (Shah et al. 2005), it is well explained that the key difficulty of evaluating advanced ATM operations is to address emergent behavior, i.e. behavior which emerges from the combined dynamic actions and reactions by individual systems and humans within the overall ATM system. This emergent behavior cannot be foreseen and evaluated by examining the individual behaviors alone. To understand the behavior of a complex socio-technical system we must understand how the parts act together to form the behavior of the whole (Bar-Yam 2003). It is the goal of this research to study the emergence concept in air transportation, and develop a deeper understanding of system-wide performance issues and/or benefits arising from a change in design. In this paper we aim to identify key lever points through which the safety of runway crossing operations at airports can be significantly affected. Our approach is to embrace Agent-Based Modeling and Simulation (ABMS) because it has been extensively used to: a) analyze complex socio-technical systems and their emergent behavior (Shah et al. 2005; Chen & Cheng 2010; Stroeve et al. 2003; Wolfe et al. 2009); and b) address cases where agents need to collaborate and solve problems in a distributed fashion (Klein et al. 2004). ABMS provides a platform to integrate multiple heterogeneous components at different levels. Models of actors, technological systems, and the operating environment as well as the interactions between them can be naturally covered. Therefore, it is expected that an ABMS approach will help: a) predicting airport system-wide behavior emerging from the interactions between individual components; b) identifying and understanding key lever points; and c) managing dependencies between the activities of multiple actors both at the organizational and operational level.
This paper is organized as follows. The next section shortly discusses different perspectives on emergent behaviour, and identifies examples in the air transportation domain using a comprehensive taxonomy. This is followed by the methodology section where the agent-based modeling approach used is described. Then, the subsequent section positions the runway crossing application in the broader context of agent-based modeling. Here, the main agents, their entities, as well as their interactions are described. The Monte Carlo Simulation results and conclusion are provided in the subsequent sections. A short version of this paper has been published in (Bouarfa et al. 2013).
Emergent behavior in air transportation
They are robust and resilient: There is no single-point of failure, so if a single unit fails, becomes lost or is stolen, the system still works.
They are well-suited to the messy real world: Human-engineered systems may be ‘optimal’ but often require a lot of effort to design and are fragile in the face of changing conditions. Importantly, they don’t need to have complete knowledge/understanding to achieve a goal.
They find a reasonable solution quickly and then optimize: In the real world, time matters because decisions need to be taken while they are still relevant. Traditional computer algorithms tend to not produce a useful result until they are complete (which may be too late in case of avoiding an obstacle for instance).
Different perspectives on emergence
Philosophers have long been interested in the concept of emergence, and especially in trying to establish a common definition for this vague yet very useful concept. The term has been in use since at least the time of Aristotle (Wikipedia 2012b) who referred to emergence as “the whole is something over and above its parts, and not just the sum of them all”. Two thousand years later, (Mill 1872) used the example of water to illustrate the same idea. The term emergent was said to be coined by Lewes in his multi-volume problems of life and mind (Lewes 1874–1879). Lewes argued that certain phenomena produce “qualitative novelty”, or material changes that cannot be expressed in simple quantitative terms. Quoting Lewes: “The emergent is unlike its components insofar as these are incommensurable, and it cannot be reduced to their sum or their difference.” (Casti 1997), like Lewes associates emergence with dynamic systems whose behavior arises from the interaction among its parts and cannot be predicted from knowledge about the parts in isolation. Crick explains in (Crick 1994) that: “The scientific meaning of emergent, or at least the one I use, assumes that, while the whole may not be the simple sum of its separate parts, its behavior can, at least in principle, be understood from the nature and behavior of its parts plus the knowledge of how all these parts interact”.
(Bedau 1997) distinguishes between two types of emergence namely, strong and weak emergence. Strong emergence was defined as nominal emergence in which the emergent properties are supervenient properties with irreducible downward causal powers. In the second type “weak emergence,” the system’s global behavior derives from the operation of micro-level processes, but the micro-level interactions are interwoven in such a complicated network that the global behavior has no simple explanation. Bedau argues that ‘strong’ emergence has had a prominent place in the philosophical discussions but that its scientific credentials are very poor, whereas ‘weak emergence’ is consistent with materialism and scientifically useful. Bedau proceeds to defend one version of weak emergence (noting that there are other versions), which is: “A nominally emergent property of a locally reducible system is called weakly emergent if it is derivable from all of the micro facts of this system, but only by simulation.”
(Chalmers 2002) includes a notion of “unexpectedness” or “surprise” to the definition of emergence, when providing alternative definitions for strong and weak emergence. Other authors also refer to the notion of surprise, like (Sanz 2004) who defines emergence as “just systemic behavior — nothing more, nothing less— that is difficult to predict in advance.” (Bedau 2002) explains that he left the notion of surprise absent on purpose, due to it being rather subjective. Instead, Bedau claims that with his definition of weak emergence in terms of simulation he is presenting objectivist approaches to emergence, though he notes that his classification is not exhaustive.
The diverse writings on emergence show that the term emergence captures a broad spectrum of system behavior. (Goldstein 1999) notes that “emergence functions not much as an explanation but rather as a descriptive term pointing to the patterns, structures, or properties that are exhibited on the macro-scale”. Editor Lissack acknowledged in his inaugural article on emergence and complex systems theory that “it is less than an organized, rigorous theory than a collection of ideas that have in common the notion that within dynamic patterns there may be underlying simplicity that can, in part, be discovered through large quantities of computer power and through analytical, logical, and conceptual developments.”
Identifying emergence in air transportation
Type of emergence
Type of feedback
Nominal or Simple Emergence
Predictable in principle
Multiple feedback loops
Not Predictable (Chaotic)
Multiple feedback loops
Not Predictable, even in principle
Type I – Nominal or simple emergence
This type of emergence is totally predictable due to the controlled and planned interaction of the individual components. A machine for instance has a function which is different from the function of its parts, but the overall function is well-known. In air transportation, one can think of the multitude of technical systems either on-board the aircraft or on the ground. There are no unpredicted or unexpected behavior patterns in these systems.
Type II – Weak emergence
This type of emergence describes emergence with top-down feedback, which is predictable in principle, but not in every detail. The roles of the elements and agents are flexible. Coherent global structures appear and become visible on a higher level of organization through the local interaction of several autonomous agents. The top-down feedback from the group imposes constraints on the local interactions. Examples from air transportation include a sequence of flying aircraft, which limits the possible speed adjustments of individual aircraft. The agents (e.g. flight crew) adjust their behavior and their role in the group according to the actual context and situation (e.g. following an ATC instruction, or TCAS warning). Feedback from the environment or group to the agent is possible through this form of context dependency.
Type III – Multiple emergence
This type of emergence is characterized by multiple positive and negative feedback loops appearing in complex systems with many agents. The behavior is not predictable and can be chaotic. Completely new roles can appear while old ones disappear. Although air transportation operations are feasible thanks to such feedback loops, chaotic behavior may arise following some exceptional events. There are two categories of such exceptional events: 1) catastrophic accidents involving one or two aircraft; and 2) events that push the dynamics of the air transportation system far away from its point of operation and therefore dramatically affect the performance of the system. Examples of the latter are a terror action causing closing down of air travel in large areas (e.g. 9/11 in 2001), a disease causing passengers to change their travel behaviour (e.g. SARS in 2003) or volcanic ashes blocking air travel in a large area (e.g. Iceland volcano in 2010). Examples of the former are fatal runway incursions (e.g. Linate runway collision in 2001), fatal mid-air collisions (e.g. Ueberlingen mid-air in 2002), loss of control of an aircraft flying through a hazardous weather system (e.g. Air France crash in Atlantic Ocean in 2009).
Type IV – Strong emergence
This type of emergence is not predictable, even in principle, because it describes the appearance of a completely new system in a multi-level or multi-scale system with many levels. Combinatorial explosion renders any attempt of explaining emergent macroscopic phenomena in terms of microscopic low-level phenomena useless and futile. An intermediate or mesoscopic level often protects the macroscopic level from the microscopic level, i.e. the microscopic level is irrelevant to the behavior of the macroscopic level. Life is a strongly emergent property of genes, genetic code and nucleic/amino acids, as is culture a strongly emergent property of language and writing systems. In the air transportation domain, one can think of the safety culture which is described as the product of routine aspects of everyday practice as well as organizational structure and rules (Leveson et al. 2005; Ek et al. 2007). However the causal relations are not yet understood (Sharpanskykh & Stroeve 2011).
In order to understand complex socio-technical systems and their emergent behavior, rigorous models are needed that allow us exploring their properties (Holland 2006). One of these properties, is what Holland (Holland 2006) calls ‘lever points.’ These are points where a simple intervention causes an important effect on system performance. According to Holland, all complex systems that have been studied carefully exhibit such points. In the same article, Holland suggests using ABMS to study and analyze complex systems and capture their emergent behavior.
ABMS and complex socio-technical systems
The agent-based modeling paradigm is increasingly recognized as a powerful approach to model and simulate complex socio-technical systems exhibiting emergent behavior (Holland 1997). This is because it can represent important phenomena resulting from the characteristics and behaviors of individual agents and their interactions (Railsback & Grimm 2012). (Burmeister et al. 1997) discuss the benefits of using an agent-based approach in domains that are functionally or geographically distributed into autonomous subsystems, where the subsystems exist in a dynamic environment, and the subsystems have to interact more flexibly. According to Burmeister, agent-based modeling can be used to structure and appropriately combine the information into a comprehensible form. For a large complex system such as a traffic system, they provide the tools for analyzing, modeling, and designing the whole system in terms of its subsystems, each with its own set of local tasks and capability. The integration can then be achieved by modeling the interactions among the subsystems. So agent-based modeling provide abstraction levels that make it simpler and more natural to deal with the scale and complexity of problems in these systems. Agent components can be described at a high level of abstraction, yet the resulting systems are very efficient (Burmeister et al. 1997). (Burmeister et al. 1997) conclude that agent-based modeling reduce the complexity in systems design by making available abstraction levels that lend themselves to a more natural way of modeling the problem domain. They enhance the robustness and adaptivity of systems by virtue of increasing the autonomy of subsystems and their self-organization. In the same vein, Jennings (Jennings 2000) outlines that ABMS and complex system development requirements are highly compatible. (Jennings 2000) shows that agent-based modeling techniques are particularly well suited to complex systems because: a) they provide an effective way of partitioning the problem space of a complex system; b) they provide a natural means of modeling complex systems through abstraction; and c) they capture the interactions and dependencies.
Agents in air transportation
In the ABMS domain, although there is no widely agreed definition for an agent, there is general consensus that autonomy is central to the notion of agency. (Wooldridge 2009) explains that part of the difficulty is that beyond this autonomy point various attributes associated with agency are of different importance for different domains.
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors (Russel & Norvig 2006).
An autonomous agent is a system situated within a part of an environment, which senses that environment and acts upon on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future (Franklin & Graesser 1997).
An agent is a computer system that is situated in some environment, and that is capable of autonomous action in this environment in order to meet its delegated objectives (Wooldridge 2009).
An agent is a system with the following properties (Tessier et al. 2002):
▪ It lives in an artificial world W
▪ It has facilities to sense W and to manipulate W
▪ It has a (at least partial representation of W
▪ It is goal-directed, and as a consequence it has the ability to plan its activities
▪ It can communicate with other agents
In the context of air transportation, in particular where different actors, hardware, and software are interacting elements of a complex socio-technical system, we consider agents as autonomous entities that are able to perceive and act upon their environment. These agents may be humans, systems, organizations, and any other entity that pursues a certain goal. For instance, an air traffic controller can be viewed as an agent observing his/her environment (displays, alerting systems, runway availability, etc.) and acting upon this environment (e.g. through communicating with other agents like pilots/ other controllers, or turning off runway stop-bars remotely). The agent environment is understood as all surrounding human and non-human agents.
Agent-based safety risk analysis
TOPAZ safety risk assessment methodology
Motivated by the need to model the dynamics, the stochastic, and the interactions of safety critical multi-agent systems, NLR has developed the TOPAZ safety risk assessment methodology, e.g. (Blom et al. 2001a; Blom et al. 2006). The quantitative part of TOPAZ develops and evaluates an agent-based model through running Monte Carlo simulations (Blom et al. 2009) in combination with bias and uncertainty analysis (Everdij et al. 2006). Next to these techniques, TOPAZ also integrates human performance modeling (Blom et al. 2001b), and powerful petri net modeling syntax (Everdij & Blom 2010). Applications of these modeling techniques requires dedicated expertise from safety analysts. However, when the TOPAZ toolset is available, normal safety expertise is sufficient.
Case study: ABMS of an active runway crossing operation
Model the agents and their interactions with the environment: For example human performance models of pilots and controllers interacting with technical systems.
Run Monte Carlo Simulations of both nominal and off-nominal scenarios: In order to assess the impact of agents’ individual performance on system behavior.
Analyze system behavior: Explore an array of behaviors or parameters of system performance to guide alternative selection and future development.
Active runway crossings and incursions
In many airports around the world, runway crossings are used by taxiing aircraft from the apron area to the runway and vice versa. These crossings are attractive because they reduce the taxiing time and save fuel. However, they also have safety implications, namely the risk of having a runway incursion. A runway incursion is defined by the International Civil Aviation Organization (ICAO) (ICAO 2007) as “Any occurrence at an aerodrome involving the incorrect presence of an aircraft, vehicle or person on the protected area of a surface designated for the landing and take-off of aircraft”. While a runway incursion does not imply a collision, the probability of an accident is not nil. One of the most famous aviation accidents is the Tenerife airport disaster that occurred on March 1977. Two Boeing 747 aircraft, operated by KLM and Pan American World Airways, respectively collided. While the Pan American aircraft was taxiing, the KLM aircraft took off, resulting in a collision causing 583 fatalities. This accident is the deadliest aviation accident in history (Wikipedia 2012a). 35 years later, runway incursions are still frequently reported in many countries. In the united states alone, preliminary data (FAA 2012) through the end of August 2012 shows a total number of 1010 runway incursions in the year 2012, a 17 percent increase over the same span in 2011.
Researchers and planners operating from different perspectives have proposed many options to address this problem, such as new technology (e.g. in aircraft, ATC tower, or Airport) and new procedures such as ICAO compliant procedures. These proposals aim to reduce the probability of runway incursions, and reduce the accident risk in case runway incursions occur. However, evaluating the impact of these proposals is a demanding task, given the large number of human operators and technical systems that closely interact at the airport. This paper evaluates the safety risk of an active runway crossing operation from an agent-based modeling perspective.
Agent-based model of the active runway crossing operation
In order to define the stochastic dynamics of the agents and their interactions unambiguously, the agent-based model has been specified in terms of Stochastically and Dynamically Colored Petri Nets (SDCPNs). SDCPNs are a powerful extension over normal Petri nets in that they are able to represent general stochastic hybrid processes in a form that supports powerful stochastic analysis (Everdij & Blom 2010). In view of the scope of the current paper, it suffices to describe this agent-based model in normal language rather than in SDCPN language.
At the highest hierarchical level, the relevant agents in the active runway crossing operation are identified. These agents are concurrently interacting with each other and include human and non-human agents. The human agents are the flight crew operating the aircraft, and the runway controller handling the traffic on the runway and its crossings. The non-human agents represent the aircraft and ATC system. In the context of the active runway crossing operation, the ATC system is comprised of three components where each component is performing a number of functions.
These components include: 1) the Radio/Telecommunication (R/T) system used for communication between the controller and flight crew; 2) the Surveillance system used for providing radar track data; and 3) the alerting system used to generate ATC alerts in safety critical situations.
In Figure 2 we use the following abbreviations for the different agents:
○ PF-TX: represents the Pilot Flying the Taxiing Aircraft.
○ PF-TO: represents the Pilot Flying the Taking-Off Aircraft.
○ ATCo-R: represents the Runway Controller.
○ AC-TX: represents the Aircraft Taxiing.
○ AC-TO: represents the Aircraft Taking-Off.
○ ATC System: represents the ATC system.
The main entities within these agents as well as their interactions are described below.
The concept of situation awareness (SA) addresses perception of elements in the environment, their interpretation, and the projection of their future status (Endsley 1995). Stroeve et al. (Stroeve et al. 2003) have captured these perception, interpretation, and project notions of SA mathematically in terms of three components namely, State SA, Mode SA, and Intent SA. The state SA represents the awareness by one agent of other agents. In the context of the active runway crossing operation, this might be the awareness of the flight crew about the state (position and speed) of their own aircraft and other aircraft, or the awareness of the controller about traffic state at the airport. The mode SA represents the awareness by one agent of the mode of other agents. This mode could be for instance the flight phase of an aircraft, or status of an ATC alert. The intent SA represents the awareness by one agent of the intent of other agents. The intent includes continuous states of agents as well as the related times at which these states are expected to be achieved. For example, intent SA may represent the expectations by a runway controller of aircraft destination, and the time it will arrive at a certain waypoint. The timing of the situation awareness updates depends on the initiation and duration of related tasks (e.g. monitoring, communication, coordination).
Task Identification: Considers the ways the human operator identifies the tasks that need to be performed at a particular time instance.
Task Scheduling: Determines which tasks may be performed concurrently, as well as a priority among the tasks that cannot be performed concurrently.
Task Load: Describes the number of tasks to be performed and/or the resources required by tasks at the level of visual, auditory, cognitive and motor performance.
Decision Making: Decision Making processes are based on decision rules dedicated to the scenario considered. In an active runway crossing operation, this could be how to react to a conflict situation.
Task execution: Considers the dynamic and stochastic performance characteristics of tasks.
This Cognitive Control entity considers that humans can function in a number of cognitive control modes such as: Strategic, Tactical, Opportunistic, and Scrambled (Hollnagel 1993). The cognitive control mode may depend on the taskload sub-entity which describes the range of tasks to be done or the situation awareness of the human. It influences human aspects such as the planning horizon and the accuracy of task performance.
The model of the surveillance system provides position and velocity estimates for both aircraft. The surveillance data is used by the alerting system.
The alerting system generates two types of alerts in case surveillance data indicate a safety critical situation. The alerts include 1) a stopbar violation alert in case AC-TX has passed the stopbar, and 2) a runway incursion alert in case AC-TX is within a critical distance of the runway center-line and AC-TO has exceeded a velocity threshold in front of the runway crossing. There is a chance that the surveillance system is not available, resulting in track loss and no alerts being generated.
The model for the R/T communication system between ATCo-R and pilots accounts for the communication system of the aircraft, the communication system of the ATCo-R, the tower communication system, and the frequency selection of the aircraft communication system. The nominal status of these communication systems accounts for direct non-delaying communication. The model accounts for the chance of delay or failure of the communication systems.
The operational conditions in the active runway crossing operation are characterized by two elements namely the runway configuration and visibility conditions. For the first element, runway characteristics such as runway length, width, location of stopbars, and number of branches are specified. Each branch is of the class holding, crossing, or exit. For the second element, four different visibility conditions are represented.
Environment of each agent
Active runway crossing operation
In the active runway crossing operation, we consider a departure runway that is used by AC-TO and has a crossing taxiway at a distance of 1000 m from the runway threshold (see Figure 6). The runway crossing is used by the taxiing aircraft (AC-TX) to cross the active runway. In this experiment, we focus on the scenario that the taxiing aircraft is crossing the runway while it should not, due to a wrong intent situation awareness of its flight crew. Such condition has been shown in previous work to have a strong effect on accident risk (Stroeve et al. 2003; Stroeve et al. 2009). Evaluating the conditional collision risks associated with this condition provides more insight on the collision risk contributions.
In the active runway crossing operation, AC-TX enters the taxiway leading to the runway crossing at a position close to the stopbar, whereas AC-TO initiates take-off from a position near the runway threshold (Figure 6). The entrance time of AC-TX is uniformly distributed around the take-off time of AC-TO. In addition, both AC-TX and AC-TO may be medium-weight or heavy-weight. The operation is assumed to be without restricted visibility range in line with visibility condition 1 of ICAO (ICAO 2004). This implies that the involved human operators can visually observe the traffic situation.
The pilots PF-TO and PF-TX are both responsible for the safe conduct of the flight operations and should actively monitor for potential conflicting traffic situations. ATCo-R is responsible for the safe and efficient traffic handling on the runway and the runway crossings.
During taxiing and take-off, both PF-TX and PF-TO visually monitor the traffic situation at stochastically distributed times. Both pilots may detect a safety critical situation by their own observation. PF-TO observes a conflict when AC-TX is observed to be within a critical distance to the runway centerline. PF-TX observes a conflict when AC-TO is observed to be approaching AC-TX with an increasing speed, and AC-TX is within a critical distance from the runway centerline.
Next to their own monitoring actions, pilots of both AC-TX and AC-TO may also detect a conflict following a call from ATCo-R. In this case, the R/T communication system is used for communication between ATCo-R and the flight crew. ATCo-R may detect a conflict during monitoring if AC-TX is observed to have passed the stopbar, or following an ATC alert. The ATC alerting system may generate two types of alerts to warn ATCo-R: 1) A runway incursion alert for the situation that AC-TX is crossing the runway in front of AC-TO that has initiated take-off; and 2) A stopbar violation alert for the situation that AC-TX crosses an active stopbar.
These alerts consist of audible warnings and an indication on the ground surveillance display. The alerts are based on radar tracking data (Aircraft position and velocity estimates) provided by the surveillance system.
Following conflict detection, PF-TX may decide to start a full braking action or continue crossing in case AC-TX is within a critical distance of the runway center line. Likewise, PF-TO may decide to start a full braking action or continue taking off if it is too late to brake.
Results and discussion
Monte Carlo simulation results
PF-TX does not actively monitor the traffic situation visually, such that he may only detect a conflict via a call of ATCo-R.
PF-TO does not actively monitor the traffic situation visually, such that he may only detect a conflict via a call of ATCo-R.
ATCo-R does not actively monitor the traffic situation visually, such that he may only detect a conflict through ATC alerts.
The ATC alerting system does not generate any alerts.
The conditions above refer to the situation at the start and during the active runway crossing operation, and they are not assumed to hold prior to the occurrence of the crossing operation. Combining these conditions results in sixteen cases where agents are either monitoring or not monitoring the traffic situation. A total of 1 million Monte Carlo simulation runs were performed for each of the sixteen combinations. The Monte Carlo method was used to capture the stochastic nature of the various agents in the active runway crossing operation. The conditional collision risk was obtained by dividing the number of collisions by the number of performed simulations for each case.
The conditional collision risk results in Figure 7 can be divided into two main categories. The first category is characterized by low to medium risk increase factors and includes the first seven cases C1 to C7 on the left-hand side. The second category is characterized by high risk increase factors and includes cases C8 to C16 on the right-hand side.
Case C2 - ATC alerts are not generated.
Case C3 - ATCo-R is not actively monitoring the traffic situation visually.
Case C4 - ATC alerts are not generated and ATCo-R is not actively monitoring the traffic situation visually.
Case C5 - PF-TO is not monitoring the traffic situation visually.
Case C7 - Both PF-TO and ATCo-R are not actively monitoring the traffic situation visually.
In the second category, the risk increase factor with respect to C1 is between 60 and 556. The risk levels are high and are in the range between 1.0E-2 (case C9) and 9.5E-2 (case C16). In almost all cases of the second category, PF-TX was not actively monitoring the traffic situation (cases C9-C16). The only exception is Case C8 where the conditional collision risk is around 1.8E-2. In this case however, the performance of all other agents was affected to attain similar risk levels corresponding to cases without active monitoring of PF-TX.
Figure 7 gives some insights on the role of each agent in restricting the conditional collision risk in case the performance of other agents is affected. The role of each agent is explained below:
The results show that an actively monitoring PF-TX restricts the risk in cases where the performance of one or two agents is affected (cases C1 to C7). The risk increase factor with respect to case C1 is between 1.1 and 12.5. In addition, all cases without active monitoring of PF-TX, are characterized by a high conditional collision risk. The risk increase factor with respect to C1 varies between 60 and 556.
The conditional collision risk is increased by a factor of 1.9 in the hypothetical case C5 where PF-TO is the only agent not monitoring the traffic situation. In case C8, where no ATC alerts are generated and both PF-TO and ATCo-R are not monitoring, the risk is higher by a factor of 76 with respect to case C4. This indicates that an active monitoring of PF-TO considerably restricts the risk in cases that involve both the lack of monitoring of ATCo-R, and malfunctioning of ATC alerts (Case C4 versus C8). The risk is also reduced by the active monitoring of PF-TO (factor of 11) in case no ATC alerts are generated (Case C2 versus C6). The role of PF-TO in restricting the risk is however limited when PF-TX is not actively monitoring the traffic situation. This can be noticed when comparing cases C9 with C13, C10 with C14, C11 with C15, and C12 with C16.
The comparison of cases with and without active traffic monitoring by ATCo-R indicate that in most cases, the controller’s observation does not significantly restrict the risk. This can be noticed when comparing case C1 with case C3. The active monitoring by ATCo-R does reduce the risk with only a factor of 1.1. Close factors can be found when comparing cases C2 with C4, C5 with C7, C9 with C11, C10 with C12, C13 with C15, C14 with C16. However, an interesting remark can be made when comparing case C6 with C8. Here the controller’s own visual observation does play a role in restricting the conditional collision risk when the PF-TO is not monitoring the traffic situation and the ATC alerts are not functioning.
The comparison of cases with and without alerts indicates that in most cases ATC alerts barely reduce the collision risk. This can be noticed when comparing cases C1 and C2. In case C1, ATC alerts are generated in case of safety-critical situations, and can be detected by ATCo-R. In case C2, No ATC alerts are generated as described in the experiment set-up. Looking at both cases, the risk increases by a factor of 1.1 meaning that ATC alerts do not significantly reduce the conditional collision risk. A similar remark can be made when comparing C3 with C4, C9 with C10, and C11 with C12. In all these cases, the presence of ATC alerts does reduce the risk by a factor no larger than 1.5. This indicates that alerts to ATCo-R are often too late resulting in late instructions to the PF-TX as manifest from C9 and C11. Only in cases without active monitoring of PF-TO and at the same time with active monitoring of PF-TX, do alerts restrict the risk by higher factors. This can be noticed when comparing cases C5 with C6, and C7 with C8. Here, the conditional collision risk is reduced by a factor of 6.5 and 46 respectively due to ATC alerts. However, if the PF-TX is out of the monitoring role, the contribution of ATC alerts in restricting the risk is reduced to a factor of 1.9 and 4.2 for the cases C13 versus C14, and C15 versus C16 respectively. These results indicate that ATC alerts are of little help to ATCo-R in cases where PF-TX is not actively monitoring the traffic situation. They are often too late to prevent collisions.
Monte Carlo simulation events
Possible sequences of events before a collision
Possible sequences of events
High risk restriction cases
Low risk restriction cases
(E1 ∧ E2) → E3 → E7 → E10
(E1 ∧ E2) → E6 → E9 → E10
(E1 ∧ E2) → E4 → E8 → E7 → E10
C1, C3, C5, C7
C9, C11, C13, C15
(E1 ∧ E2) → E5 → E8 → E7 → E10
C1, C2, C5, C6
C9, C10, C13, C14
(E1 ∧ E2) → E4 → E8 → E9 → E10
C1, C3, C5, C7
C9, C11, C13, C15
(E1 ∧ E2) → E5 → E8 → E9 → E10
C1, C2, C5, C6
C9, C10, C13, C14
(E1 ∧ E2) → E4 → E8 → (E7 ∧ E9) → E10
C1, C3, C5, C7
C9, C11, C13, C15
(E1 ∧ E2) → E5 → E8 → (E7 ∧ E9) → E10
C1, C2, C5, C6
C9, C10, C13, C14
(E1 ∧ E2) → E10
The results suggest that if PF-TX is not actively monitoring the traffic situation, then the conditional collision risk is high, and then the role of PF-TO, ATCo-R, and alerts in restricting the risk is rather limited. This can be explained by the condition considered in this experiment regarding the intent situation awareness of PF-TX. Here, PF-TX crosses the runway without contacting ATCo-R because he is not aware about the runway crossing. PF-TX can only detect a conflict in two cases: 1) when PF-TX observes that AC-TO is approaching AC-TX (Event E3 in Figure 10) or 2) when PF-TX timely receive an R/T call from ATCo-R (Event 8 in Figure 10). This means that if PF-TX is not actively monitoring the traffic situation, the only way to detect a conflict is through an R/T call from ATCo-R (Event E8). For PF-TX, this call often comes too late because AC-TX has already started crossing. A similar evaluation of the results for the PF-TO shows another picture. For PF-TO, the call from ATCo-R does restrict the collision risk considerably. This can be noticed by comparing cases C5 and C8. This is mainly because PF-TO has more time compared to PF-TX to start a collision avoidance braking action, as is manifest from the recorded aircraft positions. The comparison of both of these cases also shows that the call from ATCo-R was mainly triggered by the ATC alerting system.
Identifying weak emergence of type II (see Table 1) was revealed due to the development and simulation of the agent-based model in the wider context of socio-technical systems. The ABMS approach covered both the socio and technical parts through: a) developing the internal model of both human operators and technical systems; and b) capturing the totality of their interactions through Monte Carlo simulations. This is an example where emergent behavior of type II has been predicted thanks to agent-based modeling and simulation (Figure 7). In addition, the events recorded in the Monte Carlo simulation revealed unpredicted behavior regarding the sequence of conflict detection by the agents. Normally, in the context of the runway crossing operation, one would expect according to the ConOps, that conflicts would be first detected by ATCo-R who will then warn the pilots. However, this might not necessarily always be the case. It was shown that PF-TX might detect a conflict even before ATCo-R which could lead to an accident if detection by the pilot is too late (e.g. Figure 9). Such unpredictable behavior can be chaotic and falls under the category multiple emergence type III. In such type of emergence, complete new roles can appear while old ones disappear.
Emergent behavior in the open socio-technical air transportation system results from the interactions between the constituent elements and the operating environment. For an airport, these elements include human operators and their organizations, working procedures, technical systems, and weather. Changing the characteristics of these elements or the way they interact may have an impact on the overall system behavior. Both the operating environment and the socio-technical system elements influence the evolution over time. On the one hand, weather conditions for instance may be a factor in deciding which flight plan to file or which runway configuration to use. On the other hand, flight progress may change as a result of pilots and controller’s decisions. In order to identify which changes at the “elements” level do have a significant impact on the emergent behavior of the socio-technical system, models of constituent elements and their interactions are required.
In this paper, Monte Carlo Simulations of an agent-based model were performed to assess the safety risk of an active runway crossing operation. The safety performance of such an operation depends on the coordination between the runway controller, pilots, and technology all functioning together. The agent-based model has been developed in a hierarchical way. At the highest level, the relevant agents including human operators and technical systems were identified. Then the interactions between these agents were captured and included deterministic and stochastic relationships, as is appropriate for the human performance or the technical systems considered. The impact on the emergent safety behavior was studied through running simulations of both nominal and off-nominal scenarios.
It has been explained that ATM is a complex socio-technical system that exhibits various types of emergent behavior ranging from simple emergence, through weak emergence, up to strong emergence. It has been demonstrated that Agent-Based Modeling and Monte Carlo Analysis provide a platform to integrate and simulate multiple heterogeneous components at different levels, and identify different types of emergence through modeling the interactions between technical systems, human operators, working procedures, and environment.
One of the main findings is that the role of alerting systems in restricting the conditional collision risk is limited in cases without active monitoring of the PF-TX. Another interesting result is that although the role of the controller’s own observation does not restrict the risk in some cases, it plays a significant role in cases without active monitoring of PF-TO and no ATC alerts being generated. These findings are revealed due to the development and simulation of the agent-based model that covers the totality of interactions of components and their variability in performance over time. The Monte Carlo simulations make it possible to understand the potential of agents in restricting the risk in off-nominal scenarios, through capturing their stochastic nature.
aIn this paper, the aggregated simulation included extra combinations compared to (Stroeve et al. 2013). This way the role of ATC alerts in restricting the risk becomes clearer in cases where the controller is not actively monitoring the traffic situation visually (e.g. cases C3, C7, C11, and C15).
The first author would like to acknowledge SESAR Joint Undertaking for partly funding and supporting his research through a ComplexWorld PhD project; National Aerospace Laboratory NLR for making the TOPAZ TAXIR toolset for runway crossing operations available to conduct this study; Ir. Bert Bakker (NLR) for learning the first author how to use the TOPAZ TAXIR toolset in running the MC simulations and in analyzing the results; Dr. Sybert Stroeve (NLR)for valuable discussions on the agent based models used within the TOPAZ TAXIR tool set; and Prof. Jacco Hoekstra (Delft University of Technology) for stimulating discussions on emergent behavior.
- Ball M, Barnhart C, Nemhauser G, Odoni A: Air transportation: irregular operations and control. In Handbook in Operations Research and Management Science, Vol. 14. Edited by: Barnhart C, Laporte G. Amsterdam: The Netherlands: Elsevier; 2007:1–73.
- Bar-Yam Y: Dynamics of Complex Systems. Boulder, Colorado: Westview Press; 2003.
- Bar-Yam Y: About Engineering Complex Systems: Multiscale Analysis and Evolutionary Engineering. In Engineering Self-Organizing Systems: Methodologies and Applications. Edited by: Brueckner SA. Springer; 2005:16–31.
- Beart P: Emergent Behaviour. 2012. http://beart.org.uk/?page_id=63
- Bedau MA: Weak Emergence. Philosophical Perspectives 1997, 11: 375–399.
- Bedau M: Downward causation and the autonomy of weak emergence. draft for prinipia 2002.
- Blom HAP, Bakker GJ, Blanker PJG, Daams J, Everdij MHC, Klompstra MB: Accident Risk Assessment for Advanced Air Traffic Management. In Air Transportation Systems Engineering. Edited by: Donohue GL, Zellweger AG. Progress in Astronautics and Aeronautics: AIAA; 2001a:463–480.
- Blom HAP, Daams J, Nijhuis HB: Human Cognition Modeling in Air Traffic Management Safety Assessment. In Air Transportation Systems Engineering. Edited by: Donohue GL, Zellweger AG. Progress in Astronautics and Aeronautics: AIAA; 2001b:481–511.
- Blom HAP, Stroeve SH, de Jong HH: Safety Risk Assessment by Monte Carlo Simulation of Complex Safety-critical Systems. In Developments in Risk-Based Approaches to Safety: Proceedings of the Fourteenth Safety-Critical Systems Symposium. Edited by: Redmill F, Anderson T. Bristol, UK: Springer; 2006.
- Blom HAP, Bakker GJ, Krystul J: Rare Event Estimation for a Large-Scale Stochastic Hybrid System with Air Traffic Application. In Rare event simulation using Monte Carlo methods. Edited by: Rubino G, Tuffin B. J.Wiley; 2009:193–214.View Article
- Borshchev A, Filippov A: From System Dynamics and Discrete Event to Practical Agent-Based Modeling: Reasons, Techniques, Tools. In Proceedings of the 22nd International Conference of the System Dynamics Society. Oxfort, England; 2004.
- Bouarfa S, Blom HAP, Curran R: Airport Performance Modeling using an Agent-Based Approach. In Proceedings of the Air Transport and Operations Symposium. The Netherlands: Delft; 2012.
- Bouarfa S, Blom HAP, Curran H, Everdij MHC: Agent-Based Modeling and Simulation of Emergent Behaviour in Air Transportation. In proceedings of the Aviation Technology, Integration, and Operations (ATIO). LA, California: USA; 2013. forthcoming forthcoming
- Burmeister B, Haddadi A, Matylis G: Applications of multi-agent systems in traffic and transportation. IEE Transactions on Software Engineering 1997,144(1):51–60. 10.1049/ip-sen:19971023View Article
- Casti JL: Would-be Worlds: How simulation is changing the frontiers of science. New York: John Wiley; 1997.
- CAS Wiki: Emergence. 2013. Available at [http://wiki.cas-group.net/index.php?title=Emergence]
- Chalmers DJ: Varieties of emergence. Granada: Templeton foundation workshop on emergence; 2002.
- Chan WKV, Son YJ, Macal CM: Agent-Based Simulation Tutorial – Simulation of Emergent Behavior and Differences between Agent-Based Simulation and Discrete-Event Simulation. Baltimore, MD, USA: Proc. Of the Winter Simulation Conference; 2010.View Article
- Chen B, Cheng H: A review of the applications of agent-technology in traffic and transportation systems. IEEE Trans Intell Transp Syst 2010,11(2):485–497.View Article
- Corker KM, Muraoka K, Verma S, Jadhav A, Gore BF: Air MIDAS A Closed-Loop Model Framework. In Foyle and Hooey, Human performance modeling in aviation. Boca Raton (FL) USA: CRC Press; 2008.
- Crick F: The astonishing hypothesis: The scientific search for the soul. New York: Charles Scribner’s Sons; 1994.
- Deregee E: Non-Stop Schiphol. Schiphol, The Netherlands: Contrail Publishers; 2006.
- Ek A, Akselsson R, Arvidsson M, Johansson CR: Safety Culture in Swedish Air Traffic Control. Safety Science 2007,45(7):791–811. 10.1016/j.ssci.2006.08.017View Article
- Endsley MR: Toward a theory of situation awareness in dynamic systems. Hum Factors 1995,37(1):32–64. 10.1518/001872095779049543View Article
- European commission and Eurocontrol: European Operational Concept Validation Methodology (E-OCVM) version 3.0. Volume II. Brussels, Belgium: Eurocontrol; 2010.
- Everdij MHC, Blom HAP: Hybrid State Petri Nets which have the Analysis Power of Stochastic Hybrid Systems and the Formal Verification Power of Automata. In P. Pawlewski, Petri Nets, Chapter 12. Vienna: I-Tech Education and Publishing; 2010:227–252.
- Everdij MHC, Blom HAP, Stroeve SH: Structured Assessment of Bias and Uncertainty in Monte Carlo Simulated Accident Risk. In Proc. 8th Int. Conf. on Probabilistic Safety Assessment and Management (PSAM8). New York, USA: American Society of Mechanical Engineers; 2006.
- Everdij MHC, Scholte JJ, Blom HAP, Stroeve SH: An investigation of emergent behaviour viewpoints in literature, and their usability in Air Traffic Management. Seville, Spain: 1st ComplexWorld Annual Conference; 2011.
- FAA: Runway Incursion Totals by quarter FY2012 vs. FY2011. 2012. Available at [http://www.faa.gov/airports/runway_safety/statistics/year/?fy1=2012&fy2=2011]
- Forrester JW: Counterintuitive Behavior of Social Systems. Technol Rev 1971, 73: 52–68.
- Franklin S, Graesser A: Is it an agent, or just a program?: a taxonomy for autonomous agents. In Intelligent Agents 3, Lecture Notes in Artificial Intelligence, Volume 1193. Edited by: Muller JP, Wooldridge MI, Jennings NR. Berlin: Springer; 1997.
- Fromm J: Types and Forms of Emergence. 2005.
- Fron X: Performance Review in Europe. In Air Transportation Systems Engineering. Edited by: Donohue GL, Zellweger AG. AIAA: Progress in Astronautics and Aeronautics; 2001:49–59.
- Goldstein J: Emergence as a construct: History and Issues. Emergence: Complexity and Organization 1999,1(1):49–72.View Article
- Holland JH: Emergence: From Chaos to Order. Reading, MA: Addison-Wesley; 1997.
- Holland JH: Studying Complex Adaptive Systems. Journal of Systems Science and Complexity 2006, 19: 1–8. 10.1007/s11424-006-0001-zMATHMathSciNetView Article
- Hollnagel E: Human Reliability Analysis: Context and Control. London Academic Press; 1993.
- Hollnagel E: The ETTO Principle: Why things that go right sometimes go wrong. Aldershot, UK: Ashgate Publishing; 2009.
- Hollnagel E, Woods DD, Leveson N: Resilience Engineering: Concepts and Precepts. Aldershot: Ashgate Publishing Ltd; 2006.
- ICAO: Advanced Surface Movement Guidance and Control Systems (A-SMGCS) Manual. Doc 9830 AN/452. 1st edition. 2004.
- ICAO: Manual on the Prevention of Runway Incursions. Doc 9870 AN/463. 1st edition. 2007.
- Jennings NR: On Agent-Based Software Engineering. Artificial Intelligence 2000,177(2):277–296.View Article
- Klein G, Feltovich P, Bradshaw JM, Woods DD: Common Ground and Coordination in Joint Activity. In Organizational Dynamics in Cognitive Work. Edited by: Rouse W, Boff K. New York: Wiley; 2004.
- Leveson NG, Barrett B, Carrol J, Cutcher-Gershenfeld J, Dulac N, Zipkin D: Modeling, Analyzing, and Engineering NASA’s Safety Culture. Phase 1 Final Report. 2005.
- Lewes GH: Problems of life and mind. london: Truebner; 1874–1879.
- Mill SJ: A system of logic ratiocinative and inductive. London: John W. Parker and Son; 1872:1843.
- PRC: An Assessment of Air Traffic Management in Europe during the Calendar Year 2008. Eurocontrol 2009.
- PRC: U.S./Europe Comparison of ATM-related Operational Performance. Brussels, Belgium: FAA and Eurocontrol; 2009.
- Railsback SF, Grimm V: Agent-based and Individual-based Modeling: A practical Introduction. Princeton, New Jersey: Princeton University Press; 2012.
- Russel S, Norvig P: Artificial Intelligence: a Modern Approach. 2nd edition. Hong Kong: Pearson Education Asia Limited and Tsinghua Univ. Press; 2006.
- Sanz R: Converging trends in complex software-intensive control. Faro, Portugal: 6th Portuguese conference on automatic control; 2004.
- Consortium SESAR: Air Transport Framework. The Performance Target D2. The European Commission and Eurocontrol: Brussels, Belgium; 2006.
- SESAR Consortium: SESAR Definition Phase, SESAR Safety Management Plan (SMP).WP4.2/Task 4.2.1, Part of DLT-0710–421–01–00. 2007.
- Shah AP, Pritchett AR, Feigh KM, Kalaver SA, Jadhav A, Corker KM, Holl DM, Bea RC: Analyzing air traffic management systems using agent-based modeling and simulation. Baltimore: Proc. 6th USA/Europe ATM R&D Seminar; 2005.
- Sharpanskykh A, Stroeve SH: An Agent-Based Approach for Structured Modeling Analysis and Improvement of Safety Culture. Comput Math Organ Theory 2011, 17: 77–117. Springer US Springer US 10.1007/s10588-011-9083-9View Article
- Stroeve SH, Blom HAP, van der Park MNJ: Multi-Agent Situation Awareness Error Evolution in Accident Risk Modeling. Budapest, Hungary: Proc. 5th USA/Europe Air Traffic Management R&D Seminar; 2003.
- Stroeve SH, Blom HAP, Bakker GJ: Systemic accident risk assessment in air traffic by Monte Carlo simulation. Safety Science 2009,47(2):238–249. 10.1016/j.ssci.2008.04.003View Article
- Stroeve SH, Blom HAP, Bakker GJ: Contrasting Safety Assessments of a Runway Incursion Scenario by Event Sequence Analysis versus Multi-Agent Dynamic Risk Modelling. Berlin, Germany: Proceedings of 9th USA/Europe Air Traffic Management Research and Development Seminar; 2011.
- Stroeve SH, Blom HAP, Bakker GJ: Contrasting Safety Assessments of a Runway Incursion Scenario: Event Sequence Analysis versus Multi-Agent Dynamic Risk Model. In Reliability Engineering and System Safety 2013, 109: 133–149.View Article
- Sussman J: Collected Views on Complexity in Systems. Course materials for ESD.04J Frameworks and Models in Engineering Systems. 2007a. MIT OpenCourseWare; ( ), Massachusetts Institute of Technology. Downloaded on [24 12 2009] http://ocw.mit.edu
- Sussman J: The ‘Clios Process’ A User’s Guide. Course materials for ESD.04J Frameworks and Models in Engineering Systems. 2007b. MIT OpenCourseWare; ( ), Massachusetts Institute of Technology. Downloaded on [24 12 2009] http://ocw.mit.edu
- Tessier C, Chaudron L, Muller HJ (Eds): Conflicting, Conflict Management in Multi-Agent Systems. Kluwer Academic Publishers; 2002.
- Weick K: The vulnerable system: An analysis of the Tenerife air disaster. J Manag 1990, 16: 571–593.
- Wickens CD: Engineering, Psychology, and Human Performance. Columbus: Merril; 1992.
- Wikipedia: Tenerife Airport Disaster. 2012a. , retrieved 27 September 2012 http://en.wikipedia.org/wiki/Tenerife_airport_disaster
- Wikipedia: Emergence. 2012b. . Retrieved 27 September 2012 http://en.wikipedia.org/wiki/Emergence
- Wolfe S, Jarvis P, Enomoto F, Sierhuis M, Putten B: A Multi-Agent Simulation of Collaborative Air Traffic Flow Management, Chapter 18. In Multi-Agent Systems for traffic and transportation engineering. Edited by: Bazzan A, Klugl F. x: IGI Global Publishing; 2009:357–381.View Article
- Wolfram S: Universality and Complexity in Cellular Automata. Physica D: Nonlinear Phenomena 1984,10(1–2):1–35.MathSciNetView Article
- Woods DD, Dekker S, Cook R, Johannesen L, Sarter N: Behind Human Error. 2nd edition. England: Ashgate Publishing Limited; 2010.
- Wooldridge M: An Introduction to Multi-Agent Systems. 2nd edition. Ltd, Publication: John Wiley and Sons; 2009.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.