Enhancing scientific response in a crisis: evidence-based approaches from emergency management in New Zealand

Abstract

Contemporary approaches to multi-organisational response planning for the management of complex volcanic crises assume that identifying the types of expertise needed provides the foundation for effective response. We discuss why this is only one aspect, and present the social, psychological and organizational issues that need to be accommodated to realize the full benefits of multi-agency collaboration. We discuss the need to consider how organizational culture, inter-agency trust, mental models, information management and communication and decision making competencies and processes, need to be understood and accommodated in crisis management planning and delivery. This paper discusses how these issues can be reconciled within superordinate (overarching) management structures designed to accommodate multi-agency response that incorporates decision-making inputs from both the response management team and the science advisors. We review the science advisory processes within New Zealand (NZ), and discuss lessons learnt from research into the inter-organisational response to historical eruptions and exercises in NZ. We argue that team development training is essential and review the different types of training and exercising techniques (including cross training, positional rotation, scenario planning, collaborative exercises, and simulations) which can be used to develop a coordinated capability in multiagency teams. We argue that to truly enhance the science response, science agencies must learn from the emergency management sector and embark on exercise and simulation programs within their own organisations, rather than solely participating as external players in emergency management exercises. We thus propose a science-led tiered exercise program, with example exercise scenarios, which can be used to enhance both the internal science response and the interagency response to a national or international event, and provide direction for the effective writing and conduct of these exercises.

1 Introduction

During a volcanic crisis, whether an isolated period of unrest or a full scale eruption and recovery, many agencies and organisations are involved in its response and management. These range from expert and technical advisors (e.g., geologists, geophysicists, engineers, and social scientists), through to emergency management agencies (civil defence, fire service, police, army, national and local government) and lifeline organisations (lines companies, transport, water). For example, during the 1980 eruptions of Mt St Helens, over 130 officials and organisations responded (Saarinen and Sell 1985); during the eruptive episodes of 1995 to 1996 in Ruapehu, NZ, over 42 organisations were involved (Paton et al. 1998a), and in the 2012 Te Maari eruptions of NZ, some 30 organisations responded.

The number of responding agencies increases as the unrest or eruptive period continues, and they need to collaborate and share knowledge to effectively respond in a crisis that creates multiple, diverse consequences. However, these organizations bring to the crisis management context diverse operational roles, different organizational objectives and political or economic pressures, and varied ways of interpreting, prioritizing and responding to issues that reflect organizational policies, practices and cultures that range from emergency services’ command and control practices to the more organic approach typical of scientific/research organizations. Ensuring that agency representatives can integrate their knowledge and expertise for response planning and implementation, and ensuring that they can continue to do so in a response environment that present complex, dynamic demands that need to be understood and managed over time, is a challenging task. We present here a literature review of the factors influencing response effectiveness, and discuss several approaches that have been developed to achieve an effective coordinated outcome, as well as how they could be integrated into the volcanology community to enhance and inform the response of volcanologists in the unique management environment created by volcanic crises. In the context of this literature, we also review and evaluate examples of NZ volcanic science advice and response practices.

First we discuss the development of science advisory groups in New Zealand since one of the earliest volcanic exercises run in 1992 (Section 2). We then discuss psychological, social and organizational influences and practices that affect response effectiveness (Section 3) and argue for the need for regular training activities to improve these competencies (Section 4). In doing so, we summarize the benefits accruing from science agencies learning from the methods that emergency management agencies routinely use, and embarking on exercise and simulation programs that mirror the complexities of the response environment in which they will make important, but non-routine, contributions within their own organisations. The paper argues that such in-house training (e.g., involving cross training, positional rotation, scenario planning, collaborative exercises, simulations, training and shared exercise writing tasks see Sections 4 and 5) is pivotal to developing the future response capability of science advisory groups and their ability to effectively complement the emergency management functions they will inevitably interact with. We also review national and international Civil Defence and Emergency Management (CDEM) exercises (Section 5) to highlight the benefits that can arise if scientists develop their own activities rather than solely participating as external players in emergency management exercises. Following a review of NZ’s 2008 “All of nation” volcanic Exercise Ruaumoko (Section 5.2) to illustrate how effective evaluation informs the development of Volcanic Science Advisory Group (VSAG) processes within NZ, the paper concludes with our proposing, in Section 6, a new exercise structure for volcanology. This will facilitate the integration of volcanological expertise into CDEM processes and how scientists and scientific agencies can proactively contribute to and capitalise on opportunities to enhance shared understanding between diverse responding agencies. Throughout this paper, we consider science and science advice providers to represent the expert source of information on hazard processes (e.g. geology, geophysics, geochemistry, geodesy, atmospheric science) and the expert source of information on social and economic impacts, including communication and behaviours (as provided by the ‘Social Consequences’ sub-advisory group of Exercise Ruaumoko; Smith 2009).

Incorporating a wide range of expertise into an advisory group process is closely related to the concept of ‘post-normal’ science (Funtowicz & Ravetz 1991; Krauss et al. 2012) which is a ‘new conception of the management of complex science-related issues’ (Funtowicz & Ravetz 2003, p. 1) where ‘facts are uncertain, values are in dispute, the stakes are high and decisions urgent’ (Funtowicz & Ravetz 1991, p. 137), particularly when these uncertainties are of an epistemological or ethical kind. Post-normal science considers these elements of uncertainty, value loading, and a plurality of legitimate perspectives to be integral to science, and that by adopting this ‘post-normal’ approach there is a recognition that risks are interpreted and managed subjectively (depending on local values and norms as well as disciplinary frameworks). This approach presents a new problem-solving framework and acknowledges that a plurality of perspectives should be structured into the informed decision making processes during uncertain high risk environments (see WSS Fellows on RIA 2014).

2 The development of volcanic science advisory groups in NZ

One of the earliest exercises conducted to explore the response to a volcanic eruption in New Zealand was Exercise Nga Puia in 1992 (Martin 1992). This exercise, based on a simulated eruption at the Okataina Volcanic Centre (Nairn 2010), informed the response plans for the region, in particular the subsequent volcanic alert level processes. A few years later, these lessons were tested during the unrest and eruptions of Ruapehu volcano in 1995–1996, when over 42 organisations responded, with the Institute of Geological and Nuclear Science (now GNS Science) acting as the major science provider (Johnston et al. 2000) (Figure 1).

Figure 1

The information flow during the response to the 1995 Ruapehu Eruption showing information flow between key agencies (Paton et al. 1999 ).

Analysis of the organizational response to these eruptions (Paton et al. 1998a) identified the prominent role the limited formalised inter-organisational networking played in creating a coordinated response, particularly with regard to issues arising from ad-hoc interaction between science and response agencies. Response agencies became inappropriately over reliant on science agencies for management information. Paton et al. (1998a; 1999) found that response agencies expected the geophysicists and volcanologists analysing volcanic activity to provide them with direct answers to all response management issues. For example, one co-author (DJ) reported how response agencies expected volcanologists to be able to answer questions about the effect of volcanic ash on sheep. Response agencies were unprepared for the need to be able to liaise with different expert sources (e.g. volcanologists, agricultural scientists, and veterinarians) to integrate input for multiple sources to make response management decisions. Similar problems emerged with regard to public health, environmental health, and utility issues. In addition to expecting certainty about volcanic hazard characteristics and future activity, response problems emanated from lack of attention in the response planning context given to understanding what sources of expertise would need to be consulted and how response management would need to integrate and interpret information while collaborating with diverse others. At the same time, the lack of networking meant that many agencies were unable to fully utilize scientific data as agencies found it was inconsistent with (unexpected) situational awareness and decision demands, discussed further in Section 3.

As a result, science advisory processes were redeveloped through the formation of a number of VSAGs (Smith 2009; Jolly and Smith 2012). During many natural hazard events, such science advisory bodies have been called upon to provide information and advice. The ability to source science advice through “one trusted source”, such as the VSAG, has proved beneficial (Ministry of Civil Defence and Emergency Management, MCDEM 2008) (Figure 2a). This approach also facilitates an integration of a wide range of expert opinions required to manage uncertainty during decision making (Lipshitz et al. 2001) and can help combat issues arising from conflict between scientists (Barclay et al. 2008).

Figure 2

Information flow processes during a NZ Volcanic crisis. (a) The information flow during Exercise Ruaumoko in 2008, outlining the operating structure of the Auckland Volcanic Scientific Advisory Group and its relationship to GeoNet and CDEM (Smith 2009). (b) The proposed model for a national hazard science advisory group, to enable integration of nationwide science capability, as proposed by Smith (2009) after Exercise Ruaumoko. Smith (2009) states that “this advisory group would be made up of appropriate subject experts from across universities, crown research institutes and other science organisations including consultancies, [and] … could play both an operational role (during events) and a strategic role for planning science activities” (p. 76). The current advisory structure of the CPVAG, TVSAG, and CAG reflect a similar advisory process (Jolly and Smith 2012).

These VSAGs represent advisory bodies that are on standby, and have plans to respond to a crisis or unrest period. The advice provided by VSAGs is vital for effective emergency management planning, intelligence gathering, and decision making and for the protection of life, infrastructure and welfare, and depending on local procedures, the VSAG may exclusively include scientists (volcanologists, meteorologists, etc.) or also include local and regional emergency managers and officials. In NZ, the earliest formalised VSAG was the Egmont Volcanic Advisory Group formed in the early 1990s alongside the deployment of the Taranaki Volcano Seismic Network (for a full history see Bayley 2004), which meets once a year to review the monitoring data and other scientific research. By 2004, this group comprised representatives from Massey University, University of Auckland, GNS Science (NZ Crown Research Institute for “Earth, geoscience and isotope research and consultancy services”, GNS Science Website 2014), Department of Conservation and Taranaki Regional Council; and sat at the advisory group level of the Taranaki Civil Defence Emergency Management Coordinating Executive Group.

This group advised Taranaki Regional Council in the development of its first Volcanic Contingency Plan in 2000, which addressed the framework of Scientific Alert Levels, the principal emergency management activities for response, the expected hazards and the monitoring network. Nowadays the Egmont Volcanic Advisory Group has evolved into the Taranaki Seismic and Volcanic Advisory Group (TSVAG), encompassing additional representatives from Victoria University of Wellington, MCDEM, the Earthquake Commission, and local CDEM groups (TRC 2013).

The experiences of the TSVAG have helped inform the process and formation of a number of other VSAGs throughout NZ, many of which have overlapping membership in terms of volcanologists and national level CDEM and response organisations, most of these contain representatives from local CDEM, response and lifeline organisations and scientists from across the NZ Universities, and Crown Research Institutes (CRIs) including GNS Science:

• The Auckland Volcanic Scientific Advisory Group (AVSAG) was established by the Auckland Civil Defence and Emergency Management Group in 2007 as part of its Volcanic Contingency Plan and in preparation for the MCDEM led volcanic Exercise Ruaumoko, to provide advice to officials about the volcanic field residing under Auckland City (MCDEM 2008; McDowell 2008; Smith 2009). This built on the pre-existing VSAG mechanisms established in the 2002 Contingency Plan (Beca Carter Hollings & Ferner 2002) which included volcanologists (from GNS Science, Auckland Regional Council, and University of Auckland), meteorologists, and specialist medical advisors. In 2007, AVSAG was the first NZ VSAG to establish formal terms of reference that were signed by member organisations (Cronin 2008), and this updated VSAG incorporated a wider range of representatives from Auckland, Waikato and Massey Universities, GNS Science, MetService, the Kestrel Group, as well as local and national CDEM representatives (Smith 2009).

• The Central Plateau Volcanic Advisory Group (CPVAG) was established in 2008 to “provide a forum for the collective planning and readiness activities for volcanic hazards in the Central Plateau” which includes the volcanoes Mt. Ruapehu, Mt. Ngauruhoe, and Mt. Tongariro (CPVAG 2009, p. 5). This group formed directly in response to the dam break lahar that occurred from the Ruapehu Crater Lake in 2007, and the recognition by key stakeholders that a combined expanded advisory group was needed for effective planning, preparedness, relationship building, and inter-agency coordination (ibid). CPVAG encompasses a Science Focus Group, a Planning Focus Group and a Communications Focus Group, all guided by a framework strategy and Contingency Plan, and who meet every six months to report back on work programmes, outcomes, and future plans. The processes set up by CPVAG were recently tested in 2012 during the Te Maari eruption and unrest period, and the evaluation of that response is currently ongoing as the eruption represents a critical opportunity to review effectiveness, and identify areas for improvement and capacity building.

• The fourth VSAG in NZ is the Caldera Advisory Group (CAG, Waikato Regional Council 2014; Potter et al. 2012), which was formed in late 2010, with a focus on the eight caldera volcanoes in the Taupo Volcanic Zone. This group formed in response to the recognition of a gap in the advice provision available for the particular effects of caldera volcanoes, and an acknowledgment that these effects could last for significant time periods (years to decades) with a “profound impact on the social and economic environments” (Waikato Regional Council 2014).

Similar science advisory group processes also exist for other hazards in NZ including the Tsunami Expert Panel, which activates in response to a local, regional, or distant source earthquake and tsunami warning. This advises officials of coastal regions at risk, expected tsunami arrival times and durations, and the expected maximum wave amplitudes at the coast, providing advice directly to the Ministry of Civil Defence Emergency Management response team (MCDEM 2010).

More recently, during the 2010–2012 earthquake and aftershock sequence in the Canterbury region, the Natural Hazards Research Platform assumed the role of national coordinator of science advice when the government declared a State of National Emergency after the fatal M6.3 event in February 2011 (Canterbury Earthquakes Royal Commission – Te Komihana Rūwhenua o Waitaha 2012). This government funded multi-party research management platform was established in 2009 to provide secure, long-term funding for natural hazard research, to encourage stakeholder involvement in research, and to promote collaborative research (Natural Hazards Research Platform 2009). In future natural hazards events, and based upon the Canterbury experiences, the various scientific advisory group sections of the wider volcanic advisory groups described above would likely fall under the coordination of the Natural Hazards Platform, or a future equivalent, as they fulfil their science advisory role; pending reviews of recent volcanic eruptions and earthquake events of the last 4 years, and changes currently under consideration for new national funding procedures.

AVSAG was the first VSAG to be comprehensively tested in a simulation. This was done through Exercise Ruaumoko, which was a MCDEM led exercise to test an all of nation response to a volcanic eruption in the Auckland volcanic field (MCDEM 2008) and was the first test of AVSAG. For this, AVSAG was co-ordinated through a tri-partite sub-group system (Volcanology, Volcano Monitoring, Social Consequences), which reported upwards to a smaller core VSAG that liaised directly with MCDEM and Auckland CDEM via on-site liaison officers in the Emergency Operation Centres (EOCs) at each location (see reviews in: Smith 2009; Doyle & Johnston 2011).

Reviews of this exercise (MCDEM 2008; McDowell 2008; Cronin 2008) identified that the AVSAG process facilitated the provision of valuable advice in a clear, timely manner. As advocated for by the International Association for Volcanology and Chemistry of the Earth’s Interior (1999), the AVSAG provided a facility for the scientists (from all contributing disciplines) to “use a single voice”, share information to reduce confusion, and to encourage efficient teamwork amongst scientists and public officials, while also encouraging integration of diverse scientific expertise and minimising communication delays. However, during the most active periods of the response towards the end of the exercise, the existence of two distinctly separate scientific sub-groups composed of the predominately university-based ‘volcanology’ group and the ‘monitoring’ group of GNS Science based scientists became unrealistic, and as stated by McDowell (2008), p. 22, the priority instead should be to have “the rapid assessment and decision-making in relation to technical data” rather than maintaining and communicating between these two separate groups. While the main advantage of this AVSAG approach was the wide range of scientific experts and competency, during the most active period the due process needed to maintain this inclusivity actually slowed down advice provision (MCDEM 2008).

A clear advantage during the exercise was the presence of a science advisor in both the National Crisis Management Centre (NCMC) and the Auckland CDEM Group EOC, providing a vital link between the VSAG assessment and the emergency management decision-making. However, during the rapid escalation of unrest and the critical moments of the crisis, there was a potential for disconnect to occur between this local and national advice provision to the AGEOC and the NCMC respectively, and this resulted in a divergence of the science advice (which was informing evacuation planning) at Local and National levels (Cronin 2008, discussed further in Section 5.2).

A potential disconnect could also occur not only between the science advice at the local and national CDEM level, but also between the local and national science research response, capability, and processes in future events. Smith (2009) suggests that to address these limitations and coordinate the scientific advice beyond the limited knowledge pool and resources in a locally impacted area, a nationwide Volcanic Science Advisory Panel (NZVAP) should have (see Figure 2b):

• “at its core national hazard monitoring capability and processes (e.g. GeoNet), with involvement of additional capability from universities and other science organisations based on thresholds of response. The intent is that GeoNet (both the technology and the science expertise of GNS Science) be the hub of any science response for earthquake, volcano, tsunami or landslide events” (p. 77).

GeoNet is a collaboration between the Earthquake Commission (EQC - NZ’s insurance provider for natural disasters; EQC Website 2014) and GNS Science and is the “official source of geological hazard information for New Zealand”. Established in 2001 it monitors geohazards (in particular earthquakes, tsunami, volcanoes and landslides) via an extensive monitoring system and archival data centre, and provides public and official information including earthquake reports and Volcanic Alert Bulletins (GeoNet website 2014). The NZVAP approach outlined by Smith (2009), encompassing GeoNet at its core, still supports the existence of regions having existing scientific or planning advisory groups with a volcanic and/or earthquake focus, but it also addresses the need for mobilisation of NZ-wide science capability, while remaining responsive to local CDEM needs (see also Section 5.2; Jolly and Smith 2012).

Developing VSAGs prior to an event can prospectively enhance the crisis response capability of scientists and the full multi-agency response alike, through the development of terms and protocols for response, information sharing and inter-agency management, and situational awareness. A prospective approach facilitates future collaboration (e.g., enhanced mental models, shared situational awareness, enhanced multi-team performance) and more effective communications between scientists and responding agencies.

3 Factors that influence response effectiveness

Even with a pre-existing VSAG, the development of effective multi-agency response needs to accommodate issues arising from, for example, differences in organisational culture, jurisdictional expectations, and differences in the economic and political pressures on participating agencies. These represent demands on the goal of prioritising tasks and information needs over and above those emanating from the complex, evolving volcanic hazard. The corresponding threats to trust, leadership or team ability, conflicts over responsibility or priorities, reputation management, and need to function under high psychological and environmental stressors (fatigue, tunnel vision, family commitments and over work) conspire to impair performance of the individual and team (Boin and’t Hart 2001; Handmer 2008; Quarantelli 1997; Sinclair et al. 2012b; Paton 1996). In addition, conflict may arise as individuals swap hierarchal/seniority position as they move from their day-to-day role to their response role.

To manage these issues, it is essential to build future response capability via the development of good team and inter team mental models. This facilitates situational awareness and enhanced decision making capability for personnel within the VSAG and key responding agencies. This should be supported by training (Sections 4 and 5) and resource planning (e.g., accommodating the need to coordinate multiple shifts throughout a response, manage fatigue, allow individuals to attend to personal and family demands) that ensures that staff can return to their role with a fresh perspective. Limiting time on shift has pragmatic benefits in reducing response risks from personnel adapting to small incremental change and losing situational awareness (e.g. Tickell 1990). In contrast, a fresh responder/scientist would recognise a significant change demanding an action. This tendency, colloquially referred to as the ‘boiling frog syndrome’^a, was particularly noticed in the volcanologist’s experience when managing the 1991 eruption at Mt. Pinatubo (Whittlesey and Buckner 1993), and highlights the need for regular shift rotation of scientists and other responders in crisis response.

It is also important to consider rotating the role of the lead science individual (or agency) within a VSAG during a response, particularly for long duration crises. In a NZ context for example, many of the responding scientists will be balancing their ‘response role’ (to provide advice to responding agencies and understand the phenomena occurring) with their ‘day-to-day role’ (including funded research, consultancy contracts and lecturing). The ‘day-to-day role’ may need to take priority at times, and thus by rotating the role of ‘lead’ individual (or agency), this will allow time and space to respond to these other competing demands, while also managing issues around fatigue and tunnel vision as the various agencies and individuals ‘share the load’. Further, by rotating these roles, individuals can develop a greater understanding of each other’s responsibilities, roles, pressures, and demands, helping to build a better shared mental model of the team response (see Sections 3.3 and 4.1). For any role swap or shift change to work effectively an effective change-over procedure (e.g., role shadowing for a specific time period) is needed to transfer situational awareness and overall response performance. This is essential to maintain decision making effectiveness in evolving crises.

3.1 Decision making

Pivotal to effective volcanic crisis management are the decision making, situational awareness, mental models (which is an individual’s representation or visualisation of a real system, including concepts, relationships, and their role within that system) and trust processes that underpin effective response. We commence discussion with an overview of the individual and group decision making processes occurring in volcanic management and response.

Analytical decision-making is defined by working through a process: identifying a problem; generating options to solve the problem; evaluating these; and implementing the preferred option (Flin 1996; Saaty 2008). This form of decision making requires time to allow this process to occur. It is the default approach adopted by scientists (and managers) due to their training. However, in emergency response a range of other decision making styles: analytical; naturalistic; procedural based; creative; and distributive (e.g. Crichton & Flin 2002); are required and need to be matched to the situation and conditions encountered by decision-makers.

The slower, more considerate, analytical decision making processes lie at one end of a continuum of decisions styles. At the “faster” end of the decision making continuum lies naturalistic decision making (NDM; Martin et al. 1997). This relies on experience garnered through real world crises as well as simulations and exercises (Crichton and Flin 2002; Klein 2008). It is commonly adopted in high risk and low time contexts and naturalistic settings which involve: ill-structured problems; uncertain, dynamic environments; shifting, ill-defined, or competing goals; action/feedback loops; time stress; high stakes; multiple players; and the effects or pressures of organizational goals and norms (Orasanu & Connolly 1993, as cited in Zsambok 1997, p. 5). For critical incident management, research has identified four key NDM processes (Crego and Spinks 1997; Crichton and Flin 2002; Pascual and Henderson 1997): 1) recognition-primed and intuition led action; 2) a course of action based upon written or memorised procedures; 3) analytical comparison of different options for courses of action; and 4) creative designing of a novel course of action; ordered by increasing resource commitment.

In a crisis, uncertainty, environmental change, risk and time pressures are amplified, making decision making (whether by scientists or responders) in this dynamic context one that is dependent on ‘task conditions’ (Martin et al. 1997), and thus throughout one incident different processes may be adopted. For example, during Exercise Ruaumoko (Sections 2.1 and 5.2), the on-site GeoNet duty officers inside both the Auckland Group EOC and the NCMC were often asked to provide answers to questions from key decision makers who required an immediate response due to response management demands. This “task condition” (short time) would have favoured the more naturalistic decision making, where the scientists would have relied upon their experience to assess the situation (both the question and the available science) to make an intuitive or recognition-primed decision (Paton et al. 1999, Klein 1998). However, earlier in the exercise, during the warning period preceding the volcanic ‘crisis’ the science advice could be carefully evaluated and compared, and relatively lower time pressures afforded scientists and decision makers the opportunity to adopt an analytical decision making approach. However, as the situation moved from this early warning phase into the crisis of impending eruption, or for a situation where scientists are responding rapidly after an eruption (e.g. the “Blue Sky” eruption of Ruapehu in September 2007), the more intuitive naturalistic style would again be adopted. As stated by Paton et al. 1999, “attention must [thus] be directed to understanding [this] naturalistic decision-making of experts, and how it can be modelled in simulations to develop this contingent management capability” (p. 44).

3.2 Situational awareness

Pivotal to effective decision-making process is a capacity to: 1) evaluate and define a problem and task characteristic via situation assessment (Endsley 1997; Martin et al. 1997); and 2) select a decision-making strategy from the four options described above (Crichton and Flin 2002). The former process is intrinsically dependent upon the situational awareness (SA) of the individual and the team (Cannon-Bowers and Bell 1997; Crichton and Flin 2002).

Situational Awareness comprises three levels (Endsley 1997, p. 270–271): 1) Perception – understanding the importance of information and cues in the environment; 2) Comprehension – combine, interpret, store, and retain information and be able to use it; and 3) Projection – prediction of future situations from existing and previous situations. Initial and ongoing SA is critical to decision making (Sarna 2002). Thus a decision-maker may make the correct decision based upon their perception of the situation, but if their situation assessment is incorrect then this may negatively influence their decision (Crichton and Flin 2002). When responding to a volcanic eruption, developing and maintaining this SA is important for both volcanologists (in their assessment of available data and future projections, information from other agencies, demands upon their advice) and emergency managers and decision makers (in their assessment and understanding of information - including science advice, resources, demands and future requirements and needs).

In reviews of the inter-organisational response of the 1995–1996 Ruapehu eruptions in NZ (Paton et al. 1998a, 1998b, 1999), several issues affected the SA of both scientists and emergency managers. In particular, “inter-organisational networking” was weak, with none of the responding agencies (e.g. fire, police, civil defence, social services, media, etc.) having an established or formalised inter-organisation network in place with GNS Science before the event, even though GNS Science acted as an information provider for 63% of the responding agencies. This resulted in organisations interacting on an “ad hoc” basis (Paton et al. 1998a, p. 7) contributing to co-ordination and communication problems and preventing their using crucial information to build and maintain SA, and thus impacting both emergency management and volcanological decision-making processes and outcomes. This was compounded by issues such as the “lack of clear responsibility for co-ordination” across responding agencies (reported by 45% of participating agencies), “inadequate communication with other agencies” (37%), and “inadequate co-ordination of response” (32%). These are all indicators of “team breakdown”, inadequately defined and co-ordinated roles, and poor communication (Paton et al. 1998a, 1999). The development of effective inter-organisational crisis communication requires (Paton et al. 1998a) “information needs [that] are anticipated and defined, that networks with information providers and recipients are organised, and [that] crisis communication systems capable of providing, accessing, collating, interpreting and disseminating information are established. … [and] shared terminology and systems” (p. 8).

The development of VSAGs and the associated Terms of Reference within NZ over the last two decades will have helped to build shared mental models of the response environment across organisations. VSAG development has included the identification of protocols for communication and networking with emergency management and key response organisations, and specifying relationship building activities to be undertaken within these groups. These activities have improved communication and information flow during a volcanic crisis and thus the shared situational awareness in that crisis. Comparison of Figures 1 and 2 illustrates the change and improvement in information flow processes between Ruapehu in 95–96 (Figure 1) and Ruaumoko in 2008 (Figure 2).

The rarity of large scale eruptions makes it important to capitalize on the learning opportunities events, exercises, and reviews provide for developing situational awareness and for facilitating the ability of scientific advisors to develop shared mental models of their and others role in response management. This feeds into training needs analysis and the development of the situational awareness competencies and decision support systems required to sustain effective situational awareness in complex, rapidly evolving and dynamic volcanic crises. It can also inform the development of the shared situational awareness required if all team members make their respective contributions to a shared task or goal. That is, to develop shared mental models of presenting problems and response options, particularly when decision inputs come from different professions and/or from participants who are spread over a large geographical area. Facilitating the latter introduces a need for distributed decision making, discussed next.

3.3 Shared mental models

The scale and complexity of volcanic crisis response results in decision-making involving people who differ in their profession, expertise, functions, roles and geographical location. This more integrative decision style is called distributed decision making (Rogalski & Samurcay 1993, as cited in Paton & Jackson 2002). As discussed in Section 3.1, an individual’s mental model impacts individual decision making, as it is their representation of the wider system and processes, including inter-agency relationships, needs and demands, and an individual’s role within a crisis. Thus, for effective distributed decision making, individuals require a good shared mental model of the response environment in time and space, which incorporates how their expertise contributes to different parts of the same plan, and their understanding of each other’s knowledge, skills, roles, anticipated behaviour or needs (Flin 1996; Marks et al. 2002; Paton and Jackson 2002; Schaafstal et al. 2001). By building a shared mental model, team members can develop an accurate expectation of the performance of their team members and themselves, leading to effective coordination without overt strategizing (Blickensderfer et al. 1998; Cannon-Bowers et al. 1998; Lipshitz et al. 2001; Salas et al. 1994).

The collection of GNS Science information by response agencies in an ad-hoc basis (Paton et al. 1998a & Paton et al. 1999) during the 1995–1996 Ruapehu eruptions describes a process where information was being provided by explicit requests only. In such cases, the information provided often also needed to be adapted and translated to meet decision needs. However, research in the decision making community has identified that effective teams move from the sharing of information by explicit request towards an approach that adopts implicit supply, where members provide not only good information, but unprompted information that is tailored in terms of content and format due to their understanding of the needs of the recipient (Lipshitz et al. 2001; Kowalski-Trakofler et al. 2003; Paton and Flin 1999). Implicit communication also facilitates the maintenance of situational awareness during periods of dynamic information, as it allows decision makers to focus on task management. For this kind of team functioning to be successful in complex, time pressured situations, (Wilson et al. 2007, as cited in Owen et al. 2013, p. 5) identified that it required the following characteristics:

• Effective communication consisting of accurate and timely information exchange, correct phraseology and closed-loop communication techniques;

• Coordinated behaviour based on shared knowledge, performance monitoring, back-up and adaptability; and

• A co-operative team orientation, efficacy, trust and cohesion.

However, as highlighted by Owen et al. (2013), p. 6 for the multi-team, multi-organisational, coordination characteristic of large-scale complex emergency management events, the challenge is not just to build an effective team, but “to understand how a team coordinates within teams” and how understanding may be “shared between teams”. When we consider a multi-agency VSAG, the responding scientists can be considered as but one team within the complex multi-team response. Owen et al. (2013) identify four distinct stages for effective and adaptive team functioning for an inter-team inter-organisational response, including: 1) situation assessment, 2) plan formulation, 3) plan execution, and 4) team learning. These, and the indicators typically used to identify whether these activities are occurring, are depicted in Table 1.

Table 1 Framework for inter-team inter-organisational coordination (Owen et al. 2013 )

A significant challenge here derives from a need to coordinate the inputs of different agencies and experts to assist the holistic management of complex hazard consequences. For example, public health specialists possess expertise concerning the specific effects of ash and gas on health. However, to mount an effective response, their input, as members of an ‘emergent’ team, must be integrated with input from, for example, volcanologists, emergency managers, social welfare, and transport agencies, to facilitate understanding of the ‘whole’ problem, prioritise issues, and to identify where is safe for people to be evacuated to, etc. This example illustrates how, for example, public health specialists and volcanologists need to bring their professional team mental model to bear on identifying their specific contribution, but also develop a superordinate (overarching) mental model that integrates all areas of expertise. A significant challenge arises because scientists, and indeed all stakeholders, need to switch between a) being autonomous actors, and b) being multi-disciplinary team members, depending on the task being undertaken (Janssen et al. 2010).

Fundamentally, in a volcanic crisis response setting, the science advisors’ and VSAGs’ role is to provide information to facilitate the response agencies understanding of the hazard issues, priorities, the wider context, impacts, and potential future outcomes, and thus build their situational awareness to aid their decision-making process. However, as stated by Doyle & Johnston (2011), it is not just a case of providing the emergency managers with all available science information, but about understanding their needs to meet their information requirements. Simply providing as much advice as possible may actually hinder the decision process, due to cognitive overload and an overuse of these available resources (Crichton and Flin 2002; Omodei et al. 2005; Quarantelli 1997). To contribute in this way, scientific advisors need to develop a shared mental model with their emergency manager counterparts both prior to an event (to develop effective plans) and during the crisis itself. This shared mental model encompasses the overlapping elements of each team member’s SA and represents the inter-team co-ordination (Endsley 1994), allowing an effective coordination amongst team members without the need for extensive overt strategizing (Salas et al. 1994; see review in Doyle & Johnston 2011).

For science response it is thus vital that scientists develop this shared mental model within the VSAG, and in the wider multi-agency response, to facilitate their ability to implicitly provide the science information required by the main decision makers at critical periods (see also Doyle & Johnston 2011). However, when dealing with the uncertainty implicit in volcanic crises the effectiveness of this information sharing relationship is influenced by the degree of trust that exists, or that needs to be developed in situ, between key players. This is particularly important given the rarity of opportunities for functional organizational interaction before a crisis occurs.

3.4 Trust

Trust plays a pivotal role in developing sustainable, functional relationships when members of diverse organizations need to collaborate to access, share and use information for decision making in response environments characterized by uncertainty (Siegrist and Cvetkovich 2000). Without trust, teams focus on task demands, not teamwork, reducing their effectiveness in tackling emergent crisis response needs (Pollock et al. 2003).

Inter-agency trust develops through collaboration. Since volcanologists and emergency managers rarely work together under normal circumstances, trust among agencies may be lacking (Dirks and Ferrin 2001). Since representatives of scientific and EM agencies typically meet and interact for the first time during a crisis, agency representatives are denied the luxury of building trusting relationships over time. Trust must be developed via other mechanisms. One approach capitalizes on the concept of swift trust (e.g. Meyerson et al. 1996).

Swift trust can be developed in temporary (EOC) organizations if certain conditions are met. Meyerson et al. (1996) argue that, firstly, swift trust is less about the interpersonal relationship factors that underpin traditional forms of trust (built up over a prolonged period of time), and more about encouraging a focus on goal achievement by facilitating the ability of participants to understand their respective contributions to a superordinate (overarching) team managing complex evolving eruption consequences. Secondly, swift trust is more likely to arise when drawing upon a small pool of representatives who have an increased chance of future interaction, with this condition creating a social setting that can foster quicker trust building between parties. Finally, swift trust avoids personal disclosure in favour of a reliance and focus on key tasks that relate to the features of the setting (i.e., the need to integrate diverse organizational and professional perspectives to tackle specific response issues as a member of a superordinate team). If all members of the superordinate EOC organization, incorporating the VSAG, adopt these roles they are more likely to be able to develop trusting relationships that facilitate effective information exchange and utilization in high risk, evolving crisis events.

Evidence of swift trust first emerged from Goodman and Goodman’s (1976) observation that some temporary groups did not have a history of trust, but developed “swift trust” through task related interaction. Evidence for the effective role that swift trust can play in multi-agency and distributed management systems comes from research into global virtual teams that exemplify temporary organizations; and the requirement for collaborative team management further supports its utility (Coppola et al. 2004; Crisp & Jarvenpaa 2013; Robert et al. 2009; White et al. 2008). The concept of swift trust has only recently been tested in multi-agency natural hazard crisis response contexts (Curnin et al. 2015). This, as well as evidence for its effectiveness in military contexts which involve the collaborative response to emergent, low-time/high-risk demands over time (Ben-Shalom et al. 2005; Hyllengren et al. 2011; Lester & Vogelgesang 2012) suggests it should be included in future volcanic crisis response protocols. Swift trust research in military contexts also highlighted the importance of selecting team members with sufficient status to be ‘heard’ in a multi-agency team context (Curnin et al. 2015).

4 Activities to improve future response capability

In the above discussion, we outlined that effective individual and team response to a crisis, such as a volcanic eruption or unrest period, is characterised by good situational awareness, strong inter-organisational networks, effective shared mental models, and high trust between responding organisations and individuals. To achieve this, and develop a common understanding of each other’s roles, dependencies, and information needs, and the over-all response environment, it is important to undertake multi-organisational and multi-disciplinary planning activities, and collaborative exercises and simulations with all team members and advisors, to help in the development of similar mental models of the task (see review in Paton & Jackson 2002; Doyle & Johnston 2011). Such a comprehensive suite of training and relationship building activities prior to an event, and detailed analysis of event and exercise response, can facilitate future response capability and identify areas for improvement. This is particularly important given the rarity of volcanic and other hazard events, and thus a lack of opportunity for real world experience.

According to Kozlowski (1998), p. 120–122, team training should be considered as a sequence or series of developmental experiences that are carried out across a series of different environments, to build “knowledge and skills in an appropriate sequence across skill levels, content and target levels”. Ideally this training and exercising needs to develop both individual and team situational awareness (SA) and explore how and when each is appropriate for response, within evolving, dynamic response environments. Team SA can be developed in post-event and post-exercise reviews that include identifying inter-agency relationship issues as opportunities for development (and not as problems). Through the analysis of past events, lessons for successful communication, advice provisions and distributed decision-making can also be learnt. However, these training activities need not necessarily develop shared mental models and the capacity for true levels of collaborative management etc. Agencies can also use them to update, write, and prepare plans, identify potential or existing issues with the response, logistical, and communication plans; while also testing such processes, systems, and communications. Adopting a suite of training activities increases opportunities for developing an understanding of the technical issues involved and the multi-agency context in which they occur (Borodzicz & van Haperen 2002).

Several training methods have been identified that can enhance naturalistic decision-making (e.g. Cannon-Bowers & Bell 1997), enhance decision skills (e.g. Pliske et al. 2001), train effective teams (e.g., Salas et al. 1997b), and develop effective critical incident and team based simulations (e.g. Crego & Spinks 1997; see review in Flin 1996, chap. 6), all of which are relevant for volcanologists and VSAGs. These include cross training, positional rotation, scenario planning, collaborative exercises and simulations, shared exercise writing tasks, and ‘train the trainer’ type tasks; in addition to workshops, seminars, and specific knowledge sharing activities.

We briefly discuss below two methods in particular: cross-training and scenario planning, as we feel that they are particularly suitable for volcanic response environments. Exercises, and the application of them to science response, are discussed in detail in Section 5, including an evaluation of the lessons learnt from Exercise Ruaumoko in the context of the key competencies discussed in Section 3. It is important to highlight that for all these, it is not just knowledge and skill development that is addressed through these activities; they also address “how the disaster context influences performance and well-being” (Paton et al. 2000, p. 176). In addition, each of the training activities can be carried out at the many levels of a response, for teams within an agency, for the entire organisation, across multiple organisations, and for the full multi-organisation response.

4.1 Cross training

Cross training enhances the awareness and knowledge that each team member has of their fellow team members’ tasks, duties and responsibilities and facilitates the holistic (shared mental model) understanding of team functioning and the respective, interdependent role of a given agency within the team (Marks et al. 2002; Schaafstal et al. 2001; Volpe et al. 1996). This is termed their interpositional knowledge (IPK). Volpe et al. (1996) reason that IPK allows team members to “anticipate the task needs of fellow team members”, leading to more effective team performance and enhanced coordination with a minimal communication requirement, important when task loads are high and individuals are too busy attending to these to be able to make explicit information requests. In the absence of IPK, there exists interpositional uncertainty which can hamper team performance (Blickensderfer et al. 1998).

Cross training is “an important determinant of effective teamwork process, communication, and performance” (Volpe et al. 1996, p. 12). Teammates who develop interpositional knowledge through cross-training: 1) interacted more effectively with each other, 2) used more efficient communication strategies, and 3) volunteered information more often (Blickensderfer et al. 1998). It facilitated these outcomes by “encouraging members to understand the activities of those around them” (Blickensderfer et al. 1998), to better anticipate their needs and assist those in need of help (see Table 2 and Schaafstal et al. 2001; Marks et al. 2002). Furthermore, cross-training can foster a sense of a shared “common bond” (Greenbaum 1979, as cited in Blickensderfer et al. 1998) amongst team members and support the establishment of morale, cohesion and confidence.

Table 2 Lessons learned from studies on cross-training teams (Blickensderfer et al. 1998 )

Cross-training encompasses three methods that become progressively more detailed and involved, and thus more effective for improving shared mental models, understanding of complementary roles, and enhancing collaboration (Blickensderfer et al. 1998; Marks et al. 2002). These are: 1) positional clarification, a form of awareness training (e.g., by discussion, lecture, demonstration or dissemination of information) where specific information is provided about other roles and responsibilities in the team (e.g., working together in an EOC, or in a science response team, for example); 2) positional modelling, a training procedure in which the duties of each team member are discussed and observed via behaviour observation (e.g., offering potential EOC participants some actual practice in the other positions: a volcanologist could be given the opportunity to play the role of, for example an intelligence or logistics officer in an EOC context); and 3) positional rotation, which involves training within the exercise context where all team members spend significant periods of time performing other team members’ jobs and roles, providing a working knowledge of each member’s specific tasks and how those tasks interact and to gain different perspectives of the overall situation. The chosen cross training method amongst these three approaches should correspond to the realistic level of interdependence of the team (Ford and Schmidt 2000).

4.2 Scenario planning

Scenario planning is a technique that creates multiple scenarios of “different futures” in ways that accommodate the perspectives of multiple agencies (i.e., to develop response scenarios that more accurately reconcile the needs, goal and expectations of diverse agencies; Bloom and Menefee 2014; Moats et al. 2008; Paton 2014). At a fundamental level, scenario planning allows an “organisation to examine several options or risks that might have been overlooked in a plan that was constructed around a single environment. The process forces managers to think about the unthinkable and to even plan for it” (Bloom and Menefee 2014). Through this process, the goal is to outline possible futures that are “credible and yet uncertain” (Keough and Shanahan 2008). Various alternative steps to the scenario planning process are outlined in Table 3.

Table 3 The generic scenario planning model of Keough and Shanahan ( 2008 ), and an example of a scenario building model for step 3 given by Schwartz’ 8-step approach (Schwartz 1996 ; as cited in Keough and Shanahan 2008 )

Scenario planning enables the integration and awareness of the various social, political, economic, cultural, and other environmental forces that underpin the histories and expectations different agencies bring to the response management environment. It also provides an opportunity to ‘rehearse the future’, promoting adaption versus reaction, and providing a safe space through which various points of view and new or unique ideas from within the team can be shared without the fear of being prejudged or automatically dismissed (Bloom and Menefee 2014). Through this process, the volcanic crisis management team can also enhance their understanding of the wider response process, the key issue and decision thresholds, and trigger points; thus facilitating their collective ability to integrate their various perspectives and develop a bigger picture of the response than would arise if this was based on an individual role or stakeholder working separately. This can enhance shared situational awareness of the issues and process amongst the participating team. Scenario planning can be conducted at the sub-agency level (e.g. a monitoring team at a volcano observatory planning the response and deployment of personnel and instrumentation to facilitate effective monitoring during outcomes of an unrest scenario), at the agency level (e.g. a government science agency and university identifying how they could share resources and support each other during a response) and the multi-agency level (e.g. the VSAG considering potential eruption and response scenarios with the local CDEM group). In particular, for scientists, it provides an opportunity to work alongside emergency managers to identify science information needs and impacts of that information on decision outcomes, thresholds and trigger points.

5 Exploring exercises in detail

In the field of emergency management, exercises to test response and procedures, and to train personnel commonly fall into one of the five categories listed in Table 4, from the small scale in-house orientation exercise to a full-scale multi-agency exercise conducted at the local, national and international level. These different types of exercises may then each be conducted at the different levels of the CDEM structure, for example in NZ they are (MCDEM 2009a):

• Tier 1: Local exercise (individual organisation)

• Tier 2: Group exercise (within CDEM group)

• Tier 3: Inter-Group exercise (across CDEM Groups, may include MCDEM)

• Tier 4: National exercise (New Zealand or part thereof, including central government).

Table 4 The five types of emergency management exercises, as described by MCDEM ( 2009a )

In NZ, MCDEM runs a voluntary participation National CDEM Exercise Programme which exercises at all levels of the CDEM structure listed above, where each tier exercise is informed by a “consistent regime of planning, observation, evaluation, feedback and continuous improvement” (MCDEM 2009a, p. 11). These are run within a 10 year schedule of exercise programming, with Tier 3 exercises every second year and Tier 4 exercises in the intervening years (MCDEM 2009b). Individual organisations can participate in each of these exercises to the scale and scope they desire (e.g., ranging from a small scale in-house orientation exercise through to a multi-agency full scale response, Table 4).

Examples of recent Tier 4 exercises in NZ include the CDEM Exercise Tangaroa conducted in NZ to test the national response to a national tsunami warning in 2010 (MCDEM website 2014a; Coetzee & Gale 2010), and Exercise Ruaumoko which tested an all of nation response to a volcanic eruption in the Auckland Volcanic Field (discused earlier, MCDEM 2008). An example of Tier 3 is the planned exercise Te Matau-a-Maui functional earthquake exercise to be run by the Hawke’s Bay CDEM group to exercise the multi-organisational cross CDEM response to a MMVII scale earthquake in the region. An example of a Tier 2 is the (regional) Bay of Plenty CDEM group exercise of a severe weather event involving a storm surge leading to flooding and significant infrastructure damage in the region with a goal to exercise lifeline utility business continuity plans.

A full list of previous and planned exercises within the NZ CDEM sector can be found on the MCDEM website (2014b). These Tier 2, 3 and 4 exercises involve a wide range of co-ordination, collaboration, and considerable inter-organisational planning depending on the Tier level. In comparison, Tier 1 exercises involve just the individual organisation, such as a university exercising its response to a critical incident on campus (such as an earthquake or armed intruder). Similar tier structures are used by other organisations both nationally and internationally (e.g. FEMA’s National Exercise Program, FEMA 2007; FEMA website 2014). For example, in NZ, Maritime New Zealand - Nō te rere moana Aotearoa (the Crown entity for Maritime safety, regulation, and emergency response) prepared contingency plans and runs exercises following three tiers depending on the level of responsibility: Tier 1 - industry (ships and onshore/offshore oil transfer sites), Tier 2 – regional councils, and Tier 3 – Maritime New Zealand.

Further to these national government led exercise schedules, there are a number of international collaboration programs to exercise response across nations. For example, Exercise Pacific Wave is run by the International Tsunami Information Center every two years to practice the sharing of information, warnings, advice, and resources while practicing government led decision making for a tsunami in the Pacific (UNESCO website 2014a). Similar exercise schedules exist for the Indian Ocean and Caribbean Sea, as well as the Mediterranean (UNESCO website 2014b). These provide useful examples of multi-agency international exercises, with a wide variety of organisations participating to various degrees (e.g. as watchers, partial participants, or full exercise participants). For example, 34 Pacific countries directly participated in Exercise Pacific Wave in 2008, and 10 additional countries ‘watched’ the exercise (i.e. received warning messages), which simulated the response to a tsunami induced by a M9.2 earthquake off northwest Japan (UNESCO website 2014c). The purpose of this exercise was to evaluate and improve warning systems, and to improve the effectiveness of ‘Member States’ in responding to a destructive tsunami, including exercising of their operational lines of communications, warning systems, emergency response procedures and decision making, as well as to promote emergency preparedness. Exercises such as these could inform future multi-national global exercises in the volcanological community to enhance and inform best practice and protocols for international collaboration and response to an event that crosses country borders. This could be led by an international organisation like the International Association of Volcanology and Chemistry of the Earth’s Interior (IAVCEI), to simulate and integrate the full scale global volcanological response to a Black Swan ^b volcanic event: such as a VEI 7 caldera eruption and the associated ash cloud, or managing the impacts of a fissure event like the 1783 Laki eruption.

The involvement of science advisors in these exercises is vital for the development of shared understanding and shared mental models between scientists and critical decision makers, as discussed above. For example, the Ruaumoko exercise in NZ (MCDEM 2008; Smith 2009; Lindsay et al. 2010) involved science organisations (such as GNS Science) from exercise inception and scenario design, through to activation and response, and were used as opportunities to practice response, explore science advice, and try decision making tools relating to evacuation. However, it is vital that the scientists fulfilling the response role are rotated through the exercise, to represent realistic shift rotation in a prolonged event, and across exercises to ensure that as many potentially responding scientists as possible have the opportunity to practice these roles and build relationships with critical decision makers. Further, as discussed in Section 3, this rotation will enable performance management techniques to be practiced and refined, such as role swapping, shift rotation, rotation of the lead science agency or individual, and the balance of time and resources dedicated to the ‘response role’ and the ‘day-to-day research role’. These become increasingly important during long duration responses providing resilience to the response (against untoward events, as well as contingent demands), while enabling a sharing of the load, management of competing or conflicting demands (both professional and personal) and maintenance of a fresh perspective via fatigue management.

Further, it is important that scientists, science agencies, and science advisory groups also lead and develop their own in-house and cross-science agency exercises to develop competencies around the science response itself. Thus we now review existing best practice guidelines for writing and evaluating such exercises.

5.1 Writing and evaluating exercises: practice guidelines

There are many guidelines existing in the emergency management and government literature that provide distinct steps to effectively write exercises (see Table 5). These all recommend the adoption of an exercise cycle, an example of which goes through the steps: 1) analysing the needs and outcomes desired for the exercise, which may be in line with process or performance improvement plans (this step may include a ‘foundation’ step assessing the status quo); 2) design and development of the exercise; 3) conducting the exercise; and 4) evaluating the exercise (see Figure 3).

Table 5 A sample of best practice guidelines available online for the design, writing and implementation of exercises (all websites last accessed on 23 April 2014)

Figure 3

The cycle of continuous improvement and the exercise management model, after the Australian Emergency Management Institute (AEMI 2012 ) and the Ministry of Civil Defence and Emergency Management (MCDEM 2009a ). Variants of this cycle are provided by each of the guidelines described in Table 5.

It is important that exercises do not just consider an event building up to a crescendo (e.g. the unrest period leading up to an eruption), but also consider non-events (the stalled eruption), response and recovery periods after an eruption (e.g. the period after a ‘blue sky’ eruption), or the issues of having a second volcanic crisis occurring alongside the focus volcano (e.g. in 2012 during the Te Maari unrest period in NZ, White Island also increased in activity requiring alert level changes). The volcanic events in NZ in 2012 provide a useful exercise scenario to examine the distribution of resources and personnel across two concurrent crises, and to identify more overt strategies for stress and fatigue management.

Exercises and simulations afford opportunities for agency representatives to develop, practice and review technical, management and team skills under realistic circumstances and to construct realistic performance expectations, and provide opportunities for developing creative problem solving competencies (within and between teams). They also provide opportunities to practice dealing with high pressure situations in a safe and supportive environment, increase awareness of stress reactions, and to rehearse strategies to minimize negative reactions (Flin 1996). However, many exercises fail to mirror the complexity of disaster response environments (Paton and Auld 2006).

There is a tendency sometimes to write these exercises like a ‘play’, providing too much structure and tailored or specific information. However, a truly challenging exercise, that provides the greatest opportunities for learning, should provide incomplete information, conflicting aspects, gaps in the situation description, ambiguous information, and unknowns; all creating an environment where participants are asking questions, not just finding solutions (Paton and Auld 2006). For example, a monitoring team exercise could provide a situation in-exercise where one or many data feeds abruptly end. This provides participants a situation to address whether a machine or data relay has broken, or whether a significant event has occurred, allowing them to exercise the situation of uncertainty and potential decision and action strategies. Exercises should be designed in ways that allow individuals and teams to progressively push their boundaries and develop progressively more sophisticated competencies and relationships (Paton and Jackson 2002). Only through exposure to exercises that challenge assumptions and allow personnel to confront novel events can preparation enhance response effectiveness for eruption events whose complexity will exceed even the most extreme simulation (Paton and Auld 2006).

Thus these exercises should aim to reproduce reality as closely as possible, so decision makers and experts can experience the needs and realities of the advisory process in turbulent conditions (Borodzicz and van Haperen 2002; Rosenthal and’t Hart 1989). However, this alleged realism can also be a danger as personnel may believe at the end of an exercise that they know what will happen in a real crisis (Borodzicz and van Haperen 2002). Evaluation of exercises and events must be carefully conducted to minimise the risk of creating an optimistic bias that overestimates future response preparedness and capability, particularly if a real event has not constituted a major test of the response system (Paton et al. 1998a). The development of an inter-agency exercise database would greatly assist this process.

Guidelines for evaluation and debriefings can be found in the emergency management literature (see Table 5 and Sinclair et al. 2012a; Sinclair et al. 2012b). However, these tend to focus on Key Performance Indicators (KPIs) and specific tasks, response plans, and communication process or outcomes and content evaluation. To ensure these exercise experiences truly enhance team response and development of a common understanding across the team, it is important to also consider team and group learning (Owen et al. 2013), as well as the assessment of individual learning processes such as the progression from unconscious incompetence (‘I don’t know what I don’t know’) through to unconscious competence (‘my skill has become second nature’) (Conger and Mullen 1981; Morell et al. 2002; Thomson et al. 2006). That is, an effective exercise is one that throws up several unanticipated issues (e.g., regarding information management, decision making, team functioning etc.) that can inform future training needs analyses and training and exercise design. Thus, exercising and scenario-based training should be regarded as an iterative process, where the debrief and evaluation is a vital part of that process to enhance the knowledge and skills of the participants, and to help them change their world view and individual paradigms, whilst also minimizing weaknesses caused by problems with exercise fidelity, such as a poor scenario or lack of reality (see review in Moats et al. 2008).

5.2 Exercise Ruaumoko: the importance of exercises and evaluation

Evidence for the benefits of exercises can be gleaned from Exercise Ruaumoko; benefits identified included opportunities to test the Auckland Volcanic Scientific Advisory Group (AVSAG, McDowell 2008). It was a valuable learning experience for all concerned, illustrating the criticality of Auckland to the New Zealand economy and the vulnerability of key infrastructure elements (MCDEM 2008, p. 7). Further, the process of preparing for the Exercise motivated the formalisation of the AVSAG structure (McDowell 2008) and helped to improve the understanding and performance of “individual elements” within a major CDEM operation (MCDEM 2008). However, one of the greatest benefits of the exercise was that it highlighted the need for considerable work “to co-ordinate the various elements” of the inter-agency response into a “cohesive [future] response” (MCDEM 2008). This exercise thus allowed responding agencies to identify response, network and communication issues that need to be modified for effective response and the importance of testing assumptions under realistic crisis conditions, such as high stress, short time, and low resource conditions with competing demands, to demonstrate faults or potential faults in the system. We thus now further discuss Exercise Ruaumoko in detail to provide an example of an exercise structure and to illustrate the benefits of effective evaluation.

Exercise Ruaumoko was a MCDEM led exercise, with over 1,500 participants from approximately 125 participating organisations (MCDEM 2008, p. 8). It was the second in a series of ongoing “All of Nation” exercises led by MCDEM (MCDEM website 2014c). Exercise Ruaumoko also provided a focus to develop and formalise the AVSAG structure (which was co-ordinated and arranged by Auckland CDEM), while also implementing Terms of Reference, and arranging formal contract agreements for the participating scientists (Cronin 2008); and to test such processes within a scenario. The three core objectives of the exercise were to: 1) understand, develop, and practice the respective roles and responsibilities of local, regional, and national agencies in response to the exercise scenario (McDowell 2008, p. 7; MCDEM 2008, p. 5); 2) embed the planning arrangements in standard processes for all participating agencies; and 3) confirm the connections between local, regional, national and international agencies. In addition, it enabled staff at the key levels to “practice and develop planning and the management of response activities using the connections and procedures that are in place”, and also practice the roles, responsibilities and procedures that are included in the National CDEM plan (MCDEM 2008, p. 55). The key sub-objectives of the exercise most relevant to science advice included the examination and testing of the coordination of science aspects and the management of public information and education. While Exercise Ruaumoko was primarily written and co-ordinated by MCDEM with Civil Defence objectives, it still provides a useful example for volcanologists of a large scale volcanology exercise, and the monitoring information and science response parts could be “re-exercised” or used as inspiration for a smaller scale science led exercise.

One of the unique aspects of Exercise Ruaumoko was its timeline, which involved periods of “unfelt earthquakes” in November 2007 and then again between 3^rd to 7^th March 2008. This was followed by a week of increased activity and full exercise play from Monday 10^th to Friday 14^th March, 2008 (MCDEM 2008, p. 11; see Table 6). Throughout this scenario, GNS Science and its duty volcanology team received raw geophysical data such that they could directly exercise their analysis and interpretation processes, participate within the AVSAG response, practice their own decision making, and produce Science Alert Bulletins, which in turn formed injects for the rest of the exercise participants (CDEM and other response and welfare agencies). The scenario focused on the lead-in to a volcanic eruption, stopping shortly after the eruption started (MCDEM 2008, p. 8), thus the concept of the exercise was to provide for pre-emergency planning rather than operating in the traditional post-event response only mode often typified of exercises (MCDEM 2008, p. 9). Detailed information regarding the exercise management and planning process, which started in 2006, can be found in MCDEM (2008). Great lengths were undertaken to ensure this exercise was realistic, with a scientist “acting the volcano” and providing monitoring data injects, and the eventual outcome (including vent location in this monogenetic field) being kept secret from all participants. The wide number of participating organisations added realism in terms of inter-organisational communication and networks (Section 3). However, realism in this exercise was reduced by AVSAG and the GeoNet “actors” not being expected to participate in the exercise outside normal business hours. On the night of Thursday 13^th, this became a particular issue as the emergency managers and decision makers continued to conduct exercise “play” throughout the night in the absence of the science advisors.

Table 6 The scenario timeline of Exercise Ruaumoko (MCDEM 2008 )

Cognisant of this issue, there are still many lessons that were identified from exercise reviews that have direct implications for the improvement of future response. Exercise evaluations we consider here include: the in-house GNS Science review of the AVSAG and GeoNet process, a NZ volcanologist who acted as an observer to the exercise including all AVSAG meetings and conference calls (Cronin 2008), MCDEM’s all of exercise evaluation (MCDEM 2008, and Table 7), and Auckland’s’ Civil Defence Emergency Management Group evaluation report (McDowell 2008). Several key issues were observed and lessons identified which we now discuss in reference to our understanding from Section 3 of situational awareness, mental models, trust and decision making for effective response, as follows:

• The AVSAG was constructed of a tri-partite sub-group system (Volcano Monitoring, Volcanology & Social Consequence, see Section 2.1), which by its nature had GeoNet (the monitoring arm of GNS Science, legislated to provide official advice) sitting as a separate sub-group to the volcanology sub-group which encompassed university and other Crown Research Institutes, Science agencies (e.g. MetService) and GNS Science; with meetings between these groups conducted via regular teleconferences. An extremely positive aspect of the AVSAG structure was its inclusiveness (McDowell 2008), however it was seen as being too “cumbersome during the escalation of events in the scenario” (Cronin 2008, p. 6). In the early stages of the exercises, information flow between these two sub-groups was adequate. However, as the crisis progressed, information transfer reduced considerably and there was poor communication (Cronin 2008; McDowell 2008). This represents a significant issue for effective individual and group situational awareness and would have impacted upon shared mental models of the situation, resources required, and decisions to be made (Section 3).

• AVSAG tasks were driven by questions provided by the Auckland EOC and the National Crisis Management Centre. As stated by Cronin (2008), very little additional advice was provided, even though some questions were not relevant. At times the questions were superseded by events, and by concentrating on these questions participants felt they “had excluded additional important comments” (Cronin 2008, p. 5). The exception to this was the social sub-group, who did pre-empt questions. When we consider this in light of our discussions in Section 3, we can see that this information sharing is typical of explicit information requests and represents a weak shared mental model of the issues and information needs, much like the ad-hoc communication characterised in the response to Ruapehu 1995–1996 (Section 3.3, & Paton et al. 1998a). This demonstrates a need to conduct further activities to develop shared mental models further, and strengthen inter-organisation networks and communication.

• Due process outlined in AVSAG was followed in situations when novel approaches would have been more appropriate (Sections 3.1 and 3.3). For example, as the exercise reached crisis, AVSAG followed process by only responding to explicit requests from the decision makers (Cronin 2008), to the point that events actually superseded their responses. Failure to recognise the implications of future status on the changing need for information represents a failure of situational awareness (in terms of future status), and weak shared mental models (understanding other demands on the recipient; Section 3). The in-house GNS Science Review also cautioned that very structured meetings could constrain science discussion, which would thereby limit full debate and exploration of novel considerations and more creative decision making processes. This highlights how decision making undertaken within normal scientific research (long time for decisions, considerable debate, more analytical decision processes) differs from that in emergency management environments (rapid decisions for public safety in high risk/low time contexts which call for more recognition primed and intuitive decision processes). It was thus felt that the AVSAG process was good for ‘peace-time’ but unwieldy “when rapid response is required”.

• At the crisis point of the exercise, as evacuation planning occurred, AVSAG was “side-stepped” despite there “being new scientific information to hand about the eruption location and possible eruption styles” (Cronin 2008, p. 4). This was a direct reflection of the tri-partite structure of AVSAG failing at the heat of the crisis, and GeoNet carrying out two roles: 1) as the monitoring arm within AVSAG, and 2) being mandated to be the main information provider to government, the latter of which took priority. The escalation of the crisis resulted in a perception that time was too short for wider deliberation (Cronin 2008; p. 5), and resulted in a focus on the rapid assessment and decision making in relation to technical data alone (McDowell 2008). This however resulted in a lack of integration across the wider volcanological community represented in the volcanology sub-group of AVSAG, and thereby a breakdown of shared situational awareness (Section 3.2) and an artificial division between these scientists (McDowell 2008).

• The role of liaison officers (GeoNet duty officers) at the Auckland Group EOC and NCMC was vital for communication, and information sharing, (see Sections 3 and 4). Auckland Civil Defence and Emergency Management Group (ACDEM) (McDowell 2008) identified that it was highly beneficial to have a scientist in the GEOC as they provided instant integration of science advice into their decision making. The presence of the liaison officers would have enhanced the situational awareness of the emergency managers, and communication within the GEOC most likely moved towards an implicit supply of information by the liaison officer as they recognised a need for their advice. However, as discussed in Section 2.1, as the crisis progressed the information delivered by the two liaison officers diverged, an artefact of the breakdown of shared situational awareness between their two dispersed geographical locations as the time and risk pressures increased. As stated by Cronin (2008), p. 7, with this approach “there is no guarantee that the two EOC’s will receive the same advice”, and in the exercise this resulted in actions by the NCMC that were a surprise to the Group EOC. This demonstrates the importance of consistent science advice due to the impact any divergence may have upon cohesive integrated emergency management decision making and response. From this, the reviews identified a need for policy to be developed for periods of extreme urgency to ensure “only one representative communicates the same message to National and Group controllers”.

• Reviews also identified a need for clarity around probabilities and uncertainty, and that there had been a use of geological terminology and phraseology that was not understood by emergency management, resulting in the recommendation to identify what the emergency managers need and how they use this information (see also Doyle et al. 2014a, b). As stated by MCDEM (2008), p. 50, “there is a need for further analysis and/or translation of primary science information to limit misinterpretation and to ensure it meets the needs of all organisations”.

• Communication infrastructure issues were also noticed, and a need for clarity around communication and reporting channels, as well as ensuring documentation was readily available for participants not able to attend meetings. This is vital for maintaining good shared situational awareness of the evolving crisis.

• Work load was identified as an issue for responding scientists due to their dual roles (both response and day-to-day roles) as well as the multitasking demands placed upon them within the EOC. This indicates a need to consider the use of multiple liaison officers within an EOC, to share these demands, while both assisting with the maintenance of situational awareness within the EOC regarding developments of the science and data inputs. However, messages must be consistent when there are multiple liaison officers within an EOC, to prevent the potential divergence of advice discussed above.

Table 7 Recommendation 14 “Understanding Volcanic Hazard and Communicating Science Advice”, from the 17 recommendations suggested by MCDEM after Exercise Ruaumoko (MCDEM 2008 , p. 50–51)

From these observations, and this comprehensive testing of AVSAG, specific recommendations for improving the science advice provision and response included (MCDEM 2008; McDowell 2008; Cronin 2008; Table 7):

• The need to investigate the options for a merged volcanology-subgroup encompassing the GNS Science/GeoNet monitoring arm in such a way that the overall group is not too cumbersome, but such that all expertise can be involved in the response.

• Recognising and ensuring that both the volcanology and monitoring sub-groups of AVSAG have access to the same data resources (which we also note would help improve shared situational awareness).

• That MCDEM must work with CDEM and Science agencies to 1) consider options for integrating local and national science capabilities and processes, 2) facilitate collaborative planning by science agencies, including universities, for post-event science investigations (Smith 2009, p. 76) while also meeting individual agency requirements (MCDEM 2008, p. 50).

• Adopting a structure where GeoNet (which has an always active on-call duty team) has the main advisory role, whilst “co-opting external AVSAG scientists and the GeoNet team into a volcanology/monitoring SAG” as needed (Cronin 2008, p. 9). This aims to ensure the wider NZ expertise can contribute to the understanding of the situation and incorporates locally based scientists into the response process.

These observations clearly illustrate the benefits of exercises that truly test scientific response, VSAG structures, and inter-organisation communication and decision-making responses. The ‘split’ observed between the monitoring and wider volcanology subgroup of AVSAG is interesting to consider in the context of the influence of group identity (Hogg 1992) and the ‘stereotyping of in- and out- groups’ that can affect collaboration in a response (Paton et al. 1998a) and reduce the capacity to develop swift trust.

As briefly introduced in Section 2, in response to these exercise evaluations, Smith (2009) (on behalf of MCDEM), outlined a potential alternative model provided by a NZ Volcanic Advisory Panel (NZVAP), to avoid such a breakdown in communication, as well as to integrate local and national science research response (Figure 2b). This model was developed through a MCDEM facilitated dialogue between volcanologists to identify how the skills of scientists in the range of different organisations (including universities, Crown Research Institutes, consultancies and councils) can be integrated “in support of a national science agency such as GNS Science that has responsibilities under the National CDEM Plan for providing warnings and advice” (Smith 2009, p. 76). This national advisory model aims to mobilise science capability while remaining responsive to local CDEM needs (ibid, p. 77), and the intention of such a national mechanism is not to override any existing scientific or planning advisory groups at the local level, but rather that it would be complementary and “provide for a level of consistency for how New Zealand-wide science capability is mobilised when a large-scale science response is needed”. Other benefits of such a group include: a) facilitating coordinated post-event investigations and data sharing arrangements, b) providing strategic advice on research direction and priorities, c) fostering connectivity across the physical and social sciences, and d) supporting alignment between researchers and research users (see also Jolly and Smith 2012).

This proposed model was still under consideration and discussion when the Canterbury earthquake sequence started in September 2010. This sequence required an extensive response of many of the same scientists and agencies, and thus the development and implementation of such a NZVAP model has been put on hold until extensive reviews of the inter-agency response to those earthquakes were conducted (including both a Royal Commission of Inquiry and a Coronial Inquiry) and until the science advisory process during those earthquakes, and the more recent volcanic eruptions of Te Maari in 2012, have been fully evaluated by agencies and through various research projects in process (see also Section 2). However, until those real event reviews are available, Exercise Ruaumoko represents a clear case study of how exercises can be used to test a new VSAG structure, identify issues in that structure and pave the way for the development of a new structure. As stated by MCDEM (2008), p. 55 “there were major achievements in the preparatory phase [of Exercise Ruaumoko] that simply wouldn’t have occurred without the context of the exercise”. After formalisation of any new NZVAP structure, the next step should be test this through another exercise (of a different scenario) to complete the exercise cycle (Figure 3) and implement any lessons from that as part of a regular evaluation of science advisory mechanisms.

6 Discussion: a new exercise structure for volcanology

It is important that science agencies both participate in and lead exercises of all scales and types (Table 4) due to the many benefits discussed above. In particular, benefits can arise from smaller exercises that may be more suitable for restricted budgets. For example a full-scale exercise the scale of Ruaumoko would require significant exercise resources in terms of both personnel and costs. For that exercise, GNS Science contributed both exercise writing and preparation time of approximately 2 weeks for 2 people, and in the exercise itself approximately 18 hours of meeting time (plus associated data analysis time) for 10 people. Thus, in total, approximately 300 hours of professional time were contributed towards exercise development, exercise play, and evaluations and debriefs (detailed costs were not available, thus these are estimates only). This equates to approximately 70,000-100,000 NZD for a science agency to participate in a full scale CDEM led national exercise like Ruaumoko with realistic timelines (including a full week of exercise play). Equivalent costs for MCDEM (as exercise writing management team, as well as in response) would be in the range of 1–1.75 full time equivalent (FTE) staff members for 12 months of exercise preparation, escalating to 3FTE for the last 3 to 4 months and then full team engagement in the exercise play itself. These FTE staffing costs would be in addition to operational expenses (in excess of approximately 200,000 NZD) for workshops, briefings, travel, report writing, and public education material. Meanwhile for regional/local CDEM, typical costs for participating in such an exercise would depend on their exercise involvement, ranging from 500,000 NZD for full scale participation in a Tier 4 exercise (Auckland CDEM during Exercise Ruaumoko) through to 5000–10,000 NZD for smaller Tier 1 and Tier 2 exercises (e.g. Wellington Regional Emergency Management Office, based on regular Exercise Phoenix series). Thus it is important that agencies consider conducting a cost-benefit analysis when deciding the scale of an exercise, being aware that benefits from conducting exercises include the considerable reduction of personnel and financial costs during a real response due to the increased capability developed in the exercise.

It is clear from the above that full scale participation in CDEM led national exercises, while extremely beneficial for the future response capability of scientists and volcanologists, can be very expensive in terms of resources and professional time. It is thus important that scientists do not just participate in CDEM led exercises, practicing in liaison roles and on expert panels, but also exercise their own agency and inter-agency response as well. By exercising within their own agencies, scientists can enhance intra- and inter-agency collaborations. By developing, writing and leading an exercise, scientific agencies can design exercises to meet their core goals in terms of capability building and competency testing; to tailor them to meet their needs for skills and process development; to meet their available budget by providing opportunities to develop smaller, less costly exercises to test processes within just a small team (e.g. a monitoring team could practice their response, analysis and decision making responses). These smaller exercises are still greatly beneficial to response capability and to improved shared mental models of the response environment within that team. However they currently occur in an ad hoc manner, with no co-ordinated Tier structure like that of the CDEM sector (Section 5). We thus suggest there is a need to formalise such a structure for volcanology to help promote a regular exercise cycle, and to help integrate such processes into CDEM frameworks, as well as to encourage documentation of such exercises for future shared use, and propose an equivalent scale as follows:

• Level 1 – In-house: within-team or within-group exercise.

• E.g. exercise the response of a specific team, such as an agency monitoring team or a university research group, to an event such as a lahar breakout and the associated opportunity for the rapid collection and dissemination of data. This could involve story boarding the response and requirements through a facilitated table top exercise, through to a functional exercise or practice run involving mobilisation, data collection procedures and radio communication tests.

• Level 2 – In-house: within-agency exercise.

• E.g. exercise the agency response to an eruptive episode, including the various responding monitoring teams within this organisation, such as: geochemistry, geophysics, geodesy, and social sciences, as well as those responsible for issuing monitoring statements, developing forecasts, and providing impact and warning advice to the public and key decision makers. Through this, an organisation can develop an understanding of the dependencies that each of these teams or groups have upon each other and their shared requirements to perform their role, as well as developing relationships across these roles. In addition, abilities to continue the agency’s day-to-day duties alongside this response can be explored and issues identified. Examples of such an exercise range from a small facilitated table top exercise to practice a monitoring meeting and the resultant change of an alert level, through to an inject led functional exercise utilising monitoring injects throughout a prolonged period of time.

• Level 3 – Regional or National: across-agency exercise.

• E.g. exercise members of a national volcanic science advisory group, for a long duration volcanic unrest period. This advisory group would bring together individuals from across universities, agencies and technical organisations, as well as any nationwide monitoring agency, and thus it provides an opportunity to exercise both their roles and others within their agencies and organisations who may support them in their role. For example, exercising a volcanic ash fall event provides an opportunity to practice interaction across agencies from the volcanological, meteorological, public health, veterinary and farming sectors, and beyond.

• Level 4 – International: across-agency exercise.

• E.g. exercise the response to large scale ash cloud hazard event. In a manner similar to Pacific Wave, this could involve members ranging from the local weather monitoring and forecast organisation, through to local and international volcanologists, meteorologists, and aviation authorities. Through this the international sharing of information necessary for local, regional and national ash cloud monitoring and forecast can be exercised, as well as collaboration in the preparation and delivery of any aviation warnings.

Each of these exercise levels could involve only the scientific agency, or other civil defence and responding organisations, and may range from orientation to full scale exercises (see Table 4), as appropriate. We use the term ‘Level’ to avoid confusion with the term ‘Tier’ used in the emergency management led exercises. These exercise levels may feed in to any of the CDEM Tier 1 to 4 scales, as desired by the co-ordinating scientific agency. For example, a scientific agency may choose to only exercise their landslide assessment team (Level 1) as part of a Tier 3 CDEM earthquake exercise, or they could choose to exercise all their responding teams to this scenario, including, for example, a full inter-agency tsunami expert panel (Level 3). The key difference for our proposed exercise structure is that the exercise planning and writing is led by the scientific agencies, and not a local, regional or national civil defence agency, and thus is directed by the core goals and competencies that the scientific agency wishes to develop.

Further to these four levels, we also suggest that volcanologists exercise with a different hazard. For example, a team of volcanologists could run 1) a table top flood exercise themselves (with the scenario provided by the appropriate agency) ‘acting’ as flood scientists; or 2) watch or participate in a full scale flood exercise, again playing the role of flood scientists. By exercising an ‘unfamiliar hazard’ (e.g. the rapid, high frequency, sudden onset flood), volcanologists cannot make their usual assumptions, and so the process can help to identify issues and unrealistic expectations of response capability. It also gives these volcanologists an opportunity to learn techniques, procedures and ‘tips’ from these other scientific communities that may be more experienced in emergency response roles. For example, some flood and weather agencies contain scientists who are very practiced at regularly switching from their ‘research’ role to a ‘response’ role, forming rapid response teams, and working alongside CDEM and local governments on a regular basis.

7 Concluding remarks

Through a review of the literature on emergency management team response, decision making, mental models, situational awareness and exercising, particularly in the NZ context, we argue that to truly enhance the science response during a disaster, science agencies, and science advisory groups, must learn from emergency management agencies and embark on a suite of training activities including exercise and simulation programs within their own organisations, rather than solely participating as external players in emergency management activities. Through this in-house training, the future response capability of science advisory groups and agencies can be enhanced, while also developing their true understanding of the needs of the emergency managers and other key decision makers. This enhanced understanding may lead to a modification of existing scientific programmes and projects, with a view to enhance outputs to meet decision makers’ needs in advance of a crisis rather than during the crisis itself. Further, by participating in these activities we can enhance the response capability by developing shared mental models across the team about the response issues, demands, tasks and interpositional knowledge for members within and across teams.

These exercises not only provide opportunities to practice communications and plans and enhance these team mental models, but also for scientific agencies to rehearse strategies to convey uncertainty and how it could be included (Doyle and Johnston 2011; Doyle et al. 2014a & Doyle et al. 2014b), and for other agencies to be able to develop associated contingent planning. In addition, exercise and scenario planning can also help to answer if we are doing the right science for the response as well as for the advancement of science. There is thus a necessity for organisations to provide structure, resources and time to help facilitate and promote such training activities, as well as to encourage performance management techniques within a response, such as role swapping, while also acknowledging our responsibility as scientists to fulfil this response role effectively.

Beyond exercises for response it is also important to acknowledge that such exercises can advance our scientific knowledge and strategies for response performance, learning, and decision competencies. Volcanic scenarios provide an important case study for these research exercises into human behaviour, learning and communication, due to their highly uncertain nature in terms of expected scale, scope, timing and impacts (e.g. Doyle et al. 2011; Dohaney 2013).

There is also a need, through such research exercises, to identify ways to develop ‘swift trust’ in an event. Trust is intrinsic to shared mental models, distributed decision making, and information sharing in a response, particularly during situations of high uncertainty. Identifying strategies to create swift trust goes beyond volcanic response management, and has implications for developing effective multi-agency collaboration when agency representatives find themselves managing any hazard crises without prior contact. For information sharing it also has implications as it moves people from an individualistic perspective (‘my information is best’) to an acknowledgement of the collective perspective and role (‘my information is one piece in the big picture of response’).

Endnotes

^a“The “boiled frog” scenario is referred to often in disaster and business communities to describe “creeping” disasters and crises, it describes a frog that when placed in a pot of cold water which is being gradually heated, will fail to recognize the increasing danger and thus get boiled alive. However, if this proverbial frog is put directly into hot water, it will recognize the dangerous situation and jump straight out of the pot. This metaphor thus describes a number of problems that can arise with creeping disasters: failing to recognize the accumulation of many small changes which can amount to a major crisis; normalization bias; and personnel fatigue and performance issues for long duration events”. (Doyle et al. 2014b, p. 86).

^bA Black Swan event (Taleb 2007) describes those events that 1) exceed our expectations (an outlier), 2) have extreme impacts, and 3) are often ‘rationalised by hindsight’. They are hard-to-predict, surprising rare events beyond “the realm of normal expectations”, often characterised by high uncertainty. The probabilities of such events are often hard to compute due to the nature of the small probabilities involved. A recent natural hazard event which could be considered as a Black Swan event is the 2011 Tōhoku earthquake and tsunami and subsequent Fukushima nuclear power plant crisis; although some state this may not satisfy the first criteria (surprise) of a Black Swan (Lewis 2012). The impact to aviation, travel, and trade during the 2010 Eyjafjallajökull eruption is another example of a Black Swan event.