Enhancing scientific response in a crisis: evidence-based approaches from emergency management in New Zealand

3.1 Decision making

Pivotal to effective volcanic crisis management are the decision making, situational awareness, mental models (which is an individual’s representation or visualisation of a real system, including concepts, relationships, and their role within that system) and trust processes that underpin effective response. We commence discussion with an overview of the individual and group decision making processes occurring in volcanic management and response.

Analytical decision-making is defined by working through a process: identifying a problem; generating options to solve the problem; evaluating these; and implementing the preferred option (Flin 1996; Saaty 2008). This form of decision making requires time to allow this process to occur. It is the default approach adopted by scientists (and managers) due to their training. However, in emergency response a range of other decision making styles: analytical; naturalistic; procedural based; creative; and distributive (e.g. Crichton & Flin 2002); are required and need to be matched to the situation and conditions encountered by decision-makers.

The slower, more considerate, analytical decision making processes lie at one end of a continuum of decisions styles. At the “faster” end of the decision making continuum lies naturalistic decision making (NDM; Martin et al. 1997). This relies on experience garnered through real world crises as well as simulations and exercises (Crichton and Flin 2002; Klein 2008). It is commonly adopted in high risk and low time contexts and naturalistic settings which involve: ill-structured problems; uncertain, dynamic environments; shifting, ill-defined, or competing goals; action/feedback loops; time stress; high stakes; multiple players; and the effects or pressures of organizational goals and norms (Orasanu & Connolly 1993, as cited in Zsambok 1997, p. 5). For critical incident management, research has identified four key NDM processes (Crego and Spinks 1997; Crichton and Flin 2002; Pascual and Henderson 1997): 1) recognition-primed and intuition led action; 2) a course of action based upon written or memorised procedures; 3) analytical comparison of different options for courses of action; and 4) creative designing of a novel course of action; ordered by increasing resource commitment.

In a crisis, uncertainty, environmental change, risk and time pressures are amplified, making decision making (whether by scientists or responders) in this dynamic context one that is dependent on ‘task conditions’ (Martin et al. 1997), and thus throughout one incident different processes may be adopted. For example, during Exercise Ruaumoko (Sections 2.1 and 5.2), the on-site GeoNet duty officers inside both the Auckland Group EOC and the NCMC were often asked to provide answers to questions from key decision makers who required an immediate response due to response management demands. This “task condition” (short time) would have favoured the more naturalistic decision making, where the scientists would have relied upon their experience to assess the situation (both the question and the available science) to make an intuitive or recognition-primed decision (Paton et al. 1999, Klein 1998). However, earlier in the exercise, during the warning period preceding the volcanic ‘crisis’ the science advice could be carefully evaluated and compared, and relatively lower time pressures afforded scientists and decision makers the opportunity to adopt an analytical decision making approach. However, as the situation moved from this early warning phase into the crisis of impending eruption, or for a situation where scientists are responding rapidly after an eruption (e.g. the “Blue Sky” eruption of Ruapehu in September 2007), the more intuitive naturalistic style would again be adopted. As stated by Paton et al. 1999, “attention must [thus] be directed to understanding [this] naturalistic decision-making of experts, and how it can be modelled in simulations to develop this contingent management capability” (p. 44).

3.2 Situational awareness

Pivotal to effective decision-making process is a capacity to: 1) evaluate and define a problem and task characteristic via situation assessment (Endsley 1997; Martin et al. 1997); and 2) select a decision-making strategy from the four options described above (Crichton and Flin 2002). The former process is intrinsically dependent upon the situational awareness (SA) of the individual and the team (Cannon-Bowers and Bell 1997; Crichton and Flin 2002).

Situational Awareness comprises three levels (Endsley 1997, p. 270–271): 1) Perception – understanding the importance of information and cues in the environment; 2) Comprehension – combine, interpret, store, and retain information and be able to use it; and 3) Projection – prediction of future situations from existing and previous situations. Initial and ongoing SA is critical to decision making (Sarna 2002). Thus a decision-maker may make the correct decision based upon their perception of the situation, but if their situation assessment is incorrect then this may negatively influence their decision (Crichton and Flin 2002). When responding to a volcanic eruption, developing and maintaining this SA is important for both volcanologists (in their assessment of available data and future projections, information from other agencies, demands upon their advice) and emergency managers and decision makers (in their assessment and understanding of information - including science advice, resources, demands and future requirements and needs).

In reviews of the inter-organisational response of the 1995–1996 Ruapehu eruptions in NZ (Paton et al. 1998a, 1998b, 1999), several issues affected the SA of both scientists and emergency managers. In particular, “inter-organisational networking” was weak, with none of the responding agencies (e.g. fire, police, civil defence, social services, media, etc.) having an established or formalised inter-organisation network in place with GNS Science before the event, even though GNS Science acted as an information provider for 63% of the responding agencies. This resulted in organisations interacting on an “ad hoc” basis (Paton et al. 1998a, p. 7) contributing to co-ordination and communication problems and preventing their using crucial information to build and maintain SA, and thus impacting both emergency management and volcanological decision-making processes and outcomes. This was compounded by issues such as the “lack of clear responsibility for co-ordination” across responding agencies (reported by 45% of participating agencies), “inadequate communication with other agencies” (37%), and “inadequate co-ordination of response” (32%). These are all indicators of “team breakdown”, inadequately defined and co-ordinated roles, and poor communication (Paton et al. 1998a, 1999). The development of effective inter-organisational crisis communication requires (Paton et al. 1998a) “information needs [that] are anticipated and defined, that networks with information providers and recipients are organised, and [that] crisis communication systems capable of providing, accessing, collating, interpreting and disseminating information are established. … [and] shared terminology and systems” (p. 8).

The development of VSAGs and the associated Terms of Reference within NZ over the last two decades will have helped to build shared mental models of the response environment across organisations. VSAG development has included the identification of protocols for communication and networking with emergency management and key response organisations, and specifying relationship building activities to be undertaken within these groups. These activities have improved communication and information flow during a volcanic crisis and thus the shared situational awareness in that crisis. Comparison of Figures 1 and 2 illustrates the change and improvement in information flow processes between Ruapehu in 95–96 (Figure 1) and Ruaumoko in 2008 (Figure 2).

The rarity of large scale eruptions makes it important to capitalize on the learning opportunities events, exercises, and reviews provide for developing situational awareness and for facilitating the ability of scientific advisors to develop shared mental models of their and others role in response management. This feeds into training needs analysis and the development of the situational awareness competencies and decision support systems required to sustain effective situational awareness in complex, rapidly evolving and dynamic volcanic crises. It can also inform the development of the shared situational awareness required if all team members make their respective contributions to a shared task or goal. That is, to develop shared mental models of presenting problems and response options, particularly when decision inputs come from different professions and/or from participants who are spread over a large geographical area. Facilitating the latter introduces a need for distributed decision making, discussed next.

3.3 Shared mental models

The scale and complexity of volcanic crisis response results in decision-making involving people who differ in their profession, expertise, functions, roles and geographical location. This more integrative decision style is called distributed decision making (Rogalski & Samurcay 1993, as cited in Paton & Jackson 2002). As discussed in Section 3.1, an individual’s mental model impacts individual decision making, as it is their representation of the wider system and processes, including inter-agency relationships, needs and demands, and an individual’s role within a crisis. Thus, for effective distributed decision making, individuals require a good shared mental model of the response environment in time and space, which incorporates how their expertise contributes to different parts of the same plan, and their understanding of each other’s knowledge, skills, roles, anticipated behaviour or needs (Flin 1996; Marks et al. 2002; Paton and Jackson 2002; Schaafstal et al. 2001). By building a shared mental model, team members can develop an accurate expectation of the performance of their team members and themselves, leading to effective coordination without overt strategizing (Blickensderfer et al. 1998; Cannon-Bowers et al. 1998; Lipshitz et al. 2001; Salas et al. 1994).

A significant challenge here derives from a need to coordinate the inputs of different agencies and experts to assist the holistic management of complex hazard consequences. For example, public health specialists possess expertise concerning the specific effects of ash and gas on health. However, to mount an effective response, their input, as members of an ‘emergent’ team, must be integrated with input from, for example, volcanologists, emergency managers, social welfare, and transport agencies, to facilitate understanding of the ‘whole’ problem, prioritise issues, and to identify where is safe for people to be evacuated to, etc. This example illustrates how, for example, public health specialists and volcanologists need to bring their professional team mental model to bear on identifying their specific contribution, but also develop a superordinate (overarching) mental model that integrates all areas of expertise. A significant challenge arises because scientists, and indeed all stakeholders, need to switch between a) being autonomous actors, and b) being multi-disciplinary team members, depending on the task being undertaken (Janssen et al. 2010).

Fundamentally, in a volcanic crisis response setting, the science advisors’ and VSAGs’ role is to provide information to facilitate the response agencies understanding of the hazard issues, priorities, the wider context, impacts, and potential future outcomes, and thus build their situational awareness to aid their decision-making process. However, as stated by Doyle & Johnston (2011), it is not just a case of providing the emergency managers with all available science information, but about understanding their needs to meet their information requirements. Simply providing as much advice as possible may actually hinder the decision process, due to cognitive overload and an overuse of these available resources (Crichton and Flin 2002; Omodei et al. 2005; Quarantelli 1997). To contribute in this way, scientific advisors need to develop a shared mental model with their emergency manager counterparts both prior to an event (to develop effective plans) and during the crisis itself. This shared mental model encompasses the overlapping elements of each team member’s SA and represents the inter-team co-ordination (Endsley 1994), allowing an effective coordination amongst team members without the need for extensive overt strategizing (Salas et al. 1994; see review in Doyle & Johnston 2011).

For science response it is thus vital that scientists develop this shared mental model within the VSAG, and in the wider multi-agency response, to facilitate their ability to implicitly provide the science information required by the main decision makers at critical periods (see also Doyle & Johnston 2011). However, when dealing with the uncertainty implicit in volcanic crises the effectiveness of this information sharing relationship is influenced by the degree of trust that exists, or that needs to be developed in situ, between key players. This is particularly important given the rarity of opportunities for functional organizational interaction before a crisis occurs.

3.4 Trust

Trust plays a pivotal role in developing sustainable, functional relationships when members of diverse organizations need to collaborate to access, share and use information for decision making in response environments characterized by uncertainty (Siegrist and Cvetkovich 2000). Without trust, teams focus on task demands, not teamwork, reducing their effectiveness in tackling emergent crisis response needs (Pollock et al. 2003).

Inter-agency trust develops through collaboration. Since volcanologists and emergency managers rarely work together under normal circumstances, trust among agencies may be lacking (Dirks and Ferrin 2001). Since representatives of scientific and EM agencies typically meet and interact for the first time during a crisis, agency representatives are denied the luxury of building trusting relationships over time. Trust must be developed via other mechanisms. One approach capitalizes on the concept of swift trust (e.g. Meyerson et al. 1996).

Swift trust can be developed in temporary (EOC) organizations if certain conditions are met. Meyerson et al. (1996) argue that, firstly, swift trust is less about the interpersonal relationship factors that underpin traditional forms of trust (built up over a prolonged period of time), and more about encouraging a focus on goal achievement by facilitating the ability of participants to understand their respective contributions to a superordinate (overarching) team managing complex evolving eruption consequences. Secondly, swift trust is more likely to arise when drawing upon a small pool of representatives who have an increased chance of future interaction, with this condition creating a social setting that can foster quicker trust building between parties. Finally, swift trust avoids personal disclosure in favour of a reliance and focus on key tasks that relate to the features of the setting (i.e., the need to integrate diverse organizational and professional perspectives to tackle specific response issues as a member of a superordinate team). If all members of the superordinate EOC organization, incorporating the VSAG, adopt these roles they are more likely to be able to develop trusting relationships that facilitate effective information exchange and utilization in high risk, evolving crisis events.

Evidence of swift trust first emerged from Goodman and Goodman’s (1976) observation that some temporary groups did not have a history of trust, but developed “swift trust” through task related interaction. Evidence for the effective role that swift trust can play in multi-agency and distributed management systems comes from research into global virtual teams that exemplify temporary organizations; and the requirement for collaborative team management further supports its utility (Coppola et al. 2004; Crisp & Jarvenpaa 2013; Robert et al. 2009; White et al. 2008). The concept of swift trust has only recently been tested in multi-agency natural hazard crisis response contexts (Curnin et al. 2015). This, as well as evidence for its effectiveness in military contexts which involve the collaborative response to emergent, low-time/high-risk demands over time (Ben-Shalom et al. 2005; Hyllengren et al. 2011; Lester & Vogelgesang 2012) suggests it should be included in future volcanic crisis response protocols. Swift trust research in military contexts also highlighted the importance of selecting team members with sufficient status to be ‘heard’ in a multi-agency team context (Curnin et al. 2015).

4.1 Cross training

Cross training enhances the awareness and knowledge that each team member has of their fellow team members’ tasks, duties and responsibilities and facilitates the holistic (shared mental model) understanding of team functioning and the respective, interdependent role of a given agency within the team (Marks et al. 2002; Schaafstal et al. 2001; Volpe et al. 1996). This is termed their interpositional knowledge (IPK). Volpe et al. (1996) reason that IPK allows team members to “anticipate the task needs of fellow team members”, leading to more effective team performance and enhanced coordination with a minimal communication requirement, important when task loads are high and individuals are too busy attending to these to be able to make explicit information requests. In the absence of IPK, there exists interpositional uncertainty which can hamper team performance (Blickensderfer et al. 1998).

Cross-training encompasses three methods that become progressively more detailed and involved, and thus more effective for improving shared mental models, understanding of complementary roles, and enhancing collaboration (Blickensderfer et al. 1998; Marks et al. 2002). These are: 1) positional clarification, a form of awareness training (e.g., by discussion, lecture, demonstration or dissemination of information) where specific information is provided about other roles and responsibilities in the team (e.g., working together in an EOC, or in a science response team, for example); 2) positional modelling, a training procedure in which the duties of each team member are discussed and observed via behaviour observation (e.g., offering potential EOC participants some actual practice in the other positions: a volcanologist could be given the opportunity to play the role of, for example an intelligence or logistics officer in an EOC context); and 3) positional rotation, which involves training within the exercise context where all team members spend significant periods of time performing other team members’ jobs and roles, providing a working knowledge of each member’s specific tasks and how those tasks interact and to gain different perspectives of the overall situation. The chosen cross training method amongst these three approaches should correspond to the realistic level of interdependence of the team (Ford and Schmidt 2000).

4.2 Scenario planning

Scenario planning enables the integration and awareness of the various social, political, economic, cultural, and other environmental forces that underpin the histories and expectations different agencies bring to the response management environment. It also provides an opportunity to ‘rehearse the future’, promoting adaption versus reaction, and providing a safe space through which various points of view and new or unique ideas from within the team can be shared without the fear of being prejudged or automatically dismissed (Bloom and Menefee 2014). Through this process, the volcanic crisis management team can also enhance their understanding of the wider response process, the key issue and decision thresholds, and trigger points; thus facilitating their collective ability to integrate their various perspectives and develop a bigger picture of the response than would arise if this was based on an individual role or stakeholder working separately. This can enhance shared situational awareness of the issues and process amongst the participating team. Scenario planning can be conducted at the sub-agency level (e.g. a monitoring team at a volcano observatory planning the response and deployment of personnel and instrumentation to facilitate effective monitoring during outcomes of an unrest scenario), at the agency level (e.g. a government science agency and university identifying how they could share resources and support each other during a response) and the multi-agency level (e.g. the VSAG considering potential eruption and response scenarios with the local CDEM group). In particular, for scientists, it provides an opportunity to work alongside emergency managers to identify science information needs and impacts of that information on decision outcomes, thresholds and trigger points.

5.1 Writing and evaluating exercises: practice guidelines

There are many guidelines existing in the emergency management and government literature that provide distinct steps to effectively write exercises (see Table 5). These all recommend the adoption of an exercise cycle, an example of which goes through the steps: 1) analysing the needs and outcomes desired for the exercise, which may be in line with process or performance improvement plans (this step may include a ‘foundation’ step assessing the status quo); 2) design and development of the exercise; 3) conducting the exercise; and 4) evaluating the exercise (see Figure 3).

Who	What & where
New Zealand Ministry of Civil Defence and Emergency Management	CDEM Exercises, Director’s Guidelines for Civil Defence Emergency Management Groups [DGL 010/09]
	http://www.civildefence.govt.nz/assets/Uploads/publications/dgl-10-09-cdem-exercises.pdf
US Department of Homeland Security	Homeland Security Exercise and Evaluation Program has a range of documents available outlining effective exercise writing at: https://www.llis.dhs.gov/HSEEP
US Department of Homeland Security, Federal Emergency Management Institute	FEMA’s National preparedness directorate on National Training and Education runs a series of online courses across the range of Emergency Management at http://www.training.fema.gov/, which includes the unit IS-139 on ‘Exercise Design’ which contains a range of useful resources. http://training.fema.gov/emiweb/is/is139lst.asp
	See also the Ready.gov resources at: http://www.ready.gov/business/testing/exercises
UK Cabinet Office & National Security and Intelligence	Emergency planning and Preparedness: exercises and training
	https://www.gov.uk/emergency-planning-and-preparedness-exercises-and-training
	including ‘The exercise planners guide’: https://www.gov.uk/government/publications/the-exercise-planners-guide
National Directorate for Fire and Emergency Management, Ireland	A framework for major emergency management, Guidance Document 4: ‘A guide to planning and staging exercises’, May 2011. http://www.mem.ie/guidancedocuments/a%20guide%20to%20planning%20and%20staging%20exercises.pdf
Australian Emergency Management Institute	Managing Exercises, Handbook 3: http://www.em.gov.au/Publications/Australianemergencymanualseries/Documents/ManagingExercisesHandbook.PDF

It is important that exercises do not just consider an event building up to a crescendo (e.g. the unrest period leading up to an eruption), but also consider non-events (the stalled eruption), response and recovery periods after an eruption (e.g. the period after a ‘blue sky’ eruption), or the issues of having a second volcanic crisis occurring alongside the focus volcano (e.g. in 2012 during the Te Maari unrest period in NZ, White Island also increased in activity requiring alert level changes). The volcanic events in NZ in 2012 provide a useful exercise scenario to examine the distribution of resources and personnel across two concurrent crises, and to identify more overt strategies for stress and fatigue management.

Exercises and simulations afford opportunities for agency representatives to develop, practice and review technical, management and team skills under realistic circumstances and to construct realistic performance expectations, and provide opportunities for developing creative problem solving competencies (within and between teams). They also provide opportunities to practice dealing with high pressure situations in a safe and supportive environment, increase awareness of stress reactions, and to rehearse strategies to minimize negative reactions (Flin 1996). However, many exercises fail to mirror the complexity of disaster response environments (Paton and Auld 2006).

There is a tendency sometimes to write these exercises like a ‘play’, providing too much structure and tailored or specific information. However, a truly challenging exercise, that provides the greatest opportunities for learning, should provide incomplete information, conflicting aspects, gaps in the situation description, ambiguous information, and unknowns; all creating an environment where participants are asking questions, not just finding solutions (Paton and Auld 2006). For example, a monitoring team exercise could provide a situation in-exercise where one or many data feeds abruptly end. This provides participants a situation to address whether a machine or data relay has broken, or whether a significant event has occurred, allowing them to exercise the situation of uncertainty and potential decision and action strategies. Exercises should be designed in ways that allow individuals and teams to progressively push their boundaries and develop progressively more sophisticated competencies and relationships (Paton and Jackson 2002). Only through exposure to exercises that challenge assumptions and allow personnel to confront novel events can preparation enhance response effectiveness for eruption events whose complexity will exceed even the most extreme simulation (Paton and Auld 2006).

Thus these exercises should aim to reproduce reality as closely as possible, so decision makers and experts can experience the needs and realities of the advisory process in turbulent conditions (Borodzicz and van Haperen 2002; Rosenthal and’t Hart 1989). However, this alleged realism can also be a danger as personnel may believe at the end of an exercise that they know what will happen in a real crisis (Borodzicz and van Haperen 2002). Evaluation of exercises and events must be carefully conducted to minimise the risk of creating an optimistic bias that overestimates future response preparedness and capability, particularly if a real event has not constituted a major test of the response system (Paton et al. 1998a). The development of an inter-agency exercise database would greatly assist this process.

Guidelines for evaluation and debriefings can be found in the emergency management literature (see Table 5 and Sinclair et al. 2012a; Sinclair et al. 2012b). However, these tend to focus on Key Performance Indicators (KPIs) and specific tasks, response plans, and communication process or outcomes and content evaluation. To ensure these exercise experiences truly enhance team response and development of a common understanding across the team, it is important to also consider team and group learning (Owen et al. 2013), as well as the assessment of individual learning processes such as the progression from unconscious incompetence (‘I don’t know what I don’t know’) through to unconscious competence (‘my skill has become second nature’) (Conger and Mullen 1981; Morell et al. 2002; Thomson et al. 2006). That is, an effective exercise is one that throws up several unanticipated issues (e.g., regarding information management, decision making, team functioning etc.) that can inform future training needs analyses and training and exercise design. Thus, exercising and scenario-based training should be regarded as an iterative process, where the debrief and evaluation is a vital part of that process to enhance the knowledge and skills of the participants, and to help them change their world view and individual paradigms, whilst also minimizing weaknesses caused by problems with exercise fidelity, such as a poor scenario or lack of reality (see review in Moats et al. 2008).

5.2 Exercise Ruaumoko: the importance of exercises and evaluation

Evidence for the benefits of exercises can be gleaned from Exercise Ruaumoko; benefits identified included opportunities to test the Auckland Volcanic Scientific Advisory Group (AVSAG, McDowell 2008). It was a valuable learning experience for all concerned, illustrating the criticality of Auckland to the New Zealand economy and the vulnerability of key infrastructure elements (MCDEM 2008, p. 7). Further, the process of preparing for the Exercise motivated the formalisation of the AVSAG structure (McDowell 2008) and helped to improve the understanding and performance of “individual elements” within a major CDEM operation (MCDEM 2008). However, one of the greatest benefits of the exercise was that it highlighted the need for considerable work “to co-ordinate the various elements” of the inter-agency response into a “cohesive [future] response” (MCDEM 2008). This exercise thus allowed responding agencies to identify response, network and communication issues that need to be modified for effective response and the importance of testing assumptions under realistic crisis conditions, such as high stress, short time, and low resource conditions with competing demands, to demonstrate faults or potential faults in the system. We thus now further discuss Exercise Ruaumoko in detail to provide an example of an exercise structure and to illustrate the benefits of effective evaluation.

Exercise Ruaumoko was a MCDEM led exercise, with over 1,500 participants from approximately 125 participating organisations (MCDEM 2008, p. 8). It was the second in a series of ongoing “All of Nation” exercises led by MCDEM (MCDEM website 2014c). Exercise Ruaumoko also provided a focus to develop and formalise the AVSAG structure (which was co-ordinated and arranged by Auckland CDEM), while also implementing Terms of Reference, and arranging formal contract agreements for the participating scientists (Cronin 2008); and to test such processes within a scenario. The three core objectives of the exercise were to: 1) understand, develop, and practice the respective roles and responsibilities of local, regional, and national agencies in response to the exercise scenario (McDowell 2008, p. 7; MCDEM 2008, p. 5); 2) embed the planning arrangements in standard processes for all participating agencies; and 3) confirm the connections between local, regional, national and international agencies. In addition, it enabled staff at the key levels to “practice and develop planning and the management of response activities using the connections and procedures that are in place”, and also practice the roles, responsibilities and procedures that are included in the National CDEM plan (MCDEM 2008, p. 55). The key sub-objectives of the exercise most relevant to science advice included the examination and testing of the coordination of science aspects and the management of public information and education. While Exercise Ruaumoko was primarily written and co-ordinated by MCDEM with Civil Defence objectives, it still provides a useful example for volcanologists of a large scale volcanology exercise, and the monitoring information and science response parts could be “re-exercised” or used as inspiration for a smaller scale science led exercise.

These observations clearly illustrate the benefits of exercises that truly test scientific response, VSAG structures, and inter-organisation communication and decision-making responses. The ‘split’ observed between the monitoring and wider volcanology subgroup of AVSAG is interesting to consider in the context of the influence of group identity (Hogg 1992) and the ‘stereotyping of in- and out- groups’ that can affect collaboration in a response (Paton et al. 1998a) and reduce the capacity to develop swift trust.

As briefly introduced in Section 2, in response to these exercise evaluations, Smith (2009) (on behalf of MCDEM), outlined a potential alternative model provided by a NZ Volcanic Advisory Panel (NZVAP), to avoid such a breakdown in communication, as well as to integrate local and national science research response (Figure 2b). This model was developed through a MCDEM facilitated dialogue between volcanologists to identify how the skills of scientists in the range of different organisations (including universities, Crown Research Institutes, consultancies and councils) can be integrated “in support of a national science agency such as GNS Science that has responsibilities under the National CDEM Plan for providing warnings and advice” (Smith 2009, p. 76). This national advisory model aims to mobilise science capability while remaining responsive to local CDEM needs (ibid, p. 77), and the intention of such a national mechanism is not to override any existing scientific or planning advisory groups at the local level, but rather that it would be complementary and “provide for a level of consistency for how New Zealand-wide science capability is mobilised when a large-scale science response is needed”. Other benefits of such a group include: a) facilitating coordinated post-event investigations and data sharing arrangements, b) providing strategic advice on research direction and priorities, c) fostering connectivity across the physical and social sciences, and d) supporting alignment between researchers and research users (see also Jolly and Smith 2012).

This proposed model was still under consideration and discussion when the Canterbury earthquake sequence started in September 2010. This sequence required an extensive response of many of the same scientists and agencies, and thus the development and implementation of such a NZVAP model has been put on hold until extensive reviews of the inter-agency response to those earthquakes were conducted (including both a Royal Commission of Inquiry and a Coronial Inquiry) and until the science advisory process during those earthquakes, and the more recent volcanic eruptions of Te Maari in 2012, have been fully evaluated by agencies and through various research projects in process (see also Section 2). However, until those real event reviews are available, Exercise Ruaumoko represents a clear case study of how exercises can be used to test a new VSAG structure, identify issues in that structure and pave the way for the development of a new structure. As stated by MCDEM (2008), p. 55 “there were major achievements in the preparatory phase [of Exercise Ruaumoko] that simply wouldn’t have occurred without the context of the exercise”. After formalisation of any new NZVAP structure, the next step should be test this through another exercise (of a different scenario) to complete the exercise cycle (Figure 3) and implement any lessons from that as part of a regular evaluation of science advisory mechanisms.