An analysis of the issuance of volcanic alert levels during volcanic crises

Past and current alert level schemes

The eruption of Kelut in 1919 led to the creation in Indonesia of a `Volcano Watching Service’, which was at the time part of the Department of Mines (Neumann van Padang [1983]; Suryo and Clarke [1985]). The aim of the new service was to study how populations could be protected from volcanic eruptions and specifically considered volcano type, eruption history and volcanic unrest in order to warn and evacuate the population when necessary (Neumann van Padang [1983]). Observation `Posts’ were established at Merapi from 1934 to warn of imminent volcanic threats (Neumann van Padang [1983]). During the 1980’s there was a growing interest in formalizing eruption warnings in the framework of VALs. Although never formally published, one of the first of these was developed by the Rabaul Volcano Observatory (RVO) in response to the unrest in the caldera during the early 1980’s (Rabaul Volcano Observatory, Volcano Information Bulletin no.2, (1983); UNDRO United Nations, Office of the Disaster Relief Coordinator [1985]). When activity began at Mt St. Helens in 1980 it became apparent that the scheme in place (patterned after one used by the National Weather Service) did not capture rapid changes within the overall eruptive sequence. As an alternative, the USGS issued `factual statements’ -descriptions of current conditions and `extended outlook advisories’ that looked several weeks ahead (Newhall [1984]; Swanson et al. [1985]). There was also a concerted effort to communicate risk to the people who went back to work in the areas decimated by the lateral blast in May 1980 (Newhall [1984]). These events raised the need for comprehensive VALs.

Further VAL schemes were developed globally through the 1990’s, and some have been simplified and standardized since then (Japan Meteorological Agency [2009]; Fearnley et al. [2012]).

For all the volcanoes that we examined, the final level included the statement that `an eruption was imminent’ or that an `eruption was underway’.

Data sources and treatment

Volcanological surveys issue VALs for local and national purposes. We retrieved these VALs from online archives of volcanological and geological surveys, the `Smithsonian/USGS Weekly Volcanic Activity Report’ and the Smithsonian Institution’s monthly `Bulletin of the Global Volcanism Network’, and internal hard-copy archives of some of the surveys in the study. We also drew supporting information from EM_DAT (Emergency Events Database) (CRED Network Website [2008]) and DesInventar (Disaster Information Management System) (UNISDR Website [2011]).

To determine which volcano observatories issue alert levels, we surveyed all of the volcanoes in the GVP database, recording the ones that had any weekly or monthly bulletins. We used reports of both volcano observatories and summarized monthly reports in the Smithsonian archives. We identified volcanoes that had activity after a given observatory began using VALs. Where we were unable to determine when a specific observatory began using alert levels, we took the date of the first record of any VAL at that volcano. This survey encompassed 133 volcanoes including 291 eruptions from 17 countries. However, many data sets were incomplete, so to ensure that the records were consistent with each other we only use cases where: (1) VALs had been established at the volcano before the eruption began, and (2) the records of VAL were continuous. We also excluded volcanoes that had no eruptions in the time frame of modern alert levels (e.g. since 1991) and volcanoes that were in continuous eruption for more than 10 years. After filtering by these criteria, our database includes 60 volcanoes and 194 eruptions from 11 countries: Chile, Colombia, Ecuador, Indonesia, Japan, New Zealand, Peru, Philippines, Russia, United States and Vanuatu. The eruptions occurred between 1 January 1990 and 31 December 2013. We note here that some countries with large numbers of volcanoes, such as Italy and Iceland, are not part of our study because either they don’t publicly issue VALs or else we were not able to retrieve their data. Our complete dataset can be found in the electronic supplement (Additional file 1).

Data classification

We made a timeline of the changes in VALs at each volcano and compared it with the start dates of eruptive episodes from the GVP. We plotted VAL data from 30 days prior to the onset of an eruption through to the end of the eruption. We assumed that any unrest within 30 days prior to eruption is related to the causes of the eruption. Using a longer time window (e.g. 60 days) does not change our results.

We tracked how the VALs changed with time relative to the onset of eruption. Our main tenet is that a VAL is appropriate if it progressively increases before eruption, or if it increases then decreases when an intrusion of magma does not erupt. We identified six possible VAL patterns (Figure 1): Category 1 has been termed `Missed’, where the onset of the eruption was not preceded by an increase in the VAL, and the VAL was not moved (and usually remained low) during the course of the eruption. In Category 2 (`Too late’) the VAL was increased for the first time after the eruption began (days to weeks) but before the eruption ended. Category 3 is for `Premature’ alerts, where the VAL was increased but subsequently decreased to lower levels prior to the onset of eruption. Category 4 is called `Almost’ because the VAL was increased prior to the eruption, but not to the highest level that indicates `increased potential of eruption’ or `high alert’. Often, the VAL was changed from the `Normal’ or `Background’ level to a higher level of unrest, but no further. Category 5 is called `Timely’ because the VAL was increased in a stepwise fashion and, at the point of eruption, was at an appropriately high level. Category 6 `UwE’ (Unrest without Eruption) refers to cases in which the VAL was increased, remained elevated and then was lowered without an eruption occurring within the next 30 days. We consider Categories 4 (`Almost’) and 5 (`Timely’) to be the most appropriate issuance of alert levels in cases where unrest led to an eruption, and Category 6 `UwE’ as appropriate when unrest did not lead to eruption. We have analysed reports from these episodes of unrest and found that the `UwE’ alerts recorded in this study are well correlated with the state of the volcano, and were thus appropriate. However, we have no way of counting the episodes of unrest that did not lead to an eruption that were not represented by an increase in the VAL. We therefore, cannot estimate the percentage of `UwE’ that were “missed” by the VAL systems.

We also note that the eruption start dates in the GVP often refer to the onset of phreatic precursors to magmatic eruptions. VALs, on the other hand, are more likely to track magmatic ascent. To test whether reference to phreatic or magmatic onsets would change our characterization of VALs we did an analysis for larger eruptions, all clearly magmatic (VEI ≥ 3). Using the eruption reports we recorded the dates where magmatic activity can be assumed to have begun. This is defined as either the day that 1) Dome growth was recorded; 2) Pyroclastic flows were observed; 3) Eruption column reached 10 km or greater, or 4) the date of the maximum eruption column height. We then assessed whether there was any change in how the event would have been ranked. We found this effect to be negligible (only 1 case out of 40 would have been classified differently).

Data sources for level of monitoring

We gathered information on the monitoring network at specific volcanoes from the records that are kept by World Organisation of Volcano Observatories (WOVO) and by each observatory. To assess the level of monitoring we used a slightly modified scheme from that developed by Ewert et al. ([2005]), in the development of the National Volcano Early Warning System (NVEWS), where a numerical value was assigned to different degrees of volcano monitoring capacity. We focused on the seismic, deformation, gas and remote sensing capabilities of the network when we assessed the level of monitoring at the volcanoes in our study.

Level 0 denotes `No In-Situ’ monitoring; eruption confirmation is usually by remote sensing. We have eight volcanoes with this level of monitoring. Volcanoes with Level 1 or `Limited’ have no real time eruption data from ground-based sensors, but the volcano is within a regional seismic network. Eruption confirmation is still by remote sensing and there is a baseline inventory of satellite images available: five volcanoes from our sample have Level 1 monitoring. Level 2 network is considered `Basic’, and includes: 1 – 2 seismometers within 10 km of the volcano and any combination of: 3 continuous GPS or tiltmeters in the vicinity, repeated campaigns of gas measurements, and near-real-time images. There are 22 volcanoes that have this level of monitoring. Level 3 volcanoes have `Good’ monitoring, with 3-4 seismometers in the near field and any combination of: 6 seismometers within 20 km of the vent, least 6 continuous GPS in the vicinity and continuous or periodic gas monitoring. There are 21 volcanoes in our sample at this level. Level 4 or `Research Level Monitoring’ volcanoes include 12 - 20 stations within 20 km of the vent, sufficient continuous deformation stations to do detailed source inverse modeling and continuous gas monitoring. There are only four volcanoes in our study with this level of monitoring.

When we were in doubt about the level of instrumentation at a volcano we have awarded the higher ranking among the possible levels. Therefore our assessment can be seen as a measure of the maximum monitoring capability based on the information available. Because of the difficulty in accessing data about the changes in monitoring capability over time, we used the most up to date information. We have not been able to make an assessment of the staffing levels and capabilities at observatories, although we assume that there is loosely an increase in the number of staff and the level of training with an increasingly dense network.

Volcano and eruption styles

We divided the eruptions according to the VEI (as assigned by the GVP) and found that in general there is an increase in the number of Category 5 (`Timely’) VALs as the eruption size increases (Figure 3). For VEIs ≤ 2 there is a high percentage of `Missed’ and `Too Late’ VALs (42% and 39% , respectively) and of the 154 eruptions, we see only 3% of the VALs in the `Timely’ category for these small eruptions. As the eruption size increases to VEI 3 the VAL’s in the `Missed’ category decrease (26% ), but those in the `Too Late’ increase (41% ). There is also an increase in the percentage of `Timely’ (12% ) and `Almost’ for eruptions of VEI’s < 2 to VEI 3. For VEI ≥4 eruptions the percentages in the `Missed’, `Too Late’ and `Timely’ categories are about the same (33% ), but the sample size (n = 6) is too small for statistical significance.

We also looked at the character of the volcano to determine if this would have any effect on the issuance of VALs. For this, we simplified our classification into: (1) Open-vent volcanoes that produce many frequent, small eruptions. They are often continuously degassing and tend to have more effusive eruption products. This continuous release of gas seems to make large eruptions less likely, unless the system seals itself (Newhall [2007]), (2) Closed-vent volcanoes have less frequent eruptions and are commonly dormant for decades, centuries, or longer. About 53% of the volcanoes in our database were classified as open-vent, and produced 76% of the eruptions, whereas the remaining 47% are closed-vent, and produced 24% of all eruptions. We found a higher percentage of `Timely’ and `Almost’ VALs (10% and 19% , respectively) in closed-vent (Figure 4) than in open-vent systems (3% and 12%, respectively). Open-vent systems also show a higher number of `Missed’ events (43% ) than closed-vent systems (22%). Finally the proportion of `UwE’ at open-vent volcanoes is 28% of the total number of alerts, but this proportion increases to 42% at closed-vent volcanoes.

Role of monitoring networks and equipment

We found that there is an increase of the `Timely’ and `Almost’ categories with the increase in the capacity of the monitoring network (Figure 5). Volcanoes with monitoring Levels 0 and 1 have a high number of `Missed’ and `Too Late’ VALs (88% and 80% , respectively for 13 total events). For volcanoes with Levels 2 and 3, we find that there is an increase in the `Timely’ and `Almost’ VALs from 14 (Level 2) to 43% (Level 3). The value of the `Almost’ category for monitoring Level 4 still increases to about 50% , although given that we only have 4 volcanoes in this category, it is not clear if the increase is statistically significant. At volcanoes with low monitoring levels, the `UwE’ alerts are a very high percentage of total alerts issued (72% for Level 0 monitoring) and then broadly decrease as the monitoring level increases (33% for Level 4 monitoring).

Effect of eruption frequency on the issuance of VALs

To test whether an increased knowledge of the system (e.g. a given volcano) increases the reliability of the issuance of VALs, we looked for an increase in the number of `Timely’ and `Almost’ VAL categories with an increase in the number of eruptions at a given volcano. In our sample, 34 volcanoes had more than one eruption over the time period and this filter still includes 161 eruptions. We found a large spread in data and no relationship between the number of eruptions per volcano and the percentage of `Timely’ and `Almost’ VAL categories (Figure 6). However, some pattern emerges if we consider frequency of eruption and level of monitoring. Volcanoes with frequent eruptions (mostly, open vent volcanoes) with the highest percentages of appropriate alert levels are those with monitoring levels 2 and 3 (Figure 7). The lowest percentages of appropriate VALs are for volcanoes that have fewer eruptions (mostly, closed-vent volcanoes), and a moderately high level of monitoring (level 2 or higher).

Effect of population on VALs

The purpose of VALs is primarily to inform local populations (and, in some cases, air traffic) of the potential for volcanic hazard. Local authorities also use them to inform decision making for evacuations. We checked to see if the size of population living around a given volcano affects the issuance of VALs. We used the population figures from the Smithsonian GVP, which were obtained from the LandScan 2007 data (Oak Ridge National Laboratory [2008]). We considered the population size within 30 km of the volcano, and normalized the data by the number of volcanoes within each population bin size (Figure 8A), to account for variable number of volcanoes per bin. We found that there are more `Timely’ alerts in the bin with the highest population, but in general there are similar proportions (within a few percentage points) for the `Timely’ and `Almost’ categories for all population sizes (Figure 8A), meaning that there is no significant correlation between appropriate VALs and population size. The proportion of `UwE’ varies between about 24 and 48% for all population sizes, without a clear change with population. As an additional measure to test the influence of the population on the decision to change VALs, we compared data from two countries with very different demographics. Country A has a large population living within 30 km of volcanoes, whereas Country B has much lower numbers of people (on average less than a 100) living close to its volcanic centers.

For Country A we found that for volcanoes with populations of less than 1 million people about 21% of the VALs fall into the `Timely’ or `Almost’ (Figure 8B), but this increases to 45% for volcanoes with more than 1 million people, so there is an improvement with population size. The highest proportion of `Missed’ events occurs within the population range of 1,000 – 10,000. The `UwE’ show a somewhat bi-modal distribution for Country A with peaks for the lowest (1,000 – 10,000) and the highest (>1,000,000) populations. In contrast, for Country B with sparsely populated volcanoes (less than 2,500 people living within a 30 km radius of the volcanoes) we see that even in the lowest populations (less than 1000) there are higher proportions `Timely’ and `Almost’ VALs (Figure 8C) compared to the highly populated Country A. But there is also an increase in appropriate VAL’s with population size. There is no dominant trend in the `UwE’ alerts that can be attributed to differences in population. The general decrease in `UwE’ alerts for Country B is more likely a factor of the small sample size, rather than a true trend.

Has the issuance of VALs improved over time?

Since 1990 there has been an increase in the number of countries and institutes that have adopted a form of VAL, from one in 1990 to 60 in 2013 (Figure 9). To be able to compare the results from different years we have normalized the data by the number of volcanoes. We find that there is quite a large variety from year to year and not a linear increase of any of the categories with time (Figure 10). Many `Missed’ and `Too Late’ VALs occurred prior to 2000 while from 2004 onwards most years have higher `Timely’ and `Almost’ categories (in general > 20% ). To better test whether the VALs are improving with time, we performed a moving average analysis of the `Timely’ and `Almost’ categories (Figure 10) which shows a complex pattern but a general improvement after 2003.

We also compared the number of alerts and number of eruptions over time normalized by the number of volcanoes and some trends can be identified (Figure 11). We find that from 1996 to 2004 the normalized number of eruptions is higher than the number of alerts, which reflects that many eruptions occurred without any alert being issued. However, starting from 2005 the ratio of numbers of alerts to eruptions is similar or higher than one, so that on average at least one or more alert was issued per eruption, something that is also reflected in the decrease in `Missed’ events over this time period (Figure 10). Thus, in general we find that with time there is an increasing number of VALs issued per volcano and per eruption that could reflect an increase in the monitoring network or change in the scientific decision making process as discussed below.

How well do VALs reflect the state of a volcano?

Physical and technical parameters that affect issuance of VALs

One of the clearest trends that we have found is a general increase of `Timely’ and `Almost’ categories with VEI (Figure 3) which could be related to the difference in volcanic precursors and unrest signals for small versus large eruptions. VEI 1’s include small ash puffs and phreatic explosions that may arise from hot gas and rock interacting with ground water (such as the eruption of Mt Mayon in the Philippines in May 2013), which have few or no discernible precursors and thus no basis for raising alert levels. The larger and longer episodes of unrest that are typically associated with VEI 3 or 4 eruptions allow for more progressive increase of VALs. Larger eruptions are better anticipated than smaller ones. One example is Merapi, which had much larger unrest signals and a much larger and explosive eruption in 2010 compared to those in 2006 and other past eruptions (Surono et al. [2012]).

The more appropriate issuance of VALs at closed vents (Figure 4) is probably because magma needs to fracture rock or clear the conduit before it can reach the Earth’s surface. This may also explain why `UwE’ are more prevalent at closed systems than at open systems because the longer repose periods would lead to a greater need for re-establishing pathways within the conduit and may cause multiple episodes of unrest over an extended period prior to eruptions. The signals at these volcanoes may therefore be stronger and a higher number of `UwE’ can be expected due to several intrusions or episodes of magma ascent before an eruption is triggered.

We find that VALs at open vent volcanoes (Figure 7B) are improved by a higher number of eruptions and opportunities for the observers to be `trained’ by the events, and/or by a better level of monitoring that allows for subtle precursors to be recorded. As closed vent volcanoes have fewer eruptions, there is less opportunity for observers to experience volcanic crisis. Instead they need to rely more heavily on monitoring networks and well-constrained eruptive histories (both from detailed stratigraphic analysis and the careful review of historical documents). The lack of experience in closed-vent volcanoes or in volcanoes that reawaken after a long period of dormancy (e.g. Sinabung volcano in 2013) could be mitigated by the use of a worldwide database of volcano monitoring signals (Widiwijayanti et al. [2013]).

Another factor that significantly improves VALs is an increased level of monitoring (Figure 5). This finding could be anticipated as more data available to the volcanologist makes for better understanding of the system. However, the increase in the accuracy from Levels 0 to 1, or Level 1 to 2 is much less than from Level 2 to 3. This suggests that it is necessary to increase the monitoring network to at least 3-4 seismometers, and/or GPS and gas monitoring for an effective improvement of the appropriate issuance of the VALs. Such an increase is usually coupled with an increase in human resources, as more effort is invested in interpretation of the monitoring data. The results for monitoring Level 4 volcanoes are inconclusive because the sample size is only four volcanoes. It is also possible that an observatory with a monitoring level as dense as a Level 4 network would have a large volume of data from differing sources to reconcile before a decision could be made about the potential for eruption at a given time. The increase of the proportion of appropriate VALs and of `UwE’ with time (Figure 10) could be a reflection of an increase of the level of monitoring or understanding of the volcanic systems. Better monitoring enables detection of more subtle unrest signals that may not lead to immediate eruption but which could be related to magma intrusion at depth (e.g. Moran et al. [2011]). Unfortunately it is not possible for us to quantify this relationship because we could not determine how the monitoring level at each volcano changed with time. Another explanation is that the increase in number of `UwE’ reflects a change in the decision-making process whereby there is less reluctance to issue an alert even if the certainty of the eruption is not higher than in the past (see below).

The influence of the decision-maker(s) on the issuance of VALs

So far we have analyzed the role of the physical parameters (volcano and eruption types) and technical factors (e.g. monitoring level) that may lead to more appropriate issuance of VALs. However, a change in VAL is ultimately a scientific decision-making exercise, usually involving a single individual or a group. It is difficult to identify a quantifiable measure to characterize the effect of the decision maker on the reliability of VALs, but it is probably very large and may explain why the parameters we have investigated do not fully explain the different percentages of the categories. Decisions in high-risk situations such a volcanic crisis may be affected by the individual risk aversion of the decision-maker and by personal interest (e.g., losing reputation) (Woo [2014]). To compensate for the potential bias of the scientist-decision-maker, several workers have advocated for a more robust methodology based on probabilistic hazard assessment and cost-benefit analysis (Marzocchi and Woo [2007], [2009]; Marzocchi et al. [2012]; Woo [2014]).

Two factors from our data suggest a significant effect of the decision-making process on the appropriate issuance of VALs. One factor is the increase of `Timely’ and `Almost’ categories and another is the proportion of Alert Levels and eruptions with time (Figures 10 and 11). As noted above, these factors could reflect results from an improvement of the monitoring networks with time, but also include a realization that to improve VAL issuances it is necessary to be less conservative. Our data could indicate that decision are starting to be made from an optimal risk management perspective, rather than waiting for eruption probabilities to become very high (e.g., Woo [2014]). The other indication of influence from the decision making process is the relationship between the VALs, `UwE’, and the population size. The global population dataset shows only an increase of the `Timely’ category with increasing population density, whilst the `Almost’ VALs show no trend. However, we found that two different countries with different demographics both show an increase of appropriate VALs with increasing population suggesting that the decision makers are more risk averse when changing VALs depending on the population at risk. We find that there is a general increase of `Almost’ and `Timely’ VALs with increasing population but that this increase seems to occur at different country specific thresholds. Country B with its low population density shows an increase in `Almost’ and `Timely’ VALs at low population numbers (>100) (Figure 8B). In Country A, which has a higher population density, we see an increase at populations >10,000. This increase with population size suggests that although VAL schemes maybe designed simply to communicate the hazard, they are being implemented with risk to local populations in mind. In a specific example (Indonesia), CVGHM did not raise the VAL to the highest level at Sinabung in 2013 even when phreatomagmatic ash columns were reaching 7 – 10 km as there was little risk to the population on the ground. Instead, they waited until there was a high likelihood of a summit lava dome collapse, so that evacuations would be more commensurate with the risk (John Pallister, personal communication, July 2014).

The trends of `UwE’ alarms in Countries A and B may hint at the complexity of this decision-making process. In Country B (Figure 8C) the decrease in `UwE’ as populations increase may suggest that there is an effort to not issue alerts until there is a high level of certainty about impending eruptions. Country A, though, shows a more complex response (Figure 8B). It could be argued that the increase in `UwE’ alarms as populations increase (after 10,000 people) reflects intolerance to leaving populations at risk. It is interesting to note that the higher levels of `UwE’ seem to be coupled with higher levels of `Timely’ VALs.