Interviews
Map data classification
Clarity/ease of reading, precision/uncertainty, and aesthetics emerged as the three primary themes in participants’ discussion of map data classification. The ease with which participants were personally able to read data from the map was a motivator for map preference, but participants also expressed consideration of how easy it would be to use the map to communicate with peers and the public:
It’s more than just communicating it [the map] to fellow scientists, or fellow emergency managers, or engineers, or whatever. It’s also how do you communicate that further down to, to members of the public. – Stakeholder 5
The isarithmic maps (Figure 1B,C) were well-liked and were viewed as accurately portraying the realistic nature of the volcanic ash hazard, while also being easy to read with a high degree of precision:
The advantage is you can still quite easily see the gradational nature of the probability, but at the same time you can read off the actual percentage if you feel you need to read that off. – Scientist 4
No relationship was observed between dashed (Figure 1C) and solid isopleths (Figure 1B) and interpretation of hazard information. However, the black isopleths and labels across the map were viewed as busy and distracting, detracting from the aesthetic appeal of the map. Participants overall recommended smoothing of the isopleths, citing that straight lines and right angles resulting from the square grid used in the model were visually unappealing and could falsely imply certainty. Participants also viewed the 5% intervals on the isopleths (Figure 1B,C) as misleadingly precise, and instead preferred the 10% intervals used in the binned map (Figure 1D).
The binned map was also well-liked among participants, who viewed it as easy to read and visually appealing. Stakeholder participants also expressed that the binned map allowed for interpreting the map with more certainty, viewing binned zones as “more definitive”. Stakeholders also expressed the bins as a useful tool for visualising zones:
Although there’s defined areas between them [bins], it’s kind of useful in terms of a planning aspect that you’ve got some boundaries to work on. – Stakeholder 6
However, both scientists and stakeholders expressed that the boundaries between classes on the binned map (Figure 1D) could present a disadvantage if they were interpreted as non-gradational or “step-wise”:
Except, when you put a boundary on it, then people probably think if they’re on one side of the boundary or the other there’s a huge difference in probability when there isn’t. – Scientist 3
The background content was generally well liked, and viewed as helpful for orientation and location tasks, without cluttering the map. In contrast, geopolitical boundary limits on the data were not liked. Participants suggested that the boundary made the hazard appear to “stop” artificially. Participants also emphasized that managing and responding to hazards is a collaborative process that crosses boundaries, and that uncertainties or unknowns for the hazard in the surrounding regions introduced potential issues for interpretation and application.
Map colour scheme
Colour associations, zoning, and response emerged as the three primary themes in participants’ dialogue concerning map colour scheme. Overall, participants associated red hues with a presence of hazard and blue hues with an absence of hazard. This became particularly important in low probability areas, which would be considered as “safe” if seen in the blue-yellow-red diverging map (Figure 1E), but would still be considered as having a potential to be impacted on the red-yellow sequential map (Figure 1A-D):
My first impression of that [diverging map] is, blue would be safe. Whereas, where this [blue area] is yellow in the other maps, it implies there is some sort of risk that we need to consider. – Stakeholder 2
These associations were described by participants as “subliminal”, “genetic”, and “psychological”. The contrast between hues in the diverging map (Figure 1E) was seen as making the hazard appear smaller, as a localised zone, and was discussed as facilitating distinction between impacted zones in the context of targeting response attention:
It makes it very clear, ‘that’s the area we’re worried about. We’re not worried about anything else around the district’. – Stakeholder 7
Stakeholders also observed that the diverging colour scheme appeared very similar to other natural hazard maps that they had experience with:
Automatically [I] relate that to weather maps… and just to complicate matters worse, we’re also doing flooding maps, and the models that we use are based very much on the same colour format. So I would look at that straight away and think, ‘Oh, it’s a flood map.’…because you see the blue. – Stakeholder 8
This was the only significant difference observed between scientists and stakeholders in the interviews. The potential difficulty in reading maps due to colour blindness was raised as an issue by several participants.
Map key expression
Trust and familiarity emerged as two themes in discussion of key expression. Overall, participants viewed the verbal expressions “probability” and “likelihood” as very similar. However, “probability” was considered by stakeholders to sound more reliable, scientific, and trustworthy, and in that context, it was also interpreted as a more definitive way of expressing the hazard:
Depends on how accurate you want to be. Probability means yes, definitely it is 10% probable. Whereas 10% likelihood [means] well it might, it might not. - Stakeholder 9
“Chance” was perceived as untrustworthy, and akin to “slang”, invoking associations with gambling and horse-racing. Percentages (e.g., 10%) were overall the most preferred numerical expression, with participants explaining that they are a commonly used format in many different walks of life. However, some participants noted that natural frequencies (e.g., 1 in 10) were easier for people to get a “feeling” or gist for the value. Decimal values (e.g., 0.1) were universally considered as unfamiliar and too difficult for most people to understand. Many participants also commented that the gradational shaded key symbology was difficult to read with only three percent values marked (min., med., and max.).
Map content
Participants viewed the probabilistic maps (Figure 1A-E) as long-term planning and reference tools, and as preparedness tools to use in the event of an impending crisis before more detailed deterministic-style maps (e.g., Figure 1F, or an ash advisory bulletin) were available. Participants also expressed that 10 mm was a high threshold in terms of useful application of the map. Most participants expressed that their primary concern would be the possibility of acquiring any ash at all, explaining that a hazard threshold even as low as 1 to 2 mm thickness would have very important impacts that they would need to consider:
Any ash to me is the worst case scenario. - Stakeholder 3
Similarly, some participants expressed that probabilistic data for low ash thresholds were helpful supportive content:
I think the probability is more important than the thickness because [of] the way people work. It’s, ‘Will I get ash?’ Not, ‘How much will I get?’ …They’re not sort of thinking ‘We can do this with 10 mill[imetre]s; we can’t do this with 100.’ It’s kind of ‘Oh, ok, we have to deal with volcanic ash.’ Saying that there’s a 30 or 40 percent chance…drives home a pretty strong message that they’ve got to deal with it. – Scientist 2
Many participants also suggested that addition of text explaining the possible impacts of the ash thickness would make the map more relevant and useful. It was advised that the text be placed directly onto the map face, explaining that it could easily be misplaced, truncated, or disregarded if supplied separately.
Revisions to maps for the survey
The results of the pilot study interviews which were based on the six maps in Figure 1 informed some revisions to the maps in order to better meet user preferences and needs, resulting in the eight map styles explored in the survey (Figures 2, 3, 5):
-
Geopolitical boundary constraint on hazard data removed (Figure 2)
-
Colour blind accessible colours were adopted (Figure 3)
-
10% bins and isopleth intervals were adopted (instead of 5%)
-
More intervals were labelled on the gradational stretch symbology in the key
-
White contour lines were used instead of black (to reduce map noise)
-
Isopleths were smoothed to remove the jagged artefact of the modelling grid
-
≥1 mm of ash was displayed instead of ≥10 mm of ash.
Survey
Out of the 110 respondents, seven participants did not fully complete the survey. Results for questions with missing values were measured using only the number of completed responses (min. 103). Results are reported for the 5% significance level (95% confidence), where χ
2 is the chi-square value for a goodness-of-fit chi-square test (one sample), Pearson χ
2 is the chi-square value for a Pearson chi-square test of independence (two or more samples), df is the degrees of freedom, and α is the level of significance.
In response to a question about whether they had ever experienced volcanic activity or had any memories associated with volcanic activity (Table 1, Question 4), 68.8% of participants responded “yes”. Of those, 64.0% described specific volcanic activity in New Zealand. When asked to rank their opinion about how important it is to have a volcanic hazard map for the potentially active volcanoes in New Zealand on a 5-point Likert scale (Table 1, Question 5), 99.1% of respondents expressed that it was important to some degree, with 73.4% ranking it “very important”.
Map data classification
Results for questions about data classification (Table 1, Question 6–13; Figure 2) are shown in Figure 7A-D. When reading probabilistic hazard for the 78 km2 urban area of Whakatane, respondents were more likely to give a range of values (e.g., 40 - 45%) as opposed to a single value (e.g., 40%) when using the binned data classification map (Figure 2B) (χ
2 = 114.919, df = 1, α < 0.001) or the isarithmic map (Figure 2C) (χ
2 = 117.237, df = 1, α < 0.001) (Figure 7A). In contrast, respondents were equally likely to give either a range of values or a single value when using the gradational shaded map (Figure 2A) (χ
2 = 0.092, df = 1, α = 0.762) (Figure 7A). Of those who responded with a range, more participants (17.9%) reported a 5% interval range when using the isarithmic map (Figure 2C) than for any other data classification map style. Ten percent ranges were the most commonly estimated range interval for all data classification types. The actual hazard values for Whakatane ranged between approximately 40 – 45%. When reading hazard from the isarithmic map (Figure 2C), respondents were more likely to estimate a single or range of values within 40 – 45% (very accurate) or 35 – 45% (generally accurate) than when using the other two maps (Figure 7B). When using the gradational shaded map (Figure 2A), respondents were more likely to under- or over-estimate the hazard value (Figure 7B).
More than half of respondents interpreted the qualitative level of hazard as “medium” regardless of map data classification style used (Figure 7C). The level of qualitative hazard interpreted from each map did not change for 71.8% of respondents who chose only one hazard level for all three maps. Scientists were more likely to show a change in the level of qualitative hazard interpreted from the different maps than stakeholders (Pearson χ
2 = 7.987, df = 2, α = 0.018), with 40.0% of scientists choosing two hazard levels among the three maps, and 3.3% choosing three, and 25.0%, and 0.0% of stakeholders choosing two and three levels of hazard, respectively.
The isarithmic data classification map (Figure 2C) was ranked as the most preferred map choice, and the gradational shaded map (Figure 2A) was ranked the least preferred (Figure 7D). There was no difference in the preferences of scientists and stakeholders, with 37.5% of both scientists and stakeholders preferring the binned data classification map (Figure 2B), and 62.5% preferring the isarithmic (Figure 2C) (Pearson χ
2 = 0.000, df = 1, α = 1.000). Respondents wrote an average of 30 words in response to an open-ended text question about map data classification style preference. Three main themes emerged concerning the influence of map data classification style on respondent’s interpretation of the hazard map: clarity/ease of reading, precision/uncertainty, and realistic hazard representation. Many respondents stated that the presence of gradational shading (Figure 2A,C) was favourable because it represented the gradational nature of ashfall hazard more realistically:
Clearest because it shows that the hazard is gradational, steadily decreasing away from the source. – Scientist 30
Similarly, the binned map (Figure 2B) was seen by many users to be an unrealistic portrayal of ash hazard:
The [actual] change is gradational, so [the binned map] is too stepwise. Gives the wrong impression. – Stakeholder 54
However, using gradational shading only (i.e., without isopleths) (Figure 2A) was overall seen as requiring too much effort to read with any degree of precision. When labelled probability isopleths were included on the gradational shaded map, many participants found the resulting isarithmic map (Figure 2C) much easier to read with a higher degree of precision:
[It is] instantly obvious which range of values an area falls within, plus you can see where in the range it falls, so you can get quite a precise value just by quickly looking at the [isarithmic] map. – Stakeholder 87
Less reliance on the key symbology was also seen as increasing precision:
The labels indicating the probability band removes any confusion associated with the colour symbology. – Scientist 104
Map content
Results for questions about map content (Table 1, Question 14–15; Figure 3) are shown in Figure 8A-B. Both types of map content investigated (Figure 3) were viewed as helpful to some degree by the majority of respondents, with 95.2% of respondents ranking the fixed ash threshold map (Figure 3A), and 89.4% ranking the fixed probability threshold map (Figure 3B), as very helpful, helpful, or somewhat helpful (Figure 8). Among the 97.2% of respondents who chose a preference, no statistically significant difference exists between preference for receiving a map with content describing a fixed ash threshold, a fixed probability threshold, or one of each (χ
2 = 1.529, df = 2, α = 0.465). There was no statistically significant difference in the preferences of scientists and stakeholders (Pearson χ
2 = 2.111, df = 2, α = 0.348).
Map key expression
Results for questions about key expression (Table 1, Question 16–17) are shown in Figure 9. When presented with three different combinations of verbal and numerical phrases of probabilistic hazard in the map key, 73.4% of respondents said the phrases used did not change the level of hazard they interpreted from the map. However 14.6% of respondents thought that the natural frequency expression (1 in 4) made the hazard seem greater than the percentage (25%). Despite the majority of respondents saying that the key had no influence on hazard perception, 93.2% of respondents chose a preferred key expression (Figure 9). More than half of respondents preferred having both a percent “probability” and natural frequency “likelihood” expressed in the key. The second most preferred key expression was a percent probability. Of the 22.3% of respondents who entered text as optional commentary, many described the key with a natural frequency “likelihood” expression only as “amateurish”, “subjective”, and “awkward”, with one respondent citing that “likelihood” has a different and specific meaning in their field of work. Percent probability was generally described as a common, familiar expression that was readily understood by both professionals and the public. Some respondents suggested that providing the natural frequency likelihood alongside a percent probability may help the message be received by a larger audience. No significant difference existed in the key expression preference of scientists and stakeholders (Pearson χ
2 = 0.892, df = 3, α = 0.892).
Hazard curves
Results for questions about hazard curves (Table 1, Question 18–22; Figure 4) are shown in Figure 10A-B. When ranking how helpful it would be to be provided with hazard curves for chosen locations on the map, 55.4% of respondents said the curves were very helpful, helpful, or somewhat helpful. However, scientists found the hazard curves significantly more helpful than stakeholders, with 73.3% of scientists ranking the curves as helpful to some degree, while only 48.1% of stakeholders did (Pearson χ
2 = 6.324, df = 2, α = 0.042). Nine percent of stakeholders were not sure about the helpfulness. Performance in reading a hazard curve for Whakatane improved significantly when respondents used the hazard curve with the 80% confidence area (Figure 4B), with 70.3% of respondents choosing the most correct answer, compared to 12.6% when using the hazard curve with 10th and 90th percentile lines (Figures 4A, 10A). More than 46% of respondents chose a response option in which the 10th percentile was incorrectly described as 10% confidence when using the hazard curve with 10th and 90th percentile lines (Figure 4A), compared to just 7.9% when using the hazard curve with 80% confidence area (Figure 4B) (response option 2, Figure 10A). In describing the ease/difficulty of reading the two hazard curves, 71.9% of respondents found the curve with 10th and 90th percentiles difficult or very difficult to read, and 26.2% found it average, easy, or very easy, compared to 41.6% and 57.4% of respondents for the 80% confidence area curve, respectively (Figure 10B).
Map colour scheme
Results for questions about colour scheme (Table 1, Question 23–25; Figure 5) are shown in Figure 11A-B and Figure 12A-C. For more than 67% of respondents, colour scheme had an effect on how they perceived the level of hazard (χ
2 = 14.957, df = 2, α = 0.001) (Figure 11A). The blue-yellow-red diverging colour scheme (“diverging”) (Figure 5C) and red-yellow sequential colour scheme (“red”) (Figure 5A) were the most preferred colour schemes (Figure 11B). Among the top two preferred colour schemes, chosen by 89.1% of respondents, the red map (Figure 5A) was preferred by a majority 59.3% of scientists, and the diverging map (Figure 5C) was preferred by a majority 64.3% of stakeholders (Pearson χ
2 = 5.072, df = 1, α = 0.024).
Respondents wrote an average of 16 words when explaining their map colour scheme preference. Four main themes emerged concerning the influence of colour scheme: colour associations, cultural/social/mental connotations, zoning, and risk and response. Figure 12 highlights the words most frequently used by respondents in discussion of each colour scheme. Red hues were associated with concepts of danger, the presence of hazard, and volcanoes (Figure 12A). In contrast, blue hues were associated with concepts of safety, the absence of hazard, and water (Figure 12B):
Red always denotes hazard to me. – Stakeholder 89
The blue looks negative rather than low. The colour blue is usually associated with water. – Stakeholder 57
Many respondents remarked that the colour associations were evoked by cultural, social, or psychological connotations. Respondents used words such as “intuitively”, “universally”, “subliminal”, “socialised”, “logical”, and “used worldwide”, to describe the reasons for their strong associations with the colours red and blue. The associations with red and blue colours were consistent for both diverging and sequential colour schemes:
[The diverging map] give the impression of safety [in the blue areas] and emphasizes danger in the red areas... [The red map] gives the impression of increasing danger closer to the volcano. – Scientist 99
Red-blue seems to imply hazard and non-hazard, instead of hazard and less hazard. – Stakeholder 58
When describing the diverging colour scheme map, many respondents explained that they liked that the contrast made it much easier to “distinguish” and “delineate” the map into discrete “zones” (Figure 12C). However, the zones of colour had different context for different users. While responses describing the red map primarily focused on describing the map in the context of hazard, there was a marked increase in discussion of “risk” with the diverging colour scheme map among stakeholders (Figure 12B,C):
The blue de-emphasises lower likelihood areas allowing for a more risk-based focus. – Stakeholder 46
In some cases, the area of transition for the divergent colours was explicitly linked to response action:
The transition between hot and cold colours should be carefully set at a standard point, as anywhere in cold colours is unlikely to receive attention when planning in government departments is carried out. – Stakeholder 91
Map explanatory text and map format
In responding to questions about explanatory text (Table 1, Question 26–27; Figure 6), 94.1% of respondents said that providing informative text about the volcano, probabilistic hazard, and possible volcanic ash impacts on the map was helpful to some degree. The same proportion of respondents also viewed explanatory text on the map as important to some degree, with 50.5% ranking it “very important”. Results for questions about map format (Table 1, Question 28) are shown in Figure 13. The most popular format for receiving volcanic hazard maps was PDF, followed by GIS layer and JPEG (Figure 13). More than three-quarters of respondents (75.3%) chose more than one of the five format options provided.