I love me some subconscious gender bias and its hidden malevolent effects on society. So you’d think I’d have been excited when reports came out of a study claiming that female-named hurricanes are more deadly than male-named ones, because people don’t take them as seriously and don’t prepare or evacuate adequately.
Even more than rooting out subconscious gender bias, though, I love critically digging into scientific work and picking apart how strong its data, interpretations, and conclusions are. Especially when they sound fishy to me, and this one sounded a bit fishy. Gender bias is trendy right now, and while I think it can and does have powerful effects, that hurricane-deadliness thing is a huge claim. So as interested as I was in the original study, I didn’t get really excited until I started seeing some critiques.
Even more than both those things? I love names and naming and the psychology of names and name trends and all that. I try to keep that particular love quiet most of the time because it’s boring and weird to most people and not particularly useful. But this vortex of three keen interests of mine amounted to my spending a solid chunk of the last 24 hours reading, analyzing, playing with numbers, and generally geeking out in the worst way.
Unfortunately, I don’t have access to the actual writeup of the study… fortunately, some of the data is available for public view here, which is awesome. So I’m doing the best I can with what I have.
- The authors took data on all hurricanes since 1950, including the hurricane name’s position on a scale from most masculine to most feminine, and number of deaths associated with the hurricane.
- Statistical analysis showed that names that were ranked as more feminine were associated with more deaths.
- To test their hypothesis that the higher death toll was due to social perceptions of a female-named hurricane as milder than a male-named, the authors did a series of experiments where participants predicted the severity of an invented hurricane, or indicated whether they’d evacuate or take other precautions for it. The scenarios were identical except for the gender of the hurricane name: Victor or Victoria, Alexander or Alexandra, as examples. Participants overall took the male-named hurricanes more seriously than the female-named ones.
- The authors take the hypothetical scenario response results as evidence that people in general take female-named hurricanes less seriously, and are less likely to adequately prepare or evacuate, leading to a higher death toll.
Fine so far as it goes. Now let’s look at the critiques:
- Up until 1979, all hurricanes had female names. Over time since 1950, hurricanes have become less deadly, which confounds the study’s data. (The study authors did do a separate analysis of hurricanes from 1950-1978 and 1979-2012, but the results did not reach statistical significance, which the authors say is because each group is too small.)
- None of the six studies on hypothetical scenarios, which the authors used to demonstrate the causal link they’re claiming, used a population that is necessarily representative of the coastal town-dwellers who are actually impacted by hurricanes: three of the studies used undergrads, three used participants recruited via an online platform.
- There are a lot of known major factors that impact a hurricane’s deadliness, and it’s unlikely (and insufficiently demonstrated) that a subconscious gender bias is strong enough to have a significant effect above those other factors. (I know, I worded that vaguely. That’s not the part that I examined, so read the article for more details if you’re interested.)
The first bullet point is huge. If it’s true that hurricanes in earlier decades were more deadly than hurricanes in more recent years, then that’s a major confound that potentially overthrows the whole study. The other issues are relevant and important, but the first one is major.
- The analysis looked not only at male or female designation of names, but at the perceived masculinity or femininity of a name. (For non-hurricane examples, most people likely perceive Angelica as a more feminine name than Jean, and Brock as a more masculine name than Julian.) It was the name’s place on the masculine-to-feminine scale that related to its deadliness, not simply the male/female designation of the name, and those results held true even before 1979 with more feminine female names accompanying more deadly hurricanes.
- Years elapsed since the hurricane (in other words, how long ago the hurricane happened) is not correlated with the hurricane’s deadliness.
They addressed the other critiques, but their commentary amounted to “Well, yeah, that’s an unfortunate feature of the resources we had available, but that doesn’t disprove it.” Which is true.
From reading the articles linked above, I had two major questions:
- Have hurricanes actually become less deadly over time? It seems to me that the study’s authors and its critics are saying directly contradictory things about this, and it makes a huge difference.
- How did the study authors determine the relative masculinity and femininity of names? And how valid was this ranking?
To my delight there is a spreadsheet available giving the hurricane names, ranking on masculinity or femininity, death toll, and lots of other data. So I’ve been able to play around with it a bit myself, and while my stats knowledge is pretty limited, I can do basic math.
To the first question, “have hurricanes become less deadly over time?” there seems to be a direct contradiction between what the study authors say — “how long ago the storm occurred did not predict its death toll” — and what the critics say — “hurricanes have also, on average, been getting less deadly over time.” Of course those sentences have different words in them, and it’s possible for both to be literally correct. And in fact, as far as I can tell, they are. This is why you have to read your science reporting carefully, kids!
I am sure there’s a sophisticated statistical regression thingy one could do with this data, and it might be that I’m doing it completely wrong, but here’s what I got. Using the good ol’ average formula, I came up with these numbers:
- The average death toll of all hurricanes from 1950-1978 (when all hurricanes got female names) was 27.
- The average death toll of all hurricanes from 1979-2012 (when hurricanes got male and female names) was 16.
Thus far, the critics seem to be winning. That’s a pretty substantial difference between the all-female group and the male-female group. Without doing any fancier calculations, this layman’s eye is gleaming with suspicion.
To get a sense of whether the decline was smooth over time, I did the average death toll by decade, and here’s what I got:
- 50s: 24
- 60s: 36
- 70s: 16
- 80s: 6
- 90s: 15
- 00s: 19
- 10s: 68
Two major things jump out: the 80s were a great time to be in a hurricane, and the 10s were terrible. Of course, there were only three years counted in the 10s, 2010-2012, and in fact only three hurricanes happened in those years, so as a decade it should be discounted entirely. And one of those hurricanes was Sandy, which has the third highest death toll on the list (159, after Diane in 1955 with 200 deaths, and Camille in 1969 with 256. Hurricanes Audrey and Katrina were excluded as huge outliers, with death tolls above 400 and 1800.)
The authors’ statement, “how long ago the hurricane occurred did not predict its death toll,” may well be true. The weirdly low-fatality 80s plus the highly deadly Sandy coming right in at the end could blur the overall trend. (Please, someone who’s competent with statistics chime in and explain all the things to me!) But that doesn’t change the fact that hurricanes in the three decades of female-only names were, on average, more deadly than hurricanes since male names began being used.
So what about the author’s second rebuttal, that it wasn’t just male or female designation of a name, but ranking on a scale of masculinity to femininity, that predicted deadliness? They argue that even in the all-female era of hurricane names, a Hurricane Angelica would be taken less seriously than a Hurricane Jean, and thus lead to more fatalities. They rated all hurricane names on a scale from 1 to 11, with 1 being extremely masculine and 11 being extremely feminine, and used that rating in their analysis.
I wasn’t able to find out how they assigned the masculinity-femininity ratings to each name, and I’d be much obliged if someone who has access to the article could tell me. Because obviously, the analysis is only as good as the validity of those ratings. And I am — to put it mildly — skeptical of how valid such ratings could possibly be, especially in terms of supporting the authors’ hypothesis.
Name perception is a tricky thing. It’s formed by a lot of factors, sound and association being the biggest. I felt fairly confident that most people reading this would agree that Angelica is a more feminine name than Jean: it’s got more syllables, an ‘a’ ending, and the word ‘angel’ contained in it. Those are three big markers of femininity that add up to a pretty indisputable trend in how we’re likely to perceive it. But what about, say, Flossy and Edith? Alma and Ione? Erin and Sandy? In each of those pairs of names, which do you think is more feminine? How easy is it for you to judge? I’ll put the answers according to the authors in white text: (Edith, Alma, Sandy) Is that what you’d have said?
I did a little quick and dirty experiment on my facebook wall, asking friends to rank five names from the list in order of least to most feminine. The names were Erin, Sandy, Barbara, Cleo, and Ginger. After 13 respondents (I SAID it was quick and dirty) I tallied up my results. One thing I noticed was that absolutely no one ranked all five names the same way. Ginger was always in the top 3, Erin always in the bottom 3, but the others were all over the map. Sandy was particularly interesting: 5 of my respondents ranked it as a 1 (least feminine), and 5 ranked it as a 4 or 5. When the scores were averaged it ended up ranked right in the middle. I was curious about Sandy because I think of it as an androgynous name that’s used mostly for females, and thus on a 1-11 scale of masculinity/femininity, I’d rate it around a 7. The study authors rated it at 9, solidly in the middle of the feminine category, above names like Florence, Jeanne, Connie, and Opal. One person suggested that it’s an association with Sandy from Grease which leads to a more feminine impression of the name. So some people hear “Sandy” and think Olivia Newton-John, while others hear it and think, “Probably a girl but could be a boy.”
Interestingly, when I averaged my respondent’s ranking of each name, I came up with the exact same order that the study’s ratings of each name gave: Erin, Cleo, Sandy, Barbara, Ginger. Although none of my friends ranked them in exactly that order, the average matched neatly with the names’ order in the study. So yes, while individual ratings of a name may differ considerably, the aggregate impression over a large group of people might be stable.
But the question then comes in, which large group of people? If there’s one thing you can say definitively about name perceptions, it’s that they change dramatically over the years. Names like Florence, Mildred, Bertha, and Edith were once considered youthful, fresh, sweet names for a baby girl. Now they conjure up an image of a grandma or great-grandma. I think of Ashley, Stephanie, Jennifer and Samantha as normal feminine, young-woman names, but they’re on the cusp of becoming mom names, the kind of names that Barbara, Carol, and Donna are to me. Our impression of a name is profoundly tied to the generation we were born in. And this is important to assessing the study because the whole argument is based on the assumption that people’s subconscious impressions of a name — and specifically its masculinity or femininity — are shaping their behavior.
I don’t doubt for a minute that the name given to a hurricane gives people a qualitative impression of it, based on how they normally think of that name. Names shape how we see things and people; that’s why I find them so fascinating. But the impression I have of Hurricane Carol now is very different than the impression a community in 1954 would have had of Hurricane Carol. In 1954, most Carols were in their teens and twenties. Carol then was something like Madison today, just in terms of when it became popular and was being commonly used.
I assume that the study authors’ name ratings were obtained by asking a (hopefully large and diverse) sample of people to rate the names on femininity and masculinity. If they did their job right, they also checked their ratings for validity among a few different groups of people. Even if they did, though, that doesn’t mean the ratings are valid when applied to the specific people relevant to their hypothesis: the community of people that was responding to news of an incoming hurricane.
In fact, when it comes to earlier decades, we can be pretty confident that they aren’t: the entire cultural milieu, the people that were associated with a name, and the names that people thought sounded perfect for their baby girls and boys were entirely different.
None of this disproves the study. It’s possible that perceptions of a name’s “femininity” remain stable over time even while the images of what kind of person goes with that name shift decade by decade. But in my view there is not nearly enough evidence that this is the case, and some good reasons to think that it may not be.
I also question how valid the correlation is between masculinity-femininity rating and associated deaths based simply on the distribution of names on the masculinity-femininity scale. The names tend to cluster at one end of the scale or the other, with the overwhelming majority lying up at the feminine end. There are 25 names in the bottom half of the scale, rated between 1 and 6: all but 3 of these are rated between 1 and 3. So the bottom half of the scale has fewer names and they’re heavily weighted toward the low end. There are 58 names in the top half of the scale, 46 of which are rated between 8 and 10. So the top half has over twice as many names, and they’re weighted toward the top end but nearer the middle than on the masculine side of things. There is only one name each in the 3, 4, and 5 zones, and only three in the 6. Hopefully the authors did some fancy math to correct for any effects that this grossly uneven distribution might have contributed to, but without seeing the study I don’t know, and I remain skeptical.
As best I can tell from what’s available to me, the criticisms of the original study by Jeff Lazo are sound. Hurricanes did kill more people on average in the era of all-female names than since then. The authors’ counter-argument that they also found an effect of relative femininity of a name depends on fine-tuned rankings of a name’s femininity and masculinity, the validity of which I’m highly skeptical. It is possible that the authors’ contention is true and that the name given to a hurricane impacts people’s preparedness level, to an extent that its effect is noticeable over the other relevant factors (media coverage, economic issues, etc.) but it is not sufficiently demonstrated by the study thus far.