Where and why does the chain break? Women in geoscience and letters of recommendation for postdoctoral fellowships

While women in geosciences are awarded 40% doctoral degrees, they hold less than 10% of full professorial positions. In looking for the cause of this disparity, the postdoctoral years have been identified as a crucial step, before and during which many women leave the Academia.

A recent study by Dutt et al., published this month in Nature Geoscience, investigated biases in recommendation letters for postdoctoral fellowships and the relationship between tone and length of these letters and the gender of the applicants. This study suggests that “women are disadvantaged right from the beginning of their geoscience careers because they are possibly perceived as not contributing as much as their male colleagues”.

With the aim of spreading awareness of possible reasons of disparity between women and men in the Geosciences, we invited the lead author of this study, Dr. Kuheli Dutt — Assistant Director for Academic Affairs & Diversity at Lamont-Doherty Earth Observatory (LDEO) at Columbia University (see Biography at the end) — to discuss with us about the nature and the implications of their findings, as well as possible initiatives to boost careers in geoscience for women.

What was the motivation behind this study?

Previous research has suggested that men and women are described differently in letters of recommendation. Fields have included psychology, medicine, chemistry/biochemistry, but there wasn’t such a study for the geosciences – a male-dominated discipline. So with more than 1200 letters contributed by 1100 individuals from around 500 institutions in 54 countries, this was a rich dataset with which to explore this topic. The dataset was archival and covered the period 2007-2012.

What is the main finding of your study? What are the main points in accord/in contrast with respect to previous studies?

The main findings are that women are significantly less likely to receive outstanding letters compared to men. It did not matter which region the letters were from or whether the recommender was male or female. Also, letters from the Americas were significantly longer than any other region. Letter tone seemed to be equivalently distributed among world regions, i.e. no particular region had stronger or weaker letters. Given the large, international dataset, our results are important because they uncover what appears to be a systemic problem in the geosciences, one that is consistent with the possibility of widespread implicit bias. I’d like to stress “implicit” here because we are not trying to assign blame or accuse anyone of being consciously sexist.

The main findings are that women are significantly less likely to receive outstanding letters compared to men. […] [This] appears to be a systemic problem in the geosciences, one that is consistent with the possibility of widespread implicit bias. I’d like to stress “implicit” here because we are not trying to assign blame or accuse anyone of being consciously sexist.

Our results are similar to studies such as Schmader et al. (2007) and Madera et al. (2009), which found that men were described in stronger terms in recommendation letters. Our results were also consistent with Moss-Racusin et al. (2012) and Reuben et al. (2014), which tested for implicit gender bias. Moss-Racusin et al. (2012) found that faculty consistently ranked male applicants higher than identical female applicants for a lab manager position. Reuben et al. (2014) found that people were twice as likely to pick a male candidate over a female candidate for a math problem, even though both male and female applicants were equally good at the math problem.

Could you summarise the method you used to code the letters in three categories? How did you avoid any subjective evaluation of the letters?

Before coding, all letters were stripped of any identifying information including gender information and references to gender pronouns. So at the time of coding the letters, we did not know the gender of the applicant or recommender. Each letter had been assigned a unique serial number, as had each applicant and each recommender – that way we were able to keep track of which applicants got multiple letters and which recommenders had written more than one letter. About the actual coding, we used a coding manual, which was developed using a combination of: i) input from previous studies such as Trix & Psenke (2003); Madera et al. (2007); Schmader et al. (2009); ii) guidelines for quantitative content analysis (Riffe et al, 2005);  and iii) feedback from senior scientists at the host institution, specifically scientists who serve or have served on postdoctoral selection committees. These scientists were shown a subset of randomly selected anonymized comments from the letters to which they were asked to assign labels of excellent, good or doubtful. These were used as exemplars while developing the coding manual.

Women tend to be described in more communal terms (“caring”, “nurturing”) whereas men as “confident” and “dynamic”.  […]   Leadership qualities are more likely to be associated with terms like “confident” and “dynamic”, which are viewed as stronger predictors for professional success than terms like “caring” or “nurturing”.

Given the large size of the dataset, the coding manual explicitly defined the content for each overall tone, and this coding scheme was applied across all letters. A simple way to understand the coding scheme is: the “excellent” letters praised the candidate’s outstanding scientific capabilities and/or scientific leadership. Such letters were likely to portray the applicant as conducting “groundbreaking” research, or as a “role model” or “leader in the field” or “rising star”. The “good” category had letters that offered clear praise of the candidate but were not outstanding. And the “doubtful” category had letters that were either negative or cast doubt on the candidate’s scientific caliber.

Did you find any relationship between the age of the referees and the tone of the letters?

Unfortunately we were not able to test that, since the only information we retained (due to confidentiality issues) was gender and region.

A recommendation letter is not only a list of scientific expertise, but especially a subjective description of interpersonal, communication and leadership skills. Could the bias found in your letters be due to different words commonly used in many cultures to differently describe women and men?

Yes, that would appear to be a very reasonable explanation. As Madera et al. (2009) showed, women tend to be described in more communal terms (“caring”, “nurturing”) whereas men as “confident” and “dynamic”. While there is nothing wrong with these terms by themselves, the issue is that leadership qualities are more likely to be associated with terms like “confident” and “dynamic”, which are viewed as stronger predictors for professional success than terms like “caring” or “nurturing”. Another reason we believe that the bias is implicit is that both male and female applicants got approximately the same proportion of doubtful letters. The number of doubtful letters was too small to do a detailed statistical analysis, so we just reported the percentages. If recommenders were consciously sexist, then it stands to reason that women would have likely received more of such doubtful letters, which was not the case here.

The biological sciences […] tend to have much better representation of women than say physics or engineering or geoscience. Leslie et al showed that women tend to be underrepresented in fields where raw innate talent is a perceived as a requirement for success, since women are stereotyped as not possessing such raw innate talent.

Women hold 40% of the doctoral degrees (at federally funded R&D centres) but take only 24% of the postdoctoral positions. This bottleneck in the geosciences career is not negligible. Your work suggests that one important cause may be the bias in the recommendation letters. However, your female and male samples seem to suggest that an important issue is the percentage of female applicants (362, 29.5%) compared to males (862, 70.5%). In your opinion, why do so few females apply for a postdoctoral position? Which are the main difficulties in work and life they have to deal with after being awarded the PhD? Do you have an idea regarding the alternative paths females embark after the doctoral studies?

Bias in recommendation letters is just one piece of the problem, and is, in my opinion, indicative of a bigger problem, i.e. the difference in how men and women are perceived. For example, the Wenneras and Wold (1997) study showed that women postdocs needed 2.5 times more publications than men in order to be assigned the same ranking of scientific competence as men. And a Berkeley report showed that postdoctoral years are associated with the largest leak in the pipeline. That report also found that married women with children were significantly less likely to have a tenure-track position compared to married men with children; and that single women without children were approximately as successful as married men with children to receive a tenure-track position.

I think we need to be clear that having children is not the real problem (assuming an institution has supportive policies such as paid time off and stop-the-clock provisions). The biological sciences and some other disciplines tend to have much better representation of women than say physics or engineering or geoscience. Leslie et al. (2015)showed that women tend to be underrepresented in fields where raw innate talent is a perceived as a requirement for success, since women are stereotyped as not possessing such raw innate talent.

Given that at the postdoc level we have around 40% women in the geosciences which becomes around 10% at the full professor level, we need to ask which of these two scenarios is more likely with respect to the leak in the pipeline:

1. i) Those women who were drawn to the geosciences, and were competent enough to be awarded a doctoral degree, were suddenly not good enough anymore, and/or no longer interested in a subject they spent years studying and investing in; OR
2. ii) Those women faced more struggles, be it the absence of female role models and supportive networks, or having to deal with the stereotype threat of not being perceived as “dynamic” or “leader” the way men are.

In terms of alternative career paths that women pursue – I can only offer anecdotal information. These include: support staff/scientist type of positions or part-time positions; education, outreach and media programs; science administration/policy; industry.

It is also important to mention here that there has been some awareness in higher education about these issues, and nowadays some institutions are making efforts to address the problem. For example, at my own institution, where I serve as the diversity officer, we have gone from 18% women at the junior scientist level in 2006 to 45% today. As these women advance through the ranks they can serve as role models for other women, which will hopefully lead to better overall representation of women in these fields.

Your study does not mention the “actual” qualification of the applicants, probably because it is very difficult to quantitatively estimate it for several reasons. However, female and male students still receive different cultural and educational inputs since an early age in most of the cultures/countries. Also, at some point in life, biology may force women to slow down their educational/professional activities. Did you have a chance to look at the distribution of the actual qualification among PhD graduates to ensure that a sample of female applicants can indeed be expected to be equally qualified as a sample of males? Since a real gap in the qualification between male and female researchers may exist for the above mentioned factors, in your opinion, could it be reasonably expected that the smaller percentage of “excellent” recommendation letters actually reflect a smaller percentage of excellent female researchers?

We were unable to control for applicant qualifications because all letters had been stripped of any identifying information, i.e. it was not possible for us to assign a CV to each letter. Given this, we were unable to *statistically* rule out the possibility that the men may have been better qualified than the women. I highlight “statistically” because given the large and international nature of our dataset, it is highly unlikely that globally there is a systemic deficit only in the quality of female applicants. Our assertion is strengthened by the fact that our results are similar to studies that were able to control for applicant qualifications (Trix and Psenke, 2003; Schmader et al., 2007; Moss-Racusin et al., 2012; Reuben et al., 2014). Our results uncover what appears to a systemic problem across the geosciences, one that is consistent with implicit gender bias, and how women’s contributions are perceived compared to men.

Often, reviewers and referees are very well-meaning but unaware of their own implicit biases. Do you think this study should be put into recommendations for referees, and how to achieve this without frustrating those that are already doing their best to act fairly? After all, female researchers can hardly ask themselves for better recommendation letters…

The first step is for people to be aware that there is a problem. I would suggest starting a dialogue on implicit bias, and having workshops on how to write recommendation letters. To be clear, I am NOT suggesting that everyone should start writing outstanding letters! In my opinion, here is a simple method of asking recommenders to assess their own letters:

a)    Do recommenders believe that the male applicants they have written letters for are in fact better than the female applicants? If so, why? (And there might well be valid reasons for this.)

b)   If recommenders don’t necessarily believe that the male applicants they wrote letters for were better than female applicants, did they unconsciously choose certain words and phrases that have gendered connotations? (And this might well be the case, given the way male and female roles are perceived in society.)

c)    Based on the answers to the above questions, a simple exercise is for recommenders to look at letters that they wrote for males and compare them to what they wrote for females. I suspect that for many recommenders, especially those unfamiliar with implicit bias research, this exercise might prove to be an eye-opener.

The important thing to stress here is that we are not trying to blame or shame anyone; in fact, I would suggest that recommenders do the above exercise in the privacy of their own offices or in any non-threatening, non-judgmental environment. Our results shed light on the problem, and we hope that people use these results to engage in meaningful discussions and next steps to address the problem.

References.

Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. S., & Block, C. J. (2016). Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nature Geoscience. 

References to all other mentioned studies can be found on the Nature Geoscience page of the study.

Dr. Kuheli Dutt is the Assistant Director for Academic Affairs & Diversity at Lamont-Doherty Earth Observatory (LDEO) at Columbia University. In this capacity she serves as the chief diversity officer, and her office aims to enable better representation of women and minorities among Lamont scientists and research professors. With this goal, Kuheli participates in the following areas: appointments and promotions; search procedures; salary structures; family leave policies; institutional governance; and postdoctoral affairs. Kuheli also serves on the Columbia University Commission on the Status of Women (a body of the Columbia Senate), and the University Life Task Force on Race, Ethnicity and Inclusion. A social scientist by training with a PhD in public policy, Kuheli has presented at national level conferences on advancing diversity, and is the co-author of the 2015 AGU-Wiley book “Women in the Geosciences”.

SM ECS-reps Lucia Gualtieri, Laura Parisi and Laura Ermert had the pleasure to be engaged in this interesting conversation with Dr. Kuheli Dutt.

In our experience, whenever we get into the ‘opportunities’ discussion, the comments flow plenty…so please add your thoughts in the space below, or write us on Twitter @EGU_Seismo or Facebook!