An interdisciplinary research team has found statistical evidence of women being under-cited in academic literature. They are now studying similar effects along racial lines.
By Izzy Lopez
Scientific papers are the backbone of a research community and the citation of those papers sparks conversation in a given field. This cycle of publication and citation leads to new knowledge, but what happens when implicit discrimination in a field leads to papers by minority scholars being cited less often than their counterparts? A new team of researchers has come together to ask this question and dig into the numbers of gender and racial bias in neuroscience.
The team members include physicist and neuroscientist Danielle Bassett, J. Peter Skirkanich Professor of Bioengineering at the University of Pennsylvania, with secondary appointments in the Departments of Neurology and Psychiatry in Penn’s Perelman School of Medicine, statistician Jordan Dworkin, then a graduate student in Penn Medicine’s Department of Biostatistics, Epidemiology and Bioinformatics, and ethicist Perry Zurn, an Assistant Professor of Philosophy at American University.
Their study on gender bias, which recently appeared in Nature Neuroscience, reports on the extent and drivers of gender imbalance in neuroscience reference lists. The team has also published a perspective paper in Neuron that makes practical recommendations for improving awareness of this issue and correcting for biases.
They are now working on a second study, led by Maxwell Bertolero, a postdoctoral researcher in Bassett’s lab, that considers the extent and drivers of racial imbalances in neuroscience reference lists.
Together, Bassett, Dworkin and Zurn are using their combined research strengths to uncover the under-citation of women or otherwise minority-led papers in neuroscience and to assess its significance. This research is fundamental in highlighting a true gap in representation in research paper citations, which can have detrimental effects for women and other minorities leading science. In addition, they provide actionable steps to address the problem and build a more equitable future.
Your research team is a distinctive one. How did you come together for a study about gender discrimination in neuroscience citations?
Jordan Dworkin: It was a fortunate coincidence. In the run-up to a big neuroscience conference, I started seeing discussions on Twitter about gender-based discrimination in neuroscience. There were stories being shared of women’s papers being overlooked and reviewers seeing reference lists that were almost entirely made up of men. It was illuminating, especially because some people in the discussion were hesitant to take those experiences at face value. This skepticism, and occasional combativeness, seemed to stem from the view that citations are an untouchable, scientific bastion where researchers’ decisions are fully objective. The tension between that view and scholars’ lived experiences encouraged me to explore the existing literature on this issue.
As it turns out, there is really strong literature on issues of diversity and citation in science. Some disciplines have done field-specific investigations, such as the foundational studies in political science, international relations, and economics, but there wasn’t yet any research in neuroscience. Since biomedical sciences often have different approaches to citation, it seemed that it would be worth doing a deeper neuroscience-specific investigation to give quantitative backing to the issue of gender bias in neuroscience research.
Danielle Bassett: When Jordan and I started working together on this project, I knew it was important. To do it right, we needed to present the information in a way that made it actionable, with clear recommendations about how each of us as scientists can help address the issue. We also needed to add someone to the team with expertise in gender theory and research ethics. We especially wanted to make sure we were discussing gender bias in a way that was informed by recent advances in gender studies. That’s when we brought Perry in.
Perry Zurn: I’m a philosopher by training, with a focus on ethics and politics. Citations are both an ethical and a political issue. Citations reflect whose questions and whose contributions are recognized as important in the scholarly conversation. As such, citations can either bring in marginalized voices, voices that have been historically excluded from a conversation, or they can simply replicate that exclusion. My own field of philosophy has just as much of a problem with gender and racial diversity as STEM fields, something Dani and I have been talking about for a long time. This work seemed like a natural point of collaboration.
Describe this study and what it means for promoting gender diversity in neuroscience.
Bassett: For years now, various scholars and activists in science have drawn attention to issues of gender and racial inequities in the field. Most of these conversations, however, have placed responsibility for change in the hands of people in power, such as journal editors, grant reviewers, department chairs, presidents of scientific societies, etc. But many of the imbalances people notice, whether in conversation with peers or through studies like ours, are perpetuated by researchers at all levels. Given that every research project is built on prior research, and therefore every paper has a reference list of citations, every researcher can make a difference. Who we choose to cite matters.
Dworkin: To understand the role of gender in citation practices, we looked at the authors and reference lists of articles published in five top neuroscience journals since 1995. We accounted for self-citations, and various potentially relevant characteristics of papers, and we found that women-led papers are under-cited relative to what would be expected if gender was not a consideration in citation behavior. Importantly, we also found that the under-citation of women-led papers is driven largely by the citation behavior of men-led teams. We also found that this trend is getting worse over time, because the field is getting more diverse while citation rates are generally staying the same.
For a very simple example, if there were 10 women and 90 men neuroscientists in 1980, then citing 10% women would be roughly proportional. But with a diversifying field, say there are now 200 women and 200 men neuroscientists and citations are still 10% women. Sure, the percentage of women cited didn’t go down, but that percentage is now vastly lower than the true percentage in the field. That’s a dramatic example, but it shows you that if we’re going to call for equality in scientific citation, the number of women-led papers on a given reference list should reflect, or even exceed, the number of available and relevant women-led papers in a field, and our work found that it does not.
Bassett: This under-citation of women scientists is a key issue because the gaps in the amount of engagement that women’s work receives could have detrimental downstream effects on conference invitations, grant and fellowship awards, tenure and promotion, inclusion in syllabi, and even student evaluations. As a result, understanding and eliminating gender bias in citation practices is vital for addressing gender imbalances in a field.
Why are citations important to gender representation in neuroscience?
Dworkin: Unlike hiring and grant funding, citations are something every researcher participates in. For example, as a graduate student I did not have any role on a faculty search committee, or any power in an academic society to decide on conference speakers, but I still have reference lists in all my papers. Citations are a unique area where all researchers play a direct role, where each person has a chance to reflect on their own practices and use those practices to create change in their field. Their ubiquity means that citations function as a conversation within a field, and their presence or absence can signal whose work is valued and whose is not. On a more concrete level, citations are often used as metrics for a variety of important, potentially career-defining, decisions.
Bassett: There are a lot of underrepresented scholars who have fantastic ideas and write really interesting papers but they’re not being acknowledged — and cited — in the way they deserve. And there are great role models for all the young women who are thinking about going into science, but unless the older women scientists are being cited, the younger ones will never see them. Without serious changes in the field, and a deep commitment to gender and racial diversity, many young women and minority scientists won’t stick with it, they won’t be hired, they won’t be promoted, and they won’t be put in the textbooks.
Zurn: Exactly. I think it’s important not only to think about who we’re citing as leading scientists, but also what sorts of people we’re representing as scientists at all. If you are looking at neuroscience as a field and you see predominantly white cisgender men in the research labs and the reference lists, then you begin to think that is what a neuroscientist looks like. But this homogeneity is neither representative of an increasingly diverse field like neuroscience, nor supportive of continuing efforts to diversify STEM in general. We need to expand what a scientist looks like and citations are one way to do that.
Read the full interview on the Penn Engineering blog.
Danielle Bassett also has appointments in Penn Engineering’s Department of Electrical and Systems Engineering and Penn Arts & Sciences Department of Physics and Astronomy.
Jordan Dworkin is now an Assistant Professor of Clinical Biostatistics in the Department of Psychiatry at Columbia University.
Kristin Linn, Assistant Professor of Biostatics, Russell Shinohara, Associate Professor of Biostatistics, and Erin Teich, a postdoctoral researcher in Bassett’s lab, also contributed to the study published in Nature Neuroscience. It was supported by the National Institute of Neurological Disorders and Stroke through grants R01 NS085211 and R01 NS060910, the John D. and Catherine T. MacArthur Foundation, the Alfred P. Sloan Foundation, and the National Science Foundation through CAREER Award PHY-1554488.