Scientists who eventually peer into the universe with NASA’s powerful new James Webb Space Telescope will be the lucky ones whose research proposals made them through a highly competitive selection process.
But those who haven’t made the cut this time can at least know they’ve got a good shot, thanks to lessons from another popular NASA observatory.
Webb’s selection process is carefully designed to minimize the impact of biases or unconscious biases by forcing decision makers to focus on the scientific merit of the proposal rather than the person who made it.
“They evaluate every one of these proposals. They read it. They don’t know who wrote it,” explains Heidi Hamill, a multidisciplinary scientist with the James Webb Space Telescope. “These propositions are evaluated in a double-anonymous fashion, so that all you can see is science.”
This is a recent innovation in the time distribution of space telescopes. A change that only occurred after years of hard work by astronomers who worried that not everyone who wanted to use the Hubble Space Telescope was getting the same amount of attention.
Shows bias in who wins telescope time
One of the first clues came when Iain Neill Reid looked for signs of any potential gender bias in the acceptance rate of Hubble’s proposals. He’s the associate director for science at the Space Telescope Science Institute, the science operations center for both Hubble and now Webb.
His results, published in 2014, were astounding. The proposals submitted by women had a lower acceptance rate than the proposals led by men. This discrepancy has remained constant for more than twelve years, throughout the period of time it is analysed.
“I was surprised by how consistent it was,” Reed says. “There was a systemic effect.”
To try to fix this, he and his colleagues eventually developed the “blind” proposal review process now used on the other major Hubble, Webb and NASA telescopes. So far, the evidence suggests that this leveles the playing field – although this measure was initially opposed by much of the astronomy community.
Since any telescope in space is a rare and precious resource, NASA wants to devote its time to the most promising science. Anyone in the world can make a suggestion about where a space telescope should point, and there are so many demands that the majority of ideas should be rejected.
Even before the James Webb Space Telescope was launched, for example, the first call of proposals directed 1,173 ideas that required 24,500 hours of prime observation time. But only 6000 hours were available.
says Jane Rigby, an astrophysicist at NASA’s Goddard Space Flight Center who works as an operations project scientist for the new telescope.
And although Hubble was launched more than 30 years ago, astronomers are still clamoring for its use. Each year they submit 1,000 or more suggestions.
“Only the top 20% of those proposals will actually make it to the telescope to get the time,” Reid says.
Focus on science, not scientists
After his study that showed a gender disparity in the acceptance rates of Hubble’s proposals, Reed and his colleagues tried different solutions. First, instead of putting the name of the main scientist on the first page of the proposal, they tried to put it on the second page. Then they tried to use only initials. Nothing works.
“Then we got logical and said, ‘Let’s actually talk to some experts in the social sciences, because they can understand this better than us,'” Reed says.
They reached out to Stephanie Johnson of the University of Colorado and her then-student, Jessica Kirk, who is now at the University of Memphis. The couple sat in meetings that evaluated and rated the proposals. They noted that often, the discussion centered on who made the proposal, rather than scientific considerations.
Johnson recalls, “There might be a question about it, like, ‘Oh, you know, that sounds really good, but can they really do that?'” “A lot of times, there’s someone going to talk in the room and say, ‘I know that person… they’re going to find out, because that’s what they are.’”
“This assessment exists not just for science and research, but for researchers,” Kirk adds.
This means that already established and well-known astronomers have got an extra leg up.
“They got a pass,” Reed says. “They had a minimum, in some ways, to beat, of scientists who were coming into this field quite recently and had no track record.”
Adriana Manrique Gutierrez, NASA illustrator
Johnson and her colleagues recommended making the review process completely blind and anonymous. Not only will the evaluation committees not be able to see any names, all proposals must be written in such a way that it is absolutely impossible to know who the proposal is.
Some doubted that this new system would work
The institute surveyed the astronomy community to find out what it thinks about this potential change.
“You can imagine, the sudden reaction was actually very polar,” says Lou Strulger, vice president of instruments at the Space Telescope Science Institute and chair of the task force on the unknown proposal.
He says about half of those who responded favored the idea – and those tended to be women or people who were relatively young.
“They thought this would be a good way to make it fairer but to encourage new people to get involved,” he says.
But many astronomers have objections.
“I’ve ranged from ‘This will totally disturb the way science is done’ to ‘You’re basically going to fool yourself into giving time to people who don’t know what they’re doing’ — all sorts of things,” Stolger recalls.
However, the director of the institute gave the green light, and they continued moving forward. In 2018, astronomers conducted the first truly anonymous review of Hubble’s proposals. Priya Natarajan, a theoretical astrophysicist at Yale University, was there and headed the process. Sometimes she says that someone will try to guess who made the proposal.
“But the community’s acceptance was so massive that there would be other people on the boards who would say, ‘Oh no, no, come on, let’s stick to science,'” she says.
The gender difference flipped: ‘I was shocked’
A commitment to science has had a real impact. That year, for the first time ever, the acceptance rate for shows led by women was higher than for those led by men. the The difference was between the sexes flipped over.
“I was stunned,” says Natarajan. “There was an effect right away.”
And when members of the selection committees were finally allowed to see who had made a proposal they deemed worthy of the telescope’s time, Strulger says they never objected that the person was not up to the job, although they were often surprised.
“There was a lot of ‘Oh, that wasn’t at all what I thought was some kind of reaction,'” Strulger says.
Data from the past few years suggests that this process continues to help close the gap between men and women in acceptance rates for Hubble’s proposals, and may have improved equity in other ways as well.
There has been a huge rise in credits for astronomers who had never used Hubble before, Strulger says. “It went from about ten a year to 50 a year.”
Moreover, data from Webb’s first round of proposals shows hints of similar results with “a much closer gap in male and female acceptance rates,” says Strolger.
“This appears to be working, and it appears to be working as we expected.”
What other biases can influence telescope users?
However, anonymizing everything doesn’t solve all the problems in making sure everyone has equal opportunities, says Johnson, who notes that unconscious bias can influence who gets benefits in astronomy such as mentors and job opportunities.
“It’s not perfect. It doesn’t eliminate systemic bias, and I don’t know the effect of double anonymity in terms of creating greater racial equality,” she says. “But it seems to have removed some of the gender bias.”
Attempting to track equity issues is complicated by the fact that the Space Telescope Science Institute has historically not collected demographic information on those submitting research proposals.
“Partially by policy and partly by federal law, we are not permitted to collect this information,” Strollger explains.
For this reason, when Reed conducted his initial study looking at sex and Hubble, the best he could do was make assumptions about sex based on the principal scientist’s name or knowledge of people in the field.
Researchers are now looking for ways to learn more about applicants, perhaps by allowing people to voluntarily or anonymously provide information about themselves to a third party.
“We hope that by providing ways we can access more demographic data, we can begin to see where other biases might lie,” says Stolger.