CHARLOTTESVILLE, Va., Dec. 7, 2021 /PRNewswire/ — A large-scale systematic investigation to replicate high-impact, preclinical cancer biology experiments identified barriers to conducting replications and observed weaker evidence for the findings compared with the original experiments. Unnecessary friction in the research process may be slowing the advancement of knowledge, solutions, and treatments.
Today, eLifepublishes the final outputs of the Reproducibility Project: Cancer Biology, an 8-year effort to replicate experiments from 53 high-impact papers published between 2010 and 2012. Tim Errington, the Director of Research at the Center for Open Science and project leader said: “The purpose of the project was to transparently assess the extent to which there are challenges for conducting replications and obtaining similar evidence of published findings in cancer biology research.”
Launched in 2013, the Reproducibility Project: Cancer Biology was a collaboration between the Center for Open Science, a nonprofit culture change organization with a mission to improve openness, integrity, and reproducibility of research, and Science Exchange, the world’s first online R&D marketplace whose mission is to accelerate scientific discovery. With support from Arnold Ventures (formerly the Laura and John Arnold Foundation), the team conducted a systematic process to select high-impact cancer research papers published between 2010 and 2012. Based on the selection criteria, most of the papers came from high-profile journals such as Nature, Science, and Cell. A total of 193 experiments were selected for replication.
The team designed replication experiments of key findings from each paper by reviewing the methodology and requesting information about protocols and availability of reagents. Then, appropriate expertise for conducting the experiments was sourced through the Science Exchange marketplace.
For each paper the detailed protocols for the replication experiments were written up as a Registered Report and submitted to eLife for peer review; moreover, work on the replication experiments could not begin until the Registered Report had been accepted for publication. The completed replication experiments were then written up as a Replication Study, peer reviewed and published in eLife. Two of the papers published today are capstone summaries of the entire project.
The first paper “Challenges for Assessing Replicability in Preclinical Cancer Biology” reports on the challenges confronted when preparing and conducting replications of 193 experiments from 53 papers. None of the experiments were described in sufficient detail to design a replication without seeking clarifications from the original authors. Some authors (26%) were extremely helpful and generous with feedback, and some authors (32%) were not at all helpful or did not respond to requests. During experimentation, about two-thirds of the experiments required some modification to the protocols because, for example, model systems behaved differently than originally reported. Ultimately, 50 replication experiments from 23 papers were completed, a small proportion of what were planned. Errington noted, “We had challenges at every stage of the research process to design and conduct the replications. It was hard to understand what was originally done, we could not always get access to the original data or reagents to conduct the experiments, and model systems frequently did not behave as originally reported. The limited transparency and incomplete reporting made the efforts to replicate the findings much harder than was necessary.”
The second paper, “Investigating the Replicability of Preclinical Cancer Biology“, reports a meta-analysis of the results of the 50 replication experiments that did get completed. Many of these experiments involved measuring more than one effect (e.g., measuring the influence of an intervention on both the tumor burden and overall survival), and the 50 experiments that were completed included a total of 158 effects. Most of these effects (136) were reported as positive effects in the original papers, with 22 being reported as null effects. The meta-analysis also had to take into account that 41 of the effects were reported as images rather than as numerical values in the original papers. Replications provided much weaker evidence for the findings compared to the original experiments. For example, for original positive results, replication effect sizes were 85% smaller than the original effect sizes on average.
The team also used a number of binary criteria to assess whether a replication was successful or not. A total of 112 effects could be assessed by five of these criteria, and 18% succeeded on all five, 15% succeeded on four, 13% succeeded on three, 21% succeeded on two, 13% succeeded on one, and 20% failed on all five. Collectively, 46% of the replications were successful on more criteria than they failed and 54% of the replications failed on more criteria than they succeeded.
Summarizing, Errington noted, “Of the replication experiments we were able to complete, the evidence was much weaker on average than the original findings even though all the replications underwent peer review before conducting the experiments to maximize their quality and rigor. Our findings suggest that there is room to improve replicability in preclinical cancer research.”
Brian Nosek, Executive Director from the Center for Open Science and co-author added, “Science is making substantial progress in addressing global health challenges. The evidence from this project suggests that we could be doing even better. There is unnecessary friction in the research process that is interfering with advancing knowledge, solutions, and treatments. Investing in improving transparency, sharing, and rigor of preclinical research could yield huge returns on investment by removing sources of friction and accelerating science. For example, open sharing of data, materials, and code will make it easier to understand, critique, and build upon each other’s work. And, preregistration of experiments and analysis plans will reduce the negative effects of publication bias and distinguish between planned tests and unplanned discoveries.”
These papers identify substantial challenges for cancer research, but they occur amid a reformation in science to address dysfunctional incentives, improve the research culture, increase transparency and sharing, and improve rigor in design and conduct of research. Science is at its best when it confronts itself and identifies ways to improve the quality and credibility of research findings. The Reproducibility Project: Cancer Biology is just one contribution in an ongoing self-examination of research practices and opportunities for improvement.
Supporting Information Additional supporting information about the project is also available via this OSF link. This includes a fact sheet, background information, a list of independent researchers that have agreed to be listed as possible contacts for interviews, and a guide with links to navigating the content of the RP:CB papers, reviews, and supporting information.
About Center for Open Science Founded in 2013, COS is a nonprofit technology and culture change organization with a mission to increase openness, integrity, and reproducibility of scientific research. COS pursues this mission by building communities around open science practices, supporting metascience research, and developing and maintaining free, open source software tools, including the Open Science Framework (OSF). For more information, visit cos.io.
About Science Exchange Founded in 2011 with the goal to accelerate scientific discovery, Science Exchange is an online marketplace powering scientific outsourcing for the R&D industry – providing companies with instant access to scientific services from a trusted network of contract research organizations. Science Exchange’s R&D marketplace simplifies scientific outsourcing and eliminates contracting delays so scientists can access innovation without the administrative burdens. Since 2011, Science Exchange has raised more than $70 million from Norwest Venture Partners, Maverick Ventures, Union Square Ventures, Collaborative Fund, Windham Ventures, OATV, the YC Continuity Fund, and others. For more information, visit scienceexchange.com.
About eLife eLife is a non-profit organisation created by funders and led by researchers. Our mission is to accelerate discovery by operating a platform for research communication that encourages and recognises the most responsible behaviours. We review selected preprints in all areas of biology and medicine, while exploring new ways to improve how research is assessed and published. eLife receives financial support and strategic guidance from the Howard Hughes Medical Institute, the Knut and Alice Wallenberg Foundation, the Max Planck Society and Wellcome. Learn more at elifesciences.org/about.
About Arnold Ventures Arnold Ventures is a philanthropy dedicated to tackling some of the most pressing problems in the United States. We invest in sustainable change, building it from the ground up based on research, deep thinking, and a strong foundation of evidence. We drive public conversation, craft policy, and inspire action through education and advocacy. For more information, visit arnoldventures.org.
More than 40 trillion gallons of rain drenched the Southeast United States in the last week from Hurricane Helene and a run-of-the-mill rainstorm that sloshed in ahead of it — an unheard of amount of water that has stunned experts.
That’s enough to fill the Dallas Cowboys’ stadium 51,000 times, or Lake Tahoe just once. If it was concentrated just on the state of North Carolina that much water would be 3.5 feet deep (more than 1 meter). It’s enough to fill more than 60 million Olympic-size swimming pools.
“That’s an astronomical amount of precipitation,” said Ed Clark, head of the National Oceanic and Atmospheric Administration’s National Water Center in Tuscaloosa, Alabama. “I have not seen something in my 25 years of working at the weather service that is this geographically large of an extent and the sheer volume of water that fell from the sky.”
The flood damage from the rain is apocalyptic, meteorologists said. More than 100 people are dead, according to officials.
Private meteorologist Ryan Maue, a former NOAA chief scientist, calculated the amount of rain, using precipitation measurements made in 2.5-mile-by-2.5 mile grids as measured by satellites and ground observations. He came up with 40 trillion gallons through Sunday for the eastern United States, with 20 trillion gallons of that hitting just Georgia, Tennessee, the Carolinas and Florida from Hurricane Helene.
Clark did the calculations independently and said the 40 trillion gallon figure (151 trillion liters) is about right and, if anything, conservative. Maue said maybe 1 to 2 trillion more gallons of rain had fallen, much if it in Virginia, since his calculations.
Clark, who spends much of his work on issues of shrinking western water supplies, said to put the amount of rain in perspective, it’s more than twice the combined amount of water stored by two key Colorado River basin reservoirs: Lake Powell and Lake Mead.
Several meteorologists said this was a combination of two, maybe three storm systems. Before Helene struck, rain had fallen heavily for days because a low pressure system had “cut off” from the jet stream — which moves weather systems along west to east — and stalled over the Southeast. That funneled plenty of warm water from the Gulf of Mexico. And a storm that fell just short of named status parked along North Carolina’s Atlantic coast, dumping as much as 20 inches of rain, said North Carolina state climatologist Kathie Dello.
Then add Helene, one of the largest storms in the last couple decades and one that held plenty of rain because it was young and moved fast before it hit the Appalachians, said University of Albany hurricane expert Kristen Corbosiero.
“It was not just a perfect storm, but it was a combination of multiple storms that that led to the enormous amount of rain,” Maue said. “That collected at high elevation, we’re talking 3,000 to 6000 feet. And when you drop trillions of gallons on a mountain, that has to go down.”
The fact that these storms hit the mountains made everything worse, and not just because of runoff. The interaction between the mountains and the storm systems wrings more moisture out of the air, Clark, Maue and Corbosiero said.
North Carolina weather officials said their top measurement total was 31.33 inches in the tiny town of Busick. Mount Mitchell also got more than 2 feet of rainfall.
Before 2017’s Hurricane Harvey, “I said to our colleagues, you know, I never thought in my career that we would measure rainfall in feet,” Clark said. “And after Harvey, Florence, the more isolated events in eastern Kentucky, portions of South Dakota. We’re seeing events year in and year out where we are measuring rainfall in feet.”
Storms are getting wetter as the climate change s, said Corbosiero and Dello. A basic law of physics says the air holds nearly 4% more moisture for every degree Fahrenheit warmer (7% for every degree Celsius) and the world has warmed more than 2 degrees (1.2 degrees Celsius) since pre-industrial times.
Corbosiero said meteorologists are vigorously debating how much of Helene is due to worsening climate change and how much is random.
For Dello, the “fingerprints of climate change” were clear.
“We’ve seen tropical storm impacts in western North Carolina. But these storms are wetter and these storms are warmer. And there would have been a time when a tropical storm would have been heading toward North Carolina and would have caused some rain and some damage, but not apocalyptic destruction. ”
Associated Press climate and environmental coverage receives support from several private foundations. See more about AP’s climate initiative here. The AP is solely responsible for all content.
It’s a dinosaur that roamed Alberta’s badlands more than 70 million years ago, sporting a big, bumpy, bony head the size of a baby elephant.
On Wednesday, paleontologists near Grande Prairie pulled its 272-kilogram skull from the ground.
They call it “Big Sam.”
The adult Pachyrhinosaurus is the second plant-eating dinosaur to be unearthed from a dense bonebed belonging to a herd that died together on the edge of a valley that now sits 450 kilometres northwest of Edmonton.
It didn’t die alone.
“We have hundreds of juvenile bones in the bonebed, so we know that there are many babies and some adults among all of the big adults,” Emily Bamforth, a paleontologist with the nearby Philip J. Currie Dinosaur Museum, said in an interview on the way to the dig site.
She described the horned Pachyrhinosaurus as “the smaller, older cousin of the triceratops.”
“This species of dinosaur is endemic to the Grand Prairie area, so it’s found here and nowhere else in the world. They are … kind of about the size of an Indian elephant and a rhino,” she added.
The head alone, she said, is about the size of a baby elephant.
The discovery was a long time coming.
The bonebed was first discovered by a high school teacher out for a walk about 50 years ago. It took the teacher a decade to get anyone from southern Alberta to come to take a look.
“At the time, sort of in the ’70s and ’80s, paleontology in northern Alberta was virtually unknown,” said Bamforth.
When paleontogists eventually got to the site, Bamforth said, they learned “it’s actually one of the densest dinosaur bonebeds in North America.”
“It contains about 100 to 300 bones per square metre,” she said.
Paleontologists have been at the site sporadically ever since, combing through bones belonging to turtles, dinosaurs and lizards. Sixteen years ago, they discovered a large skull of an approximately 30-year-old Pachyrhinosaurus, which is now at the museum.
About a year ago, they found the second adult: Big Sam.
Bamforth said both dinosaurs are believed to have been the elders in the herd.
“Their distinguishing feature is that, instead of having a horn on their nose like a triceratops, they had this big, bony bump called a boss. And they have big, bony bumps over their eyes as well,” she said.
“It makes them look a little strange. It’s the one dinosaur that if you find it, it’s the only possible thing it can be.”
The genders of the two adults are unknown.
Bamforth said the extraction was difficult because Big Sam was intertwined in a cluster of about 300 other bones.
The skull was found upside down, “as if the animal was lying on its back,” but was well preserved, she said.
She said the excavation process involved putting plaster on the skull and wooden planks around if for stability. From there, it was lifted out — very carefully — with a crane, and was to be shipped on a trolley to the museum for study.
“I have extracted skulls in the past. This is probably the biggest one I’ve ever done though,” said Bamforth.
“It’s pretty exciting.”
This report by The Canadian Press was first published Sept. 25, 2024.
TEL AVIV, Israel (AP) — A rare Bronze-Era jar accidentally smashed by a 4-year-old visiting a museum was back on display Wednesday after restoration experts were able to carefully piece the artifact back together.
Last month, a family from northern Israel was visiting the museum when their youngest son tipped over the jar, which smashed into pieces.
Alex Geller, the boy’s father, said his son — the youngest of three — is exceptionally curious, and that the moment he heard the crash, “please let that not be my child” was the first thought that raced through his head.
The jar has been on display at the Hecht Museum in Haifa for 35 years. It was one of the only containers of its size and from that period still complete when it was discovered.
The Bronze Age jar is one of many artifacts exhibited out in the open, part of the Hecht Museum’s vision of letting visitors explore history without glass barriers, said Inbal Rivlin, the director of the museum, which is associated with Haifa University in northern Israel.
It was likely used to hold wine or oil, and dates back to between 2200 and 1500 B.C.
Rivlin and the museum decided to turn the moment, which captured international attention, into a teaching moment, inviting the Geller family back for a special visit and hands-on activity to illustrate the restoration process.
Rivlin added that the incident provided a welcome distraction from the ongoing war in Gaza. “Well, he’s just a kid. So I think that somehow it touches the heart of the people in Israel and around the world,“ said Rivlin.
Roee Shafir, a restoration expert at the museum, said the repairs would be fairly simple, as the pieces were from a single, complete jar. Archaeologists often face the more daunting task of sifting through piles of shards from multiple objects and trying to piece them together.
Experts used 3D technology, hi-resolution videos, and special glue to painstakingly reconstruct the large jar.
Less than two weeks after it broke, the jar went back on display at the museum. The gluing process left small hairline cracks, and a few pieces are missing, but the jar’s impressive size remains.
The only noticeable difference in the exhibit was a new sign reading “please don’t touch.”
