Is cancer biology research reproducible? The answer still isn't clear - STAT | Canada News Media
Connect with us

Science

Is cancer biology research reproducible? The answer still isn't clear – STAT

Published

 on


Replication is something of a litmus test for scientific truth, and cancer biologists at the Center for Open Science wanted to see just how many of cancer’s most influential experiments stood up to it. So, for nearly a decade, they worked their way, step-by-step, through 50 experiments from 23 studies toward an answer — but like cancer research writ large, what they found is complicated.

In two new studies published Tuesday in eLife, the center found signs of trouble: 59% of the experiments couldn’t be replicated. Experiments that were replicable had effect sizes 85% smaller on average than the original studies, suggesting the studies’ conclusions may be far dimmer than first thought.

But drawing firm judgments from these findings is tricky.

“Sometimes, it’s just really hard. We do stuff in animals, not humans. Sometimes we’re going to be wrong, and that’s OK,” said Tim Errington, a cancer biologist at the Center for Open Science, a nonprofit dedicated to improving scientific research. “But maybe we’re also tricking ourselves.”

The trouble, Errington said, is that science steams ahead, and doesn’t always pause to parse what’s a tantalizing result worth pursuing and what’s a lucky fluke. Redoing experiments and validating conclusions might tell which studies are onto something real. But replication is hard, imperfect work — and the questions about reproducibility in cancer research extend to the project itself.

“How reproducible were their experiments? That would be a question,” said Atul Butte, a computational health scientist at the University of California, San Francisco, who was not involved in the effort, but whose research was replicated by the project. “Their heart is in the right place. I’m a big fan of reproducibility. I’m just not a huge fan of how they did it.”

Butte pointed to alterations in experimental protocols during the replications, which could influence the results. The only definitive conclusion that scientists seem to agree on from the project is that making sure biology research findings are ironclad is hard — ”even very hard,” Errington said.

The project began in 2013, with the researchers selecting 53 papers published from 2010 to 2012 that had garnered a high number of citations in cancer biology. There were 193 experiments from those papers that the team hoped to replicate, and they started to reconstruct each step of the experiment from the methods sections of the papers. That was the first issue.

Lab work is a bit like baking. Without a clear recipe, it’s hard to know exactly what to do, and Errington found science is rife with incomplete experimental protocols. For example, did “biweekly” mean a drug was to be administered every two weeks or twice a week? “There were tons of experiments with next to no details,” Errington said.

These details can make or break an experiment, said Kornelia Polyak, a cancer biologist at Harvard Medical School and Dana-Farber Cancer Institute who was not involved with the project. She once tried to replicate a procedure to purify breast cancer cells with a collaborator, Mina Bissell at UC Berkeley, but the two simply could not get the experiment to work.

“We thought we were doing the same thing, and we could not get the same result. It was a very frustrating experience,” Polyak said. “So, I sent my postdoc to her lab and I said, ‘go there and do it together.’ It turns out it came down to minor details like how fast you’re stirring a flask.”

The Center for Open Science reached out to the original investigators for every study they tried to replicate, hoping to fill in any gaps, get raw data, and input on how to redo their experiments. Sometimes that worked, Errington said, but often labs just couldn’t remember how they did the work. “They couldn’t find their own stuff,” he said. “They would spend time hunting down people who did the experiments but had since left the lab.” This sometimes forced the team to give up the replication, whittling those 193 experiments down to just 50.

About a third of the time, Errington said scientists either weren’t helpful providing additional details or data or just never responded.

Looking back at the project, Errington said it often felt like a series of miscommunications, missed emails, and long, wild goose chases for data. “It’s been exhausting. We never anticipated it would take this long. It took a lot more effort than we thought it would.”

That was the case for an experiment conducted by Butte and his colleagues at UCSF, which the project tried to replicate. Fraser Tan, a scientist working on the replications, emailed UCSF’s Butte six times for help on replicating an experiment. Butte forwarded one of those emails to a co-author, but it ultimately got lost in the shuffle of other work.

“To be honest with you, those looked like spam emails. I get hundreds of these a day. I never knew how important that protocol email actually was,” said Butte, who missed an email from eLife to review the replication during his move from Stanford University to UCSF. “I never saw the protocol they proposed to reproduce our work until after all the work was done.”

It’s a classic piled-under-emails problem that can happen to anyone. It’s not that people don’t want to help, but life is messy. With so many other pressing problems that need attention, things can just fall off the radar. When it comes to scientific research, though, that might mean a complete picture of how experiments were done doesn’t get fully communicated, making it harder for research to proceed.

Ultimately, Errington’s team was able to reproduce Butte’s experiment, and, as was the case with  most of the replications, found less statistical significance. But like many of the replication experiments, the team had to change some of the methods – including a statistical method used to analyze the data. When the replication paper came out, Butte felt blindsided by the changes.

“They chose an additional statistical test that we did not do,” Butte said. “An independent statistician, Robert Tibshirani, one of the best in the world, commented, saying their process was incorrect. I chased down every credential of every author [on the reproduction] and there was not a single biostatistician on their team,” he added. “Is this reproducibility?”

Independent reviewers approved any modifications to the protocols before they were carried out, Errington said. They also consulted with independent quantitative scientists through the journal eLife’s peer review process on any statistical methods. Still, he acknowledged it’s possible that any modifications may have altered the replications’ results.

“Human biology is very hard, and we’re humans doing it. We’re not perfect, and it’s really tricky,” he said. “None of these replications invalidate or validate the original science. Maybe the original study is wrong — a false positive or false signal. The reverse may be true, too, and the replication is wrong. More than likely, they’re both true, and there’s something mundane about how we did the experiment that’s causing the difference.”

Butte agreed, adding that procedural replication, like the kind attempted by the Center for Open Science, is important. And partly thanks to the Center for Open Science’s efforts, academic journals have made strides to prevent issues in replication from occurring again, Butte said. Because scientific articles are published in online databases, publishers like Science and Nature now allow investigators to include more detailed methods and data in long supplementary files, addressing a longtime limitation in reproducibility research. Recently, the American Association for Cancer Research announced that methods sections will no longer count towards article word lengths, so researchers can wax in depth on their protocols.

Publications are also trying to create more opportunities for scientists who are interested in reproducing experiments, which are typically harder to publish in journals. AACR recently launched a new open-access journal that will consider replication study submissions. “[Replication] won’t make a career,” Errington said. “It’s not the flashy science that people want, not a positive result, because they’re redoing something. So we need to figure out how to balance that as a culture.”

“There are a lot of changes since five years ago. I think you’d have to give [the Center for Open Science] credit for that,” Butte said. “There are a lot of positives here.”

But he added, it’s not everything. Rote, perfectly identical step-by-step replication can only tell you if one experiment can be done again, not whether or not the original conclusions are truly robust, Butte said. Only investigating the same idea using several, very different experiments can tell you that. “You do the exact same experiment to get the same answer,” he said. “But these are all models anyway. We use rats and mice but, to be honest with you, we don’t care about diabetes in rats or mice. So what if you get the same answer twice?”

Instead, Butte said it’d be better to have 100 different scientists testing the same idea with 100 different models — from primates to cells in Petri dishes — and see what they can agree on. “I want to see the 60% that’s in common from all our experiments, right?” he said. “That’s the real reproducibility we should be chasing.”

Adblock test (Why?)



Source link

Continue Reading

News

Here’s how Helene and other storms dumped a whopping 40 trillion gallons of rain on the South

Published

 on

 

More than 40 trillion gallons of rain drenched the Southeast United States in the last week from Hurricane Helene and a run-of-the-mill rainstorm that sloshed in ahead of it — an unheard of amount of water that has stunned experts.

That’s enough to fill the Dallas Cowboys’ stadium 51,000 times, or Lake Tahoe just once. If it was concentrated just on the state of North Carolina that much water would be 3.5 feet deep (more than 1 meter). It’s enough to fill more than 60 million Olympic-size swimming pools.

“That’s an astronomical amount of precipitation,” said Ed Clark, head of the National Oceanic and Atmospheric Administration’s National Water Center in Tuscaloosa, Alabama. “I have not seen something in my 25 years of working at the weather service that is this geographically large of an extent and the sheer volume of water that fell from the sky.”

The flood damage from the rain is apocalyptic, meteorologists said. More than 100 people are dead, according to officials.

Private meteorologist Ryan Maue, a former NOAA chief scientist, calculated the amount of rain, using precipitation measurements made in 2.5-mile-by-2.5 mile grids as measured by satellites and ground observations. He came up with 40 trillion gallons through Sunday for the eastern United States, with 20 trillion gallons of that hitting just Georgia, Tennessee, the Carolinas and Florida from Hurricane Helene.

Clark did the calculations independently and said the 40 trillion gallon figure (151 trillion liters) is about right and, if anything, conservative. Maue said maybe 1 to 2 trillion more gallons of rain had fallen, much if it in Virginia, since his calculations.

Clark, who spends much of his work on issues of shrinking western water supplies, said to put the amount of rain in perspective, it’s more than twice the combined amount of water stored by two key Colorado River basin reservoirs: Lake Powell and Lake Mead.

Several meteorologists said this was a combination of two, maybe three storm systems. Before Helene struck, rain had fallen heavily for days because a low pressure system had “cut off” from the jet stream — which moves weather systems along west to east — and stalled over the Southeast. That funneled plenty of warm water from the Gulf of Mexico. And a storm that fell just short of named status parked along North Carolina’s Atlantic coast, dumping as much as 20 inches of rain, said North Carolina state climatologist Kathie Dello.

Then add Helene, one of the largest storms in the last couple decades and one that held plenty of rain because it was young and moved fast before it hit the Appalachians, said University of Albany hurricane expert Kristen Corbosiero.

“It was not just a perfect storm, but it was a combination of multiple storms that that led to the enormous amount of rain,” Maue said. “That collected at high elevation, we’re talking 3,000 to 6000 feet. And when you drop trillions of gallons on a mountain, that has to go down.”

The fact that these storms hit the mountains made everything worse, and not just because of runoff. The interaction between the mountains and the storm systems wrings more moisture out of the air, Clark, Maue and Corbosiero said.

North Carolina weather officials said their top measurement total was 31.33 inches in the tiny town of Busick. Mount Mitchell also got more than 2 feet of rainfall.

Before 2017’s Hurricane Harvey, “I said to our colleagues, you know, I never thought in my career that we would measure rainfall in feet,” Clark said. “And after Harvey, Florence, the more isolated events in eastern Kentucky, portions of South Dakota. We’re seeing events year in and year out where we are measuring rainfall in feet.”

Storms are getting wetter as the climate change s, said Corbosiero and Dello. A basic law of physics says the air holds nearly 4% more moisture for every degree Fahrenheit warmer (7% for every degree Celsius) and the world has warmed more than 2 degrees (1.2 degrees Celsius) since pre-industrial times.

Corbosiero said meteorologists are vigorously debating how much of Helene is due to worsening climate change and how much is random.

For Dello, the “fingerprints of climate change” were clear.

“We’ve seen tropical storm impacts in western North Carolina. But these storms are wetter and these storms are warmer. And there would have been a time when a tropical storm would have been heading toward North Carolina and would have caused some rain and some damage, but not apocalyptic destruction. ”

___

Follow AP’s climate coverage at https://apnews.com/hub/climate

___

Follow Seth Borenstein on Twitter at @borenbears

___

Associated Press climate and environmental coverage receives support from several private foundations. See more about AP’s climate initiative here. The AP is solely responsible for all content.

Source link

Continue Reading

Science

‘Big Sam’: Paleontologists unearth giant skull of Pachyrhinosaurus in Alberta

Published

 on

 

It’s a dinosaur that roamed Alberta’s badlands more than 70 million years ago, sporting a big, bumpy, bony head the size of a baby elephant.

On Wednesday, paleontologists near Grande Prairie pulled its 272-kilogram skull from the ground.

They call it “Big Sam.”

The adult Pachyrhinosaurus is the second plant-eating dinosaur to be unearthed from a dense bonebed belonging to a herd that died together on the edge of a valley that now sits 450 kilometres northwest of Edmonton.

It didn’t die alone.

“We have hundreds of juvenile bones in the bonebed, so we know that there are many babies and some adults among all of the big adults,” Emily Bamforth, a paleontologist with the nearby Philip J. Currie Dinosaur Museum, said in an interview on the way to the dig site.

She described the horned Pachyrhinosaurus as “the smaller, older cousin of the triceratops.”

“This species of dinosaur is endemic to the Grand Prairie area, so it’s found here and nowhere else in the world. They are … kind of about the size of an Indian elephant and a rhino,” she added.

The head alone, she said, is about the size of a baby elephant.

The discovery was a long time coming.

The bonebed was first discovered by a high school teacher out for a walk about 50 years ago. It took the teacher a decade to get anyone from southern Alberta to come to take a look.

“At the time, sort of in the ’70s and ’80s, paleontology in northern Alberta was virtually unknown,” said Bamforth.

When paleontogists eventually got to the site, Bamforth said, they learned “it’s actually one of the densest dinosaur bonebeds in North America.”

“It contains about 100 to 300 bones per square metre,” she said.

Paleontologists have been at the site sporadically ever since, combing through bones belonging to turtles, dinosaurs and lizards. Sixteen years ago, they discovered a large skull of an approximately 30-year-old Pachyrhinosaurus, which is now at the museum.

About a year ago, they found the second adult: Big Sam.

Bamforth said both dinosaurs are believed to have been the elders in the herd.

“Their distinguishing feature is that, instead of having a horn on their nose like a triceratops, they had this big, bony bump called a boss. And they have big, bony bumps over their eyes as well,” she said.

“It makes them look a little strange. It’s the one dinosaur that if you find it, it’s the only possible thing it can be.”

The genders of the two adults are unknown.

Bamforth said the extraction was difficult because Big Sam was intertwined in a cluster of about 300 other bones.

The skull was found upside down, “as if the animal was lying on its back,” but was well preserved, she said.

She said the excavation process involved putting plaster on the skull and wooden planks around if for stability. From there, it was lifted out — very carefully — with a crane, and was to be shipped on a trolley to the museum for study.

“I have extracted skulls in the past. This is probably the biggest one I’ve ever done though,” said Bamforth.

“It’s pretty exciting.”

This report by The Canadian Press was first published Sept. 25, 2024.

The Canadian Press. All rights reserved.

Source link

Continue Reading

News

The ancient jar smashed by a 4-year-old is back on display at an Israeli museum after repair

Published

 on

 

TEL AVIV, Israel (AP) — A rare Bronze-Era jar accidentally smashed by a 4-year-old visiting a museum was back on display Wednesday after restoration experts were able to carefully piece the artifact back together.

Last month, a family from northern Israel was visiting the museum when their youngest son tipped over the jar, which smashed into pieces.

Alex Geller, the boy’s father, said his son — the youngest of three — is exceptionally curious, and that the moment he heard the crash, “please let that not be my child” was the first thought that raced through his head.

The jar has been on display at the Hecht Museum in Haifa for 35 years. It was one of the only containers of its size and from that period still complete when it was discovered.

The Bronze Age jar is one of many artifacts exhibited out in the open, part of the Hecht Museum’s vision of letting visitors explore history without glass barriers, said Inbal Rivlin, the director of the museum, which is associated with Haifa University in northern Israel.

It was likely used to hold wine or oil, and dates back to between 2200 and 1500 B.C.

Rivlin and the museum decided to turn the moment, which captured international attention, into a teaching moment, inviting the Geller family back for a special visit and hands-on activity to illustrate the restoration process.

Rivlin added that the incident provided a welcome distraction from the ongoing war in Gaza. “Well, he’s just a kid. So I think that somehow it touches the heart of the people in Israel and around the world,“ said Rivlin.

Roee Shafir, a restoration expert at the museum, said the repairs would be fairly simple, as the pieces were from a single, complete jar. Archaeologists often face the more daunting task of sifting through piles of shards from multiple objects and trying to piece them together.

Experts used 3D technology, hi-resolution videos, and special glue to painstakingly reconstruct the large jar.

Less than two weeks after it broke, the jar went back on display at the museum. The gluing process left small hairline cracks, and a few pieces are missing, but the jar’s impressive size remains.

The only noticeable difference in the exhibit was a new sign reading “please don’t touch.”

The Canadian Press. All rights reserved.

Source link

Continue Reading

Trending

Exit mobile version