Science, the discipline in which we should find the harshest skepticism, the most pin-sharp rationality, and the hardest-headed empiricism, has become home to a dizzying array of incompetence, delusion, lies, and self-deception.
[---]
That moral case - that making errors in science is much more than just an academic matter, because of the harm it can cause - applies similarly to fields of research that directly sacrifice lives. I'm referring, of course, to research on non-human animals, where the subjects are often 'euthanized' - that is killed - as part of the experiment (for example, to examine their brains after a new drug has been administered). This kind of research is usually strictly regulated by government agencies since virtually everyone agrees it would be immoral to kill lab animals, or even just to cause them to suffice, for no good scientific reason. So animal studies don't just carry the usual of trying to produce accurate, replicable results without wasting resources. They also have an additional responsibility: ensuring that errors in their design and analysis don't render pointless pain and death that they inevitably cause. Unfortunately, a considerable proportion - by some measures, a majority - of animal research studies fail this test.
Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth by Stuart Ritchie.
There is a common misconception that religion, socialism et al., are the only sources that unleash the pain, destruction, and death. No question they caused and still causing immense pain, destruction, and death but we conveniently forget the only common factor amongst these is humans. Science is no exception; last time I checked scientists are humans.
Unless, we as a society at a meta-level change the incentives from money and fame to morality - this is not going to change.
The road to hell is paved with good intentions.
- Henry G. Bohn, A Hand-Book of Proverbs
Ultra-Hyped Fields:
Stem cells, genetics, epigenetics, machine learning, and brain imagining; for the past few years, a strong contender for the 'most hyped' award has been research on the microbiome - the countless millions of microbes that inhabit our bodies.
Perverse Incentives:
Because studies reporting positive, flashy, novel, newsworthy results are rewarded so much more than others, scientists are incentivized to generate them to the detriment of everything else. To convenience the reviewers and editors that their papers really do have all those qualities, too many of them end up bending or breaking the rules (of Mertonian norms of universalism, commonality, disinterestedness, and organized skepticism).
[---]
The system incentives scientists not to practice science, but simply to meet its own perverse demands. The incentives are at the root of so many of the dubious practices that undermine our research.
Fixing Science:(addresses symptoms but not causes - which is basic human nature)
Some of the proposed solutions for the corresponding issues:
Fake data and negligence: Algorithms (services such as GRIM and Statcheck) do and could help with these issues.
Novelty Bias: Journals should also publish null results and journals making the authors responsible for publishing further work checking whether it replicates.
Statistical bias and p-hacking: Cannot remove them completely since its scary to move towards a subjective metric (current issue with nutrition studies), use more of the Bayesian approach (although the prior is subjective), and other metrics such as multi-verse analysis (if we imagine infinite parallel universes, in each of which you ran the analysis slightly differently, in what proportion of them would you find the complete opposite? Would all these analyses converge to the same overall result?)
Preprints and Pre-registration take out a lot of issues. Registering a study involves positing a public, time-stamped document online that details what the researchers are planning to do, in advance of collecting any data. It allows us to see the hypothesis the researchers intended to test, so we can check if any of them were switched mid-study. This is all about transparency.
Replication crisis: Team Science - Large Collaborative Projects such as 'Plan S', and Open Access with funding from government and major funders, help force changes in research practice. These large-scale projects can directly address the applicability of their respective fields and because the results are being shared around a larger community of usually very opinionated scientists, they can also, in theory, act as a check on the biases of any individual scientist.
Just as publishing more null results and replication studies is a more dependable way to build our knowledge, becoming more aware of the uncertain and preliminary nature of research is, in the long run, a better way to appreciate science fully. Let's work to resist our neophiliac, magpie-like focus on shiny research findings, and instead learn to value results that are solid, even if they're less immediately thrilling. In other words, let's Make Science Boring Again.
[---]
Treating each study as a tentative step towards an answer, rather than as the answer itself.
No comments:
Post a Comment