View source for Meta-research

'''Meta-research''' is a recent field of research that studies research practices with the aim of finding evidence-based improvements.<ref name="IoannidisFanelli2015">{{cite journal|last1=Ioannidis|first1=John P. A.|last2=Fanelli|first2=Daniele|last3=Dunne|first3=Debbie Drake|last4=Goodman|first4=Steven N.|title=Meta-research: Evaluation and Improvement of Research Methods and Practices|journal=PLOS Biology|volume=13|issue=10|year=2015|pages=e1002264|issn=1545-7885|doi=10.1371/journal.pbio.1002264}}</ref><ref name="Ioannidis2018">{{cite journal|last1=Ioannidis|first1=John P. A.|title=Meta-research: Why research on research matters|journal=PLOS Biology|volume=16|issue=3|year=2018|pages=e2005468|issn=1545-7885|doi=10.1371/journal.pbio.2005468}}</ref>
It is also known as "research on research" or "the science of science" as it uses research methods to study how research is done and where improvements can be made.
It covers all fields of scientific research (including health and medical research) and has been described as "taking a bird’s eye view of science".<ref name="IoannidisFanelli2015" />
It aims to improve scientific practice as summed up by [https://www.youtube.com/watch?v=Zhvqd78J0ns John Ioannidis], "Science is the best thing that has happened to human beings [...] but we can do it better".

==History==
Meta-research has grown as a reaction to the [[Replication crisis|replication crisis]] and concerns about waste in research.<ref>{{cite journal|title=Researching the researchers|journal=Nature Genetics|volume=46|issue=5|year=2014|pages=417–417|issn=1061-4036|doi=10.1038/ng.2972}}</ref>
The earliest meta-research paper was published in 1966 and examined the statistical methods of 295 papers in ten high-profile medical journals in 1964, and found that, "in almost 73% of the reports read [...] conclusions were drawn when the justification for these conclusions was invalid."<ref name="Schor1966">{{cite journal|last1=Schor|first1=Stanley|title=Statistical Evaluation of Medical Journal Manuscripts|journal=JAMA: The Journal of the American Medical Association|volume=195|issue=13|year=1966|pages=1123|issn=0098-7484|doi=10.1001/jama.1966.03100130097026}}</ref>

In health and medical research concerns have been raised about waste due to [[Publication bias|publication bias]], inadequate research reporting, and poor study design, such as inadequate blinding.
It is estimated that 85% of the worldwide research budget in health and medical research is currently wasted.<ref name="ChalmersGlasziou2009">{{cite journal|last1=Chalmers|first1=Iain|last2=Glasziou|first2=Paul|title=Avoidable waste in the production and reporting of research evidence|journal=The Lancet|volume=374|issue=9683|year=2009|pages=86–89|issn=0140-6736|doi=10.1016/S0140-6736(09)60329-9}}</ref>
The 85% figure is supported by multiple empirical studies in a range of fields that have attempted to reproduce published peer reviewed research and failed on 75% to 90% of occasions.<ref name="BegleyIoannidis2014">{{cite journal|last1=Begley|first1=C. G.|last2=Ioannidis|first2=J. P. A.|title=Reproducibility in Science: Improving the Standard for Basic and Preclinical Research|journal=Circulation Research|volume=116|issue=1|year=2014|pages=116–126|issn=0009-7330|doi=10.1161/CIRCRESAHA.114.303819|pmid=25552691}}</ref>

Many high-profile scientific publishers are interested in meta-research and improving the quality of their publications. Many of the concerns about waste in health and medical research were described in the 2014 Lancet special issue on [http://www.thelancet.com/series/research 'Research: increasing value, reducing waste']. There is an ongoing special section in Nature on [http://www.nature.com/news/reproducibility-1.17552 "Challenges in irreproducible research"] with 12 editorials (as at August 2017). Science has had an editorial <ref name="Buck2015">{{cite journal|last1=Buck|first1=S.|title=Solving reproducibility|journal=Science|volume=348|issue=6242|year=2015|pages=1403–1403|issn=0036-8075|doi=10.1126/science.aac8041}}</ref>, a policy forum <ref name="AlbertsCicerone2015">{{cite journal|last1=Alberts|first1=B.|last2=Cicerone|first2=R. J.|last3=Fienberg|first3=S. E.|last4=Kamb|first4=A.|last5=McNutt|first5=M.|last6=Nerem|first6=R. M.|last7=Schekman|first7=R.|last8=Shiffrin|first8=R.|last9=Stodden|first9=V.|last10=Suresh|first10=S.|last11=Zuber|first11=M. T.|last12=Pope|first12=B. K.|last13=Jamieson|first13=K. H.|title=Self-correction in science at work|journal=Science|volume=348|issue=6242|year=2015|pages=1420–1422|issn=0036-8075|doi=10.1126/science.aab3847|pmid=26113701}}</ref> and a special issue <ref name="Enserink2018">{{cite journal|last1=Enserink|first1=Martin|title=Research on research|journal=Science|volume=361|issue=6408|year=2018|pages=1178–1179|issn=0036-8075|doi=10.1126/science.361.6408.1178}}</ref> on meta-research and the problems with reproducibility. PLOS ONE launched a [http://blogs.plos.org/everyone/2012/08/14/plos-one-launches-reproducibility-initiative/ Reproducibility Initiative] in 2012. [[Biology:PLOS Biology|PLOS Biology]] included a section for papers on meta-research in 2016.<ref name="KoustaFerguson2016">{{cite journal|last1=Kousta|first1=Stavroula|last2=Ferguson|first2=Christine|last3=Ganley|first3=Emma|title=Meta-Research: Broadening the Scope of PLOS Biology|journal=PLOS Biology|volume=14|issue=1|year=2016|pages=e1002334|issn=1545-7885|doi=10.1371/journal.pbio.1002334}}</ref> [[Company:BioMed Central|Biomed Central]] [https://blogs.biomedcentral.com/bmcblog/2015/07/23/reproducibility-going/ introduced a minimum standards of reporting checklist] to four titles in 2015.

[[File:Science_magazine_cover_Sep_2018.gif|thumb|right|Illustration of scientists studying scientists on the front cover of the Science special issue on meta-research]]

In the media there have been articles on the flaws in science in [https://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble The Economist], [https://www.nytimes.com/2014/01/21/science/new-truths-that-only-one-can-see.html New York Times],  [https://www.theatlantic.com/science/archive/2015/08/psychology-studies-reliability-reproducability-nosek/402466/ The Atlantic] and [https://www.theguardian.com/science/2017/feb/01/high-tech-war-on-science The Guardian].

The first international conference in the broad area of meta-research was the [http://rewardalliance.net/research-wasteequator-conference Research Waste/EQUATOR conference] in Edinburgh in 2015; the first international conference on peer review was the Peer Review Congress in 1989.<ref name="Rennie1990">{{cite journal|last1=Rennie|first1=Drummond|title=Editorial Peer Review in Biomedical Publication|journal=JAMA|volume=263|issue=10|year=1990|pages=1317|issn=0098-7484|doi=10.1001/jama.1990.03440100011001}}</ref> The first journal specifically targeting meta-research was [http://researchintegrityjournal.biomedcentral.com/ Research Integrity and Peer Review] launched in 2016. The journal's opening editorial called for, "research that will increase our understanding and suggest potential solutions to issues related to peer review, study reporting, and research and publication ethics".<ref name="HarrimanKowalczuk2016">{{cite journal|last1=Harriman|first1=Stephanie L.|last2=Kowalczuk|first2=Maria K.|last3=Simera|first3=Iveta|last4=Wager|first4=Elizabeth|title=A new forum for research on research integrity and peer review|journal=Research Integrity and Peer Review|volume=1|issue=1|year=2016|issn=2058-8615|doi=10.1186/s41073-016-0010-y}}</ref>

==Meta-research centres==
* Meta-Research Innovation Center at Stanford (METRICS) (co-directors John Ioannidis and Steven N. Goodman)
* [[Organization:Center for Open Science|Center for Open Science]] (director Brian Nosek)
* [[Organization:Meta-Research Center at Tilburg University|Meta-Research Center at Tilburg University]]
* [http://miror-ejd.eu/ Projet MiRoR] Methods in Research on Research
* [http://www.ohri.ca/journalology/ Centre for Journalology]
* [https://ecrf1.clinicaltrials.ed.ac.uk/enki/ European Network for Knowledge Impact]
* [https://www.bihealth.org/en/research/transforming-biomedical-research/quest-opening/ The Center for Transforming Biomedical Research of the Berlin Institute of Health]

==See also==
* [[EQUATOR Network]]
* [[Reproducibility]]
* [[Philosophy:Science of science policy|Science of science policy]]
* [[Scientific method]]
* [[Scientometrics]]

==Further reading==
* R. Harris, ''Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hopes, and Wastes Billions'', Basic Books, 2017 <ref name="Google Books 2017">{{cite web | title=Rigor Mortis | website=Google Books | date=2017-03-09 | url=https://books.google.com/books/about/Rigor_Mortis.html?id=lk5IDgAAQBAJ | accessdate=2017-04-09}}</ref>

==References==
{{Reflist|2}}

[[Category:Research methods]]

{{Sourceattribution|Meta-research|1}}