Home  |  About us  |  Editorial board  |  Ahead of print  | Current issue  |  Archives  |  Submit article  |  Instructions |  Search  |   Subscribe  |  Advertise  |  Contacts  |  Login 
  Users Online: 1497Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents      
RESEARCH METHODOLOGY
Year : 2014  |  Volume : 5  |  Issue : 4  |  Page : 151-153

Contradicting/negative results in clinical research: Why (do we get these)? Why not (get these published)? Where (to publish)?


1 Department of Pharmacology, Subharti Medical College, Meerut, Uttar Pradesh, India
2 Department of Prosthodontics, Subharti Dental College, Delhi-Haridwar Bypass, Meerut, Uttar Pradesh, India

Date of Web Publication11-Sep-2014

Correspondence Address:
S Kannan
Department of Pharmacology, Subharti Medical College, Delhi-Haridwar Bypass, Meerut - 250 005, Uttar Pradesh
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2229-3485.140546

Rights and Permissions
   Abstract 

Obtaining negative or contradicting results, whereas conducting a study has always been overlooked as inadequacies on the part of the researcher. Many-a-times, negative results are arrived at even after conducting the study with great care and effort. This cannot be considered, a flaw, always. Contradicting results may be arrived at because of various reasons and should be considered and published in order to arrive at a definitive result. Various journals are available which publish these contradicting results exclusively. It is the duty of the researcher to know in detail the cause and effect of these, considering the pros and cons. This article gives a bird's eye view of when, where and how to manage negative or contradicting results of a study.

Keywords: Null results, repeat studies, declining effect


How to cite this article:
Kannan S, Gowri S. Contradicting/negative results in clinical research: Why (do we get these)? Why not (get these published)? Where (to publish)?. Perspect Clin Res 2014;5:151-3

How to cite this URL:
Kannan S, Gowri S. Contradicting/negative results in clinical research: Why (do we get these)? Why not (get these published)? Where (to publish)?. Perspect Clin Res [serial online] 2014 [cited 2019 Oct 18];5:151-3. Available from: http://www.picronline.org/text.asp?2014/5/4/151/140546


   Introduction Top

"Once you replace negative thoughts with positive ones, you'll start having positive results" a famous quote by an American musician, Willie Nelson. Furthermore, Francis Bacon in The Advancement of Learning states "It is human nature for the affirmative or active to effect more than the negative or privative". Hence a few times hitting, or presence, countervails oft-times failing or absence". It is common to find evaluation of same kind of hypothesis by many researchers. The effect obtained in each of these studies may differ and the hypothesis may be accepted in some while rejected in other studies. Although adequately powered and other determining and confounding factors have been controlled, a study may show a result that contradicts the already known observations/facts. If such negative/contradicting studies are not published, the ascertained effect size remains falsified.


   Why do we get contradicting results for the same study? Top


Before a study is initiated, the scientist sets up a hypothesis in mind. Many a times, the scientist knows what he/she wants and therefore, this will influence the study results. Hence, it is prudent to confirm the effect of any intervention by repeating it in different laboratories/conditions by various scientists. When done so, the subsequent studies may either contradict or may show reduced or stronger effect size than the earlier ones. A study published in the journal nature [1] indicates that only 11% of the findings from preclinical studies carried out on anti-cancer drugs under controlled laboratory conditions were reproducible. This may either be attributed to genuine entities such as regression to the mean/declining effect or may be, pitfalls due to researchers' tendency to stick to their hypothesis and publishing only those results pertaining to, while omitting the others. This declining effect has been reported to exist in all fields of medicine, especially cardiovascular and psychiatry. [2],[3] Ioannidis [4] studied 49 trials that were cited more than 1000 times and published in journals with high impact factor and found 16% producing contradictory results, more common in nonrandomized than in randomized studies. Contradictory results are rampantly seen in genomic studies. A comprehensive analysis by Hirschhorn et al. [5] on 166 genome-wide association studies has shown that only six of them were reproducible.


   Why are these not published and so what? Top


The researcher may feel that a study with a negative result may not have good impact in their respective fields or does not help them in getting funds or promotion in their career and hence may not submit their work to any journal. A study carried out in United States [6] has confirmed that the academic competition and pressure increases the risk of scientists' bias in not publishing negative studies. The journals on the other hand may not be willing to publish a study with negative or contradicting results when compared to the earlier ones as they might not have a positive impact. Decoursey [7] argues that though there are journals that are intended to publish negative results, the journal editors usually ask explanation for the contradicting effect that was arrived as compared to previous one, which is obviously difficult to analyze and hence, it becomes difficult to publish such studies. Although, the International Committee of Medical Journal Editors (ICMJE) [8] clearly recommends that it is not obligatory for the journal editors to be under control of the comments of the peer reviewer, sometimes, the journal editor may fall prey to such process, recommending rejection of contradicting studies. [9] Not publishing the results of a study done on participants is unethical and can also be considered, a scientific misconduct. Easterbrook et al. [10] in a retrospective study from Institutional Ethics Committee records reported that the results of observational studies were less commonly published than randomized trials. Among the randomized clinical trials, nearly one-third of the results ultimately do not get published. In 1990, the proportion of papers published with positive results was found to be around 70% while in 2007, it had risen to almost 86% across different disciplines and it was almost 90% in biomedical field on the same year. [11] Further, the same study also illustrates that corresponding authors from Asian countries reported more positive results than from United States or Europe. Even in the published clinical trials, not all the results get mentioned. Of course, this is one of the main reasons for creating a central clinical trial registry platform and making it mandatory that the journals that follow ICMJE should intend to publish a clinical study only when they are registered in any such database. The results mentioned in the article should be complete and compatible with the outcome that was intended to be measured as shown in the registry. Due to the non-availability of results from these studies, a publication bias may emerge. This was rightly described by Rosenthal as "Fail safe File-drawer" problem. [12] Although the absence of these study results may affect a narrative review, more significant impact is observed in either a systematic review or a meta-analysis. Meta-analysis includes analysis of analyses and involves pooling of various studies that evaluated the same hypothesis. In such analysis, when studies of negative results are not included, the final pooled estimate will be distorted. Although tools such as "Egger's regression analysis", "Funnel Plot", "Fail-safe N", are available, none of them are shown to be perfect. [13],[14] Furthermore, it is important to include these studies in health technology assessment, where the effect and the cost of an intervention are compared with others, because, this influences the decision-making bodies such as National Institute of Health and Clinical Excellence and National Screening Committee. [15] Considering the limitation of budget allocated for conducting research studies, the availability of such negative results will defer another from repeating the same study.


   Where to publish the results of these studies? Top


In view of the importance of publishing studies with contradicting results, ICMJE has recommended journal editors to consider publishing such studies if they are genuine and scientifically valid. Recently, a few journals have emerged whose scope is to publish only those studies with contradicting results/negative results. These are Journal of Negative Results in Biomedicine, [16] Journal of Negative Results, [17] Journal of Contradicting Results in Science, [18] The all Results Journal, [19] Journal of Negative Pharmaceutical Results, [20] International Journal of Negative and Null Results, [21] Journal of Errology. [22] Apart from these journals dedicated for studies with contradicting results such as PLoS One, [23] F1000 Research [24] and Scientific Reports [25] in their publication criteria states that a study reporting negative results will be considered if their insight is useful. All of these are open access and charge for publishing the article, although F1000 research had waived these charges recently. In January 2013, as a joint venture between British Medical Journal, Center for Evidence-based Medicine in Oxford, Cochrane collaboration, Sense about Science, Dartmouth Institute, PLoS, James Lind Alliance and Bad science, an initiative called All Trials was launched. [26] This campaign urges registration and reporting of results of all the clinical trials even when it is negative or null so that bad treatment decisions or missed opportunities for good medicine or unnecessary repetition of trials will be minimized. Another initiative is from European Commission under the Seventh Framework Program called "OPEN" with an objective "To overcome failure to publish negative findings". [27] This group first attempts to identify and explore the current extent of publication bias by conducting a systematic review [28] and secondly will find their impact on health technology assessments and ways to address this issue. Schooler [29] suggests on developing an open-access repository, where a researcher lets his hypothesis and methodology to be stored before starting the study and intends to publish the results irrespective of the outcome.

To conclude, both the researcher and the journal editor should feel that studies with negative results are equally important to be published providing opportunities for more transparent disclosure of results that may impact the public health.

 
   References Top

1.Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature 2012;483:531-3.  Back to cited text no. 1
    
2.The truth wears off. Is there something wrong with the scientific method. Available from: http://www.neurofly.com/NeuroSeminar_files/2011-2.pdf. [Last accessed on 2013 Dec 04].  Back to cited text no. 2
    
3.Lee MS, Flammer AJ, Lerman A. The decline effect in cardiovascular medicine: Is the effect of cardiovascular medicine and stent on cardiovascular events decline over the years? Korean Circ J 2013;43:443-52.  Back to cited text no. 3
    
4.Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA 2005;294:218-28.  Back to cited text no. 4
[PUBMED]    
5.Hirschhorn JN, Lohmueller K, Byrne E, Hirschhorn K. A comprehensive review of genetic association studies. Genet Med 2002;4:45-61.  Back to cited text no. 5
    
6.Fanelli D. Do pressures to publish increase scientists' bias? An empirical support from US States Data. PLoS One 2010;5:e10271.  Back to cited text no. 6
[PUBMED]    
7.DeCoursey TE. It's difficult to publish contradictory findings. Nature 2006;439:784.  Back to cited text no. 7
    
8.International Committee of Medical Journal Editors. Available from: http://www.icmje.org/. [Last accessed on 2013 Dec 04].  Back to cited text no. 8
    
9.Giles J. Journals submit to scrutiny of their peer-review process. Nature 2006;439:252.  Back to cited text no. 9
[PUBMED]    
10.Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337:867-72.  Back to cited text no. 10
    
11.Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics 2012;90:891-904.  Back to cited text no. 11
    
12.Rosenthal R. The file drawer problem and tolerance for null results. Psychol Bull 1979;86:638-41.  Back to cited text no. 12
    
13.Fail-safe N. Available from: http://www.handbook.cochrane.org/chapter_10/10_4_4_3_fail_safe_n.htm. [Last accessed on 2013 Dec 04].  Back to cited text no. 13
    
14.Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006;295:676-80.  Back to cited text no. 14
    
15.Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: An updated review of related biases. Health Technol Assess 2010;14:iii, ix-xi, 1-193.  Back to cited text no. 15
    
16.Journal of Negative Results in Biomedicine. Available from: http://www.jnrbm.com/. [Last accessed on 2013 Nov 30].  Back to cited text no. 16
    
17.Journal of Negative Results. Available from: http://www.jnr-eeb.org/index.php/jnr. [Last accessed on 2013 Nov 30].  Back to cited text no. 17
    
18.Journal of Contradicting Results in Science. Available from: http://www.jcrsci.org/. [Last accessed on 2013 Nov 30].  Back to cited text no. 18
    
19.The all Results Journal. Available from: http://www.arjournals.com/ojs/. [Last accessed on 2013 Nov 30].  Back to cited text no. 19
    
20.Journal of Negative Pharmaceutical Results. Available from: http://www.pnrjournal.com/aboutus.asp. [Last accessed on 2013 Nov 30].  Back to cited text no. 20
    
21.International Journal of Negative and Null Results. Available from: http://www.journalnetwork.org/journals/science-general/negative-null-results/negative-null-results.html. [Last accessed on 2013 Nov 30].  Back to cited text no. 21
    
22.Journal of Errology. Available from: http://www.bioflukes.com/JoE. [Last accessed on 2013 Nov 30].  Back to cited text no. 22
    
23.PLoS One. Available from: http://www.plosone.org/static/publication. [Last accessed on 2013 Nov 30].  Back to cited text no. 23
    
24.F1000 Research. Available from: http://www.f1000research.com/. [Last accessed on 2013 Nov 30].  Back to cited text no. 24
    
25.Scientific Reports. Available from: http://www.nature.com/srep/. [Last accessed on 2013 Nov 30].  Back to cited text no. 25
    
26.All trials registered, all trials reported. Available from: http://www.alltrials.net/. [Last accessed on 2013 Nov 30].  Back to cited text no. 26
    
27.OPEN to overcome failure to publish negative findings. Available from: http://www.open-project.eu/. [Last accessed on 2013 Dec 04].  Back to cited text no. 27
    
28.Mueller KF, Meerpohl JJ, Briel M, Antes G, von Elm E, Lang B, et al. Detecting, quantifying and adjusting for publication bias in meta-analyses: Protocol of a systematic review on methods. Syst Rev 2013;2:60.  Back to cited text no. 28
    
29.Schooler J. Unpublished results hide the decline effect. Nature 2011;470:437.  Back to cited text no. 29
[PUBMED]    



This article has been cited by
1 Publishing Negative Results to Optimize the Biopharmaceutical Innovation
Shuangsheng Yan
MOJ Cell Science & Report. 2017; 4(4)
[Pubmed] | [DOI]



 

Top
  
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
    Why do we get co...
    Why are these no...
    Where to publish...
    References

 Article Access Statistics
    Viewed2051    
    Printed55    
    Emailed1    
    PDF Downloaded969    
    Comments [Add]    
    Cited by others 1    

Recommend this journal