Challenges in publishing null results in a journal
Despite strong support among researchers for sharing null results as outlined in Section 1, many still hesitate to do so—especially through journal publication, as seen in Section 2. What is preventing them from doing so?
This section explores the key barriers to publishing null results, including researchers’ perceived concerns as well as negative experiences they have encountered—either during the submission process or as a consequence of publication.
3.1 What are the barriers to publishing null results in a journal?
From those who have generated null results and agree they are important to share, but have not yet shared or tried to share them (n=1,489), the top two concerns about publishing null results in a journal were: ‘null results are unlikely to be accepted for publication by journals' (69%), and ‘I don’t know which journal/s to submit null results to' (52%).
Figure 8: Concerns about publishing null results in a journal, from those who have generated null results and agree they are important to share, but have not yet tried to.
Table 8: Concerns about publishing null results in a journal, from those who have generated null results and agree they are important to share, but haven’t yet tried to.
|
Which of the following concerns, if any, do you have about publishing null results in a journal? |
|||
|---|---|---|---|
|
net important ("Very important" + "Somewhat important") |
|||
|
Null results are unlikely to be accepted for publication by journals |
69% |
98% |
net |
|
I don’t know which journal/s to submit null results to |
52% |
||
|
Null results are less likely to be cited |
35% |
||
|
There is no incentive to share null results |
32% |
||
|
I did not know it was possible to publish null results in a journal |
37% |
||
|
My co-authors / research group would not wish to share null results |
25% |
||
|
I don’t know how to submit null results to a journal |
29% |
||
|
It is too time consuming |
20% |
||
|
Concerns I may be viewed negatively by my peers |
21% |
||
|
Concerns my institution or funder would not cover the cost of publishing null results |
19% |
||
|
Concerns regarding feeling embarrassed / like a failure |
16% |
||
|
Concerns it would have a negative impact on my career, e.g. lower H-index |
13% |
||
|
My institution or funder does not permit me to share null results |
9% |
||
|
I am not convinced sharing null results is very useful for the research community |
4% |
||
|
Not applicable – I do not have any concerns regarding publishing null results in a journal |
1% |
||
Sample: All responses from population who have generated null results, but not shared or tried to share them, and think sharing null results is “very” or “somewhat important” (n=1,489); Unweighted.
3.1.1 Perceived bias and reputational risk
Perceived biases can present in different ways. For example, survey respondents reported a negative connotation to using the term ‘negative results’.
“I will not say these are negative results. If we did our experiments properly without any mistakes then [the] data that we get still describe a new result.”
More about the different terms used to describe ‘null results’ by survey respondents can be found in Appendix 1.
There were also numerous fears that publishing null results would have a negative impact on their career and/or reputation. These concerns align with previous findings that best practices and research assessment’s reliance on quantitative metrics, such as citations and Journal Impact Factors, do not incentivise the publication of null results.10
On a more personal level, 21% were concerned that they ‘may be viewed negatively by [...] peers’ 16% felt ‘embarrassed/like a failure’ and 13% expressed concerns about publishing null results as potentially having a ‘negative impact on my career’.
3.1.2 Confusion about where and how to submit
For those who have generated null results and think that sharing them is important, but who did not share or try to share them (n=1,489), the second biggest concern is not knowing which journal to submit null results to (52%). Additionally, 29% also responded that they do not know how to submit null results to a journal.
Across all survey respondents (n=11,069), only 15% of researchers were aware of a journal that encourages publication of null results.
3.1.3 Likelihood of being unsuccessful
Convincing researchers that it is important to share null results is one key component of the challenge in encouraging null-results based submissions – researchers will need their publishing-related concerns, such as the likelihood of acceptance, to be addressed. 82% of researchers agreed with the statement that ‘null results are less likely to be accepted for publication at a journal’ as being their greatest concern when submitting to a journal (n=11,069).
For those who had previously submitted a manuscript based (solely or mostly) on null results to a journal (n=2,119), more than half (58%) reported that their submission had been accepted for publication, 37% had been rejected, and 5% of respondents could not remember the final decision.
Figure 9: The ultimate acceptance rate for manuscripts including null results
Table 9: The ultimate acceptance rate for manuscripts including null results
|
Please think of the most recent occasion you submitted a manuscript based (solely or mostly) on null results to a journal. What was the final decision given to this manuscript? |
|||
|---|---|---|---|
|
It was rejected - before peer review |
18% |
37% |
net |
|
It was rejected - after peer review |
15% |
||
|
It was rejected - but I can’t remember at which stage |
4% |
||
|
It was accepted for publication |
58% |
||
|
I can’t remember the final decision |
5% |
||
Sample: All responses from population who have “Submitted a manuscript based (solely or mostly) on null results to a journal” (n=2,119); Unweighted.
Of those rejected (n=777), 93% think that the inclusion of null results played a role in the publisher’s decision. Authors who were rejected before peer review were the most likely to feel that the inclusion of null results played a major role in the rejection decision.
Because of the perceived bias not just by researchers but by journals, there is scepticism about the value of investing time into publishing null results. 26% of survey respondents believe that ‘the benefits of publishing an article based on null results are outweighed by the time and effort required to write it’, whilst 20% consider it ‘too time consuming’ (n=1,489). This sentiment may stem from the expectation that multiple journal submissions are often required before a manuscript is accepted.
Some respondents shared experiences in the open text comments to this effect, which suggests that the initial acceptance rate could be lower if based solely on the first submission. However, the survey did not investigate how many submission attempts are typically required for studies with positive results, leaving no direct basis for comparison.
“The negative effect to our group was that it took significantly more effort to publish the null result. We went through three submit/reject cycles before finding a journal that would accept it, versus typically one submission for a positive result. There is a larger negative effect for the field. My perception is that our first-choice journal would have reached the broadest audience with interest in the problem we identified. Instead, colleagues are unaware of our research in the fourth-choice journal, and the idea we disproved is continually described as plausible but untested.”
“It was challenging to publish; it was almost published first in a well-respected journal in my field, and after going through the peer-review process twice and 6 months of extra work, […] was rejected. Then, we went through 11 more journals and spent two years sending them to different journals until one found them interesting and promising. However, the angle of the manuscript had to be changed from completely "null results" to a slightly less negative perspective based on what we learned from it and what can be done in the future. Otherwise, we would not get it published.”
3.2 Were there any negative experiences as a result of publishing in a journal?
Amongst respondents who had published null results (n=1,228), only 20% experienced a negative outcome or outcomes, such as being viewed negatively by peers (8%), feeling like a failure (7%), or concerns it would negatively impact their career and reputation: 6% selected ‘negative impact on my career’, 5% selected ‘my citation metrics were negatively impacted’, 3% selected ‘negative impact on my reputation in the field’. 16% had not experienced any negative outcomes.
Although the overall percentage of a negative experiences is small, acknowledging and exploring these negative experiences will enable us to assess what needs to change. In order to understand more, we asked respondents to elaborate how they knew, or why they thought, these outcomes had occurred.
“[…] at a subsequent conference the audience was underwhelmed and I felt embarrassed. […] I think this is a perfectly legitimate kind of research to publish and am frankly horrified at the attitude I encountered. Reacting adversely to such results is not what science is all about. They could be a springboard to further research.”
“Science often makes us feel like failures. All the more so when, after years of careful theory work and even some preliminary evidence of the "desired" effect, a full, expensive study ends up saying, "nah, all of this is random, you recorded only noise". It can be very demotivating and depressing […]”
“Publishing our null results created some personal tension with another research group that promotes the dependent measure which repeatedly generated the null results for us.”
“Published a paper showing that a highly touted method from a famous group is unreliable for some cases. The result was loss of a grant. The paper now is well cited, but the effect on my career was entirely negative.”
These experiences could reflect the competitive nature of funding and a lack of understanding of the value of sharing null results. Raising awareness and educating researchers about the benefits of sharing null results could help to remove this stigma.
3.1.4 Lack of support
19% of respondents were concerned that their institution or funder would not cover the cost of publishing null results, and 9% reported that their institution or funder does not permit them to share null results, which points to a lack of support in publishing null results.
When asked in a separate question about the support for sharing null results that they were aware of (n=11,069):
- 55% responded that they were unaware of any support being available
- Overall, 45% were aware of at least one type of support from either their institution/organisation, their research funder, and/or journals
- 38% were aware of support from their institution such as: policies, guidelines, trainings, access to online repositories and/or preprint servers, policies and encouragement for pre-registered reports, funding, and incentives or recognition
- 21% were aware of support from their research funder such as: policies and requirements, training or guides, funding, incentives or recognition
- 17% responded that there were journals in their field encouraging publication of null results
An assessment of the accuracy of reported perceptions was beyond the scope of this paper.
Figure 10: Awareness of types of support available for sharing null results
Table 10: Awareness of types of support available for sharing null results
|
To your knowledge, what support around sharing null results is currently available to you? |
|||
|---|---|---|---|
|
There is no support that I am aware of |
55% |
||
|
From my institution/organisation: |
|||
|
Policies and guidelines encouraging the submission of null results |
15% |
38% |
net support from |
|
Providing access to online repositories and/or preprint servers to share null results |
13% |
||
|
Training or guidance on how researchers can share null results |
13% |
||
|
Policies and encouragement for the pre-registration of studies |
12% |
||
|
Funding to publish null results in journals |
11% |
||
|
Incentives or recognition to researchers who publish null results (e.g., as part of researcher assessments) |
9% |
||
|
Funding for replication or validation studies that could give null results |
7% |
||
|
From my research funder: |
|||
|
Publication policies and requirements to publish null results and/or all research data |
16% |
32% |
net support from research funder |
|
Training or guides on how researchers can share null results |
13% |
||
|
Funding to publish null results in journals |
13% |
||
|
Incentives or recognition to researchers who publish null results |
9% |
||
|
Funding for replication or validation studies that could give null results |
9% |
||
|
From other sources: |
|||
|
Journals in my research field that encourage publication of null results |
17% |
||
Sample: All responses (n=11,069); Unweighted.
There is an indication from survey responses that providing support translates into action. Compared to those unaware of any support from their institution or funder, researchers who were aware of at least one of the listed types of support (72%) were more likely to have shared - or tried to share - their null results (if applicable), including submitting a manuscript based on their null results to a journal (34%).
For further context, a chart presenting an overview of the survey respondents’ publication behaviour can be found in Figure 13, Appendix A.2.2.
References
(Link open in a new window)
10 Springer Nature. (2025). The state of research assessment: Researcher perspectives on evaluation practices. https://stories.springernature.com/state-of-research-assessment
Built with Shorthand