Recommendations to encourage the publication of null results in journals
Concerns about publishing null results in a journal broadly fit into four categories:
- Concerns about negative bias
- A lack of understanding where and/or how to submit
- Concerns about whether submissions would be accepted for publication
- A lack of support
To tackle the gap between intention and action previously described, recommendations have been suggested below based on the survey data, identifying relevant stakeholders who could take ownership for each solution. Some recommendations are repeated as they have the potential to solve more than one challenge.
All stakeholder groups
Challenges:
Tackling negative bias — whether from peers, researchers feeling embarrassed, or journals unlikely to accept such submissions.
-and-
Practical concerns – time/effort needed, co-author approval, lack of funding for open access (article processing charges).
Recommendations:
Improve perception of null results through educational and awareness campaigns. Highlight benefits and success stories or case studies, with a focus on messaging around reducing research waste.
Use the right terminology when talking about null results to avoid negative connotations.
Challenge:
Insufficient awareness of how and where to publish.
Recommendation:
Provide guidelines or tips on where and how to publish.
Challenge:
Perceptions that publishing null results has a negative impact on career.
Recommendations:
Move forward conversations around the reform of current research assessment systems to reward researchers for good publication practices.
Improve perception of null results through educational and awareness campaigns. Highlight benefits and success stories or case studies.
Challenge:
Insufficient incentive to publish null results.
Recommendation:
Move forwards conversations around the reform of the current research assessment systems.
Challenge:
Insufficient philosophical buy-in.
Recommendation:
Improve perception of null results through educational and awareness campaigns. Highlight benefits and success stories or case studies.
Institutions
In addition to the list of potential solutions suggested for all stakeholders, institutions could help solve the below challenge.
Challenge:
Practical concerns – time/effort needed, co-author approval, lack of funding for open access (article processing charges).
Recommendation:
Provide funding for the open access publication of null results.
Funders
In addition to the list of potential solutions suggested for all stakeholders, funders could help solve the below challenge.
Challenge:
Practical concerns – time/effort needed, co-author approval, lack of funding for open access (article processing charges).
Recommendation:
Provide funding for the open access publication of null results.
Publishers
In addition to the list of potential solutions suggested for all stakeholders, publishers could help address the below challenges.
Challenge:
Tackling negative bias — whether from peers, researchers feeling embarrassed, or journals unlikely to accept such submissions.
Recommendations:
Clearly share null results policy on journal homepages.
Advertise acceptance rate for articles based on null results (if supporting data is found).
Consider introducing journal sections dedicated to null results.
Provide policy guidance to journal editors and peer reviewers on null results.
Challenge:
Insufficient awareness of how and where to publish
Recommendations:
Clearly share null results policy on journal homepages.
Recommendations for targeting
Where relevant, targeting specific subject areas, regions or seniority of researcher may be more effective than applying a uniform approach across the board.
By subject area
Medicine, Biology, and the Social Sciences are likely to be easier to encourage article submissions based on null results, since these researchers show the greatest philosophical and behavioural buy-in to sharing null results. However, the perception of negative publication bias and limited awareness of which journals consider null results persists. Clarifying policies on journal pages should make this clearer, so that a home for null results is easy to find.
By region
Researchers in Europe, Australia and North America have the greatest philosophical and behavioural buy-in to sharing null results, but the majority hold at least one concern when it comes to publishing those results (lower likelihood of acceptance, and not knowing which journals to submit to). Ensuring that these researchers are aware of journal policies can mitigate against these concerns.
Researchers in Africa and Asia – particularly Japan - have the lowest philosophical and behavioural buy-in to sharing null results. Running educational and awareness campaigns, highlighting the benefits of publishing null results and including success stories or case studies should improve these researchers’ perception of null results
By seniority
Early career researchers may require greater encouragement to consider publishing null results before they are established in their career or confident in their research (as to defend their methodology or feel able to explain why they generated null results). Concerns regarding how they will be viewed by peers can be addressed by illustrating how common null results are and that it is a natural part of the research process which should be associated with transparency and rigour, rather than failure. Early career researchers could benefit from examples of how publishing null results can benefit one’s career, especially early on, as well as being given access to funding for publishing null results, and having a research assessment framework that acknowledges the value of both null and positive publications.
Built with Shorthand