Appendices

A.1 Terminology 

The definition of ‘null results’ used throughout the survey was shared upfront. 

97% of 12,954 journal authors surveyed self-reportedly understood the definition provided, however when asked about the terms that they primarily use themselves, ‘null results’ was only used by 24% of respondents (n=11,069).  

‘Negative results’ and ‘inconclusive results’ were the top two terms used by respondents (33% and 32% respectively), with 8% using another term (such as ‘non-significant results’), and 3% not having heard of the concept before (n=11,069).  

In addition to there being wide variation in the terminology used, respondents noted that it can change depending on the circumstances, with a variation also noted by seniority, discipline, and region. Survey respondents also reported a negative connotation to using the term ‘negative results’. 

A.2 Methodology 

A.2.1 Procedure 

Springer Nature conducted a survey in order to gain insights into researcher attitudes towards and experience of sharing research consisting of mostly or solely null results – including: 

1. Attitudes towards null results 
2. Experience of using or sharing null results, particularly via journal publication 
3. Current barriers to publishing null results in journals 
4. Key players and initiatives in encouraging the journal publication of null results 

A mixture of quantitative (closed) and qualitative (open text) questions on these four topics were posed to researchers in Springer Nature’s audience who have published and expect to publish again in peer reviewed journals (at any publisher, not limited to Springer Nature). The survey questions freely available through the Figshare repository – see Appendix 4. 

The survey was programmed and hosted online on Qualtrics, a third-party survey platform provider, and used Springer Nature branding. The survey was available in English, German, French, Chinese (Simplified), and Japanese to encourage a global sample. 

The survey was live for two months, from 6th November 2024 to 6th January 2025. It was distributed via 17 campaigns across numerous channels including email (to subscribers on Springer Nature marketing lists and members of the Market Intelligence team’s research panel), social media (LinkedIn, Facebook, X, WeChat, Weibo, Medsi.cn, Sciencenet.cn, Bilibili, Zhihu); blog posts (on Springer Nature’s Researcher Community and Editor Community sites); and site intercepts (on springer.com, springeropen.com, nature.com, and bmc.com) via official Springer Nature channels.  

All respondents were invited to enter a prize draw to win one of five USD$100 Visa® Virtual Gift Card (or regionally available virtual gift card to the value of USD$100USD). Survey responses were not associated with prize draw entries as to maintain anonymity, so answers did not affect the chances of winning the prize draw. The incentive was purchased and distributed via Tremendous, a third-party gift card & rewards platform for businesses. 

A.2.2 Sample 

Due to the distribution methods, the survey sample is limited to Springer Nature’s audience (its website users; news, journal and book readers; social media followers; marketing list contacts) and the networks of any persons who may have shared the survey. The survey findings should be interpreted with this in mind. 

It should also be noted that there is the possibility of self-selection bias in the survey sample, as those who are aware of and feel strongly about null results may be more likely to complete the survey. 

Only responses that met all of the following inclusion criteria were included in the final survey sample: i) qualify for the survey (have self-reportedly previously published an article in a peer-reviewed journal and expect to publish an article in peer-reviewed journal in the future); ii) confirmation to have read and understood the provided definition of ‘null results’; iii) have completed the entire survey; iv) not identified as a duplicate response; v) not flagged as likely fraudulent (using the fields ‘Q_RecaptchaScore’, ‘Q_RelevantIDDuplicate’, ‘Q_RelevantIDFraudScore’, and ‘RelevantID Fraud check’ automatically generated by Qualtrics).  

In total, 22,612 respondents opened the survey and 20,759 started the survey (answering at least the first survey question). Of the 20,759 responses that started the survey, a total of 9,690 were excluded from the final survey sample because they did not meet all the inclusion criteria. The total base size of respondents who met all inclusion criteria for the final survey sample was 11,069. As such, 49% of those who opened the survey, and 53% of those who started the survey, were included in the final survey sample. 

Throughout this report, the total number of respondents is reported per question, as not every respondent was eligible for every survey question. 

Figure 13: Overview of survey results 

A.2.3 Analysis 

The data was analysed using Q Research Software. The dataset was unweighted. 

The data was analysed by region, broad subject field, seniority (based on job title and year of first journal publication), and, where useful, responses to other survey questions such as those regarding attitudes toward null results or experience of sharing research consisting mostly or solely of null results. There is no differentiation between industry as trends were more significant by broad subject field.  

For cross-tabulations, column comparisons used significance levels p ≤ 0.05 and p ≤ 0.001, and False Discovery Rate (FDR) (p = 0.05) was used for multiple comparison correction. Significant differences were highlighted throughout this report. 

The anonymised raw data is freely available through the Figshare repository – see Appendix 4. 

A.3 Demographics  

The global survey received responses from respondents spanning 166 countries. 

For the purpose of analysis, countries were grouped into continents. However, a few countries with particularly high journal publication output were analysed separately, as their trends may differ significantly from regional patterns. 

Figure 14: Which country/territory are you located in?

Respondents worked across a range of broad subject fields, though those in Medicine or Biology were highly represented in the sample, comprising 26% (n=2,928) and 20% (n=2,269) of the total sample, respectively). 

Figure 15: Which of the following best describes your broad subject field?

For analysis by broad subject field, ‘Physics’ and ‘Astronomy and Planetary Science’ have been merged into ‘Physics & Astronomy’ as ‘Astronomy and Planetary Science’ had a relatively low base size and answers were similar to that of ‘Physics’ respondents’ answers.

While respondents worked in a variety of settings, the majority worked in university/college (64%, n=7,094) so the sample predominantly represents researchers in academia. 

Figure 16: Which category best describes your industry?

For the purpose of analysis, respondents’ job titles were grouped by the relative seniority of the job title. Under this categorisation, approximately one third of the total sample were mid-career researchers (34%, 3,794) and almost one third were senior career researchers (30%, n=3,290). 

Figure 17: Which of the following job titles best applies to you?

Figure 18: Which of the following job titles best applies to you?

Senior career researcher: Professor, Principal Investigator, Research Director/VP of Research, and Laboratory Director/Head
Mid-career researcher: Associate Professor, Assistant Professor, and Research Scientist 
Early career researcher: PhD or Master’s Student, Postdoc, and Technician/Research Assistant 
Physician/Clinician and Healthcare profession were not categorised and therefore excluded from analysis by seniority based on job title. 

Another way to consider seniority, with respect to a survey focused on journal publishing behaviour, is the number of years’ experience in publishing their research in journals. For the purpose of analysis, the number of years’ experience was also grouped by the relative seniority of the duration of experience. Under this categorisation, almost one half of the total sample were senior career researchers (having published their first research article in a peer-reviewed journal before 2010: 48%, n=5,265).

Figure 19: Approximately, in which year did you publish your first research article in a peer-reviewed journal?

Figure 20: Approximately, in which year did you publish your first research article in a peer-reviewed journal?

A.4 Survey questions and data

Survey questions and anonymised data are available at: https://doi.org/10.6084/m9.figshare.29459036