San Francisco Declaration on Research Assessment (DORA)

Summary

The “San Francisco Declaration on Research Assessment” (DORA) is a set of recommendations aimed at improving the evaluation of scientific research, developed by a group of academic journal editors during the annual meeting of the American Society for Cell Biology (ASCB).

These recommendations focus on practices related to research articles published in peer-reviewed journals.

Promoting organizations

The DORA initiative (San Francisco Declaration on Research Assessment) is supported by a wide range of research organizations, publishers, and international funders. Among its leading promoters are the Swiss National Science Foundation and UK Research and Innovation as visionary entities; and as supporters, institutions such as the American Society for Cell Biology (ASCB), The Company of Biologists, EMBO, Howard Hughes Medical Institute (HHMI), the Netherlands Research Council (NWO), the Luxembourg National Research Fund (FNR), PLOS, eLife, the Research Council of Norway, and the Wellcome Trust. Additionally, contributing organizations include Cancer Research UK, F1000Research, the Iowa State University Library, Research Ireland, and the University of Calgary. The project also receives further support from the Arcadia foundation, which funds the TARA (Tools to Advance Research Assessment) tools for the development of new policies and practices in academic evaluation.

Objectives

The “San Francisco Declaration on Research Assessment” addresses the need to improve the way funding agencies, academic institutions, and other groups evaluate scientific research. Through the development of its recommendations, it aims to enhance how the quality of scientific output is assessed.

Specifically, its objectives are:

To raise awareness about new tools and processes in research evaluation and the responsible use of metrics that promote consistency and transparency in decision-making.
To facilitate the implementation of new policies and practices in hiring, promotion, and funding decisions.
To promote research assessment reform across all academic disciplines and at a global level.
To improve equity by advocating for greater representation of researchers in the design of research evaluation practices that directly address the structural inequalities present in academia.

Beneficiaries and stakeholders

The main beneficiaries are the target audiences of the DORA recommendations, which include funding agencies, academic institutions, journals, organizations that provide metrics, and individual researchers.

The declaration is aimed at all types of organizations across any scientific discipline, encouraging them to show their support by publicly signing it.

Results

Since the publication of the “San Francisco Declaration on Research Assessment” in 2013, more than 20,000 individuals and organizations from 148 countries have joined the initiative.

In July 2021, Arcadia, a nonprofit foundation, awarded DORA a $1.2 million grant over a three-year period, which led to the creation of the TARA project (Tools to Advance Research Assessment).

This project enables DORA to identify, understand, and make visible the criteria and standards universities use when making decisions on hiring, promotion, and tenure. Based on this information, DORA develops resources and practical guides on research assessment reform targeted at academic institutions.

In collaboration with the Luxembourg National Research Fund, DORA released a short film titled “Balanced, Broad, Responsible: A Practical Guide for Research Evaluators”, accompanied by a brief document that research funders can use to promote a more holistic approach to evaluating funding proposals.

In September 2021, DORA organized a workshop for research funders in partnership with the Funding Organizations for Gender Equality Community of Practice (FORGEN CoP), aimed at optimizing the use of narrative CVs in grant evaluation and reducing bias.

As a next step, DORA is organizing a funders’ workshop in February 2022, again in collaboration with FORGEN CoP, to assess the effectiveness of narrative CVs in grant evaluation.

Starting January 1, 2022, DORA implemented several changes in its governance procedures to achieve antiracist governance:

Distributing power beyond Europe and North America within the Steering Committee.
Reducing financial barriers for organizations in developing or transitioning economies wishing to support DORA.
Ensuring leadership positions in the Steering Committee reflect the broad diversity of the academic world.

Challenges

DORA’s recommendations focus solely on practices related to research articles published in peer-reviewed journals, thereby excluding additional outputs—such as datasets—whose importance in evaluating research effectiveness is expected to grow in the future.

Interest and transferability

In 2021, DORA launched SPACE, a tool designed to facilitate the reform of academic assessment in universities while examining the infrastructure needed to support the development of new policies and practices.

In 2018, DORA introduced its roadmap—a strategic plan to advance global research assessment reform at the institutional, national, and funding agency levels. Specifically, this roadmap aims to:

Raise awareness about the need to develop credible alternatives to the inappropriate use of metrics in research evaluation.
Investigate and promote tools and processes that support best practices in research assessment.
Expand the scope and impact of DORA’s work across all academic disciplines and into new areas.

The initiative’s website features a collection of resources designed to support the development of responsible research policies and researcher assessment practices.

Additionally, DORA hosts a repository titled “Reimagining Academic Assessment: Stories of Innovation and Change”, which includes case studies that serve as inspiration for organizations and institutions seeking to improve their academic career assessment practices.