You will also find details on IIT’s increasing efforts to adopt fair, responsible, and internationally recognized research evaluation practices.
Research Evaluation at IIT
This page provides an overview of how research at IIT is evaluated, the principles guiding our assessment, and the actions IIT is taking to align with international best practices
Research Evaluation Principles
IIT follows internationally recognized best practices, aligning with the principles of the Leiden Manifesto, the European Charter for Researchers, and the commitments of the Coalition for Advancing Research Assessment (CoARA), a global initiative aimed at reforming research evaluation practices.
To ensure a comprehensive and just assessment, research evaluation at the Italian Institute of Technology is performed according to the following principles.
IIT researchers must be informed of research evaluation activities
IIT staff and researchers are informed in advance of ongoing evaluation processes and made aware of the reasons why an evaluation is taking place. Those processes may involve a researcher's activity and the activity of a research group or center.
Comparing research groups should be done by providing contextual information
When comparing research groups, contextual information is provided when using indicators (e.g., number of group members, citational behaviors in the field, normalization by budget received, etc.). It will be recognized when it is not appropriate to provide certain types of quantitative data and when normalization is required. It is recommended that appropriate caveats regarding likely differences between research fields are acknowledged by the evaluators.
Research assessment should recognize diverse research outputs and use diverse and various approaches
It is important that research assessment seeks a variety of perspectives and considers multiple levels of analysis (academic, economic, societal) for robust evidence-based decision-making. It is recommended to include a wide range of qualitative evaluation criteria and a suite of quantitative indicators rather than a single measure in isolation.
Quantitative indicators can be useful to have informed decision-making processes and expose potential biases or preconceptions. However, they are often affected by the career stage of a researcher, their discipline, their specific focus of research, and other characteristics. Therefore, peer review (e.g., as in panels of experts or evaluation committees) must go side by side with those indicators and provide an expert judgement and context in every evaluation process in IIT.
Data sources should be reliable, robust, accurate, and transparent
Source data are made available where possible. For example, if the publication portfolio of a research group is analyzed, researchers are given information on how publications and their metadata have been sourced (e.g., Scopus, Web of Science, etc.) and can see the publication and citation data included. They are given guidance on how to request corrections. Similarly, funding and grants data, together with commercial and patenting activities shall be verifiable, too.
Where possible, the criteria of evaluation should be available to researchers and the quantitative indicators used should be easily reproducible. Researchers must be able to check and verify which bibliometric information is used to generate indicators.
Evaluation criteria, Research indicators, and data sources should be regularly reviewed and updated
The systems of evaluation used are sensitive to the evolving needs of IIT and responsive to the changing nature of the research landscape. Qualitative evaluation criteria and indicators are updated regularly and revised with the contribution of all stakeholders (scientific staff and the management).
Misplaced concreteness and false precision should be avoided
A "basket of metrics" approach will be used to avoid over-reliance on one indicator. The limitations of the metrics will be recognized, for example, the meaning and interpretation of citation counts, and the problems of false accuracy and variability associated with some journal-level metrics. The limitations of small sample sizes should also be accounted for, in any analysis.
There should be a shared understanding of best practices in research evaluation
IIT staff in charge of the evaluation processes should be continuously trained to ensure that all actors involved know of and act according to these principles. It is important to share with management the most recent best practices and pitfalls of indicators, when present.
IIT CoARA Action Plan (2024-2027)
In 2022, IIT joined the Coalition for Advancing Research Assessment (CoARA), committing to reforming research assessment practices and established a working group across IIT’s directorates.
This effort resulted in IIT’s CoARA Action Plan (2024-2027), which defines 12 key actions aimed at ensuring that research excellence is measured comprehensively, valuing a broad range of scientific contributions while maintaining integrity, fairness, and innovation in research assessment.
The 12 Actions of the Plan:
- Involvement of scientific staff at all career levels in research evaluation processes.
- Definition of institutional research evaluation principles, ensuring alignment with international best practices.
- Review of research evaluation criteria in career path assessments, including hiring, promotions, and tenure-track evaluations.
- Review of research group assessments, ensuring fairness and relevance in evaluations of teams and research units.
- Ensuring gender equality and inclusiveness in research assessment processes.
- Recognition of Open Science and Research Data Management as valuable research outputs.
- Alignment with CoARA National Chapters and Working Groups for a harmonized approach.
- Development of new tools, dashboards, and metrics to support the reformed research assessment framework.
- Enhancing transparency and awareness on research assessment indicators among researchers and management.
- Training internal and external evaluators on fair and responsible evaluation practices.
- Providing career mentoring and guidance for early-career researchers.
- Communicating institutional decisions on research assessment processes and events clearly to all stakeholders.
The full IIT CoARA Action Plan is available from the CoARA.eu website or directly on Zenodo.