Evaluation of Projects
Current policies require that proposed projects be subject to "outside evaluation" before approval by a Commission and by the Division President. In some instances rigorous critiques are obtained, analogous to those given by a referee of a journal article, while in other instances only a cursory analysis is undertaken. Past practice has often limited the description of a new project to a short paragraph, which is often inadequate for critical review by someone outside IUPAC. Since the "priority" accorded a project has little or no relation to funding, there has been little incentive to improve either the project description or the rigor of evaluation.
The SDIC believes that a system for evaluating proposals prior to approval is essential. It must be seen within IUPAC as fair and must be viewed outside IUPAC as rigorous. Moreover, at some time after completion of a project there should be systematic evaluation of the accomplishments.
Prospective evaluation. A prospective evaluation has two purposes. First, it should weed out any proposal that IUPAC does not want to endorse - for example, something that is irrelevant to IUPAC's mission; that might be embarrassing to the Union; or that might be widely perceived as being undertaken solely to give credibility to an individual's "pet project". Although there may be few such projects proposed, it is important to the Union's reputation that the review system impose some check point for such eventualities.
The second purpose of prospective evaluation is to allocate the Union's resources in an optimum way. In setting up procedures, IUPAC should be careful not to impose burdensome requirements. There should be considerable flexibility tied to the level of resources required. For example, an essentially one-person project (the product of which would ultimately be reviewed and approved broadly) might make few demands on IUPAC resources and require relatively little justification. A project that is funded within a Division's budget must be justified there but not at higher levels. A project that is proposed for central funding (the successor to "pool projects") should have adequate justification for this purpose. A project that requires outside resources (UNESCO, ICSU, industry, etc.) may require more detailed justification. For all types of projects the totality of IUPAC resources should be considered, including Secretariat support and publication requirements, as well as direct costs for the project. It is likely that the principal direct costs will continue to be for travel and subsistence; however, even with the voluntary contribution of time by Task Group members there may be instances where modest expenditures for specific purposes, such as data evaluation or purchase of reagents, might be warranted.
The Bureau should develop general criteria to guide reviewers and establish detailed procedures for prospective evaluation at the Division level and centrally. The SDIC believes that the criteria should include the following:
Inherent scientific quality of the project Anticipated impact on some aspect of the chemical sciences or interdisciplinary activities Relation to IUPAC's goals and strategic thrusts Provision for publication of results and widespread dissemination Cost/benefit ratio Retrospective evaluation. Retrospective review and evaluation of individual projects has generally not been done at the IUPAC level, but some Commissions or Divisions do look at outcomes in a serious way. The SDIC believes that the Bureau should set criteria for either a central evaluation or a mandated Division evaluation of the impact of a project and of the outcome relative to what was anticipated. The nature and timeframe for such an evaluation must be decided on a case by case basis, but must involve consultation widely with the relevant scientific community. Where applicable, quantitative assessments, such as literature citations or adoption of nomenclature by major journals, should be employed. Over a period of years, such retrospective evaluations can provide useful guidance on the selection of future projects. In addition, the retrospective review may be valuable in identifying material that can be used for public relations purposes and for future fund-raising.
Executive Summary
Formation of the SDIC
Strategic Plan
Organization and Management of Scientific Work
Responsibilities of Division Committees
Election of Division Committees and Division Officers
Project-Driven System
Conversion to a New Project-Driven System
Operation of a Project-Driven System
Evaluation of Projects
Role of the Secretariat
Financial Considerations
Summary of Recommendations on Organization and Management
Summary of Formal Actions Required
Concluding Statement
Appendix 1
Appendix 2
Appendix 3
Appendix 4
Home - News and Notices - Symposia/ Conferences - IUPAC Organizations and People
Recommendations - Provisional Recommendations - Divisions - Commissions
Standing Committees - Publications - Links - IUPAC Affiliates
Page last modified 5 October 1998.
Copyright © 1997, 98 International Union of Pure and Applied Chemistry.Questions or comments about IUPAC
please contact the Secretariat.
Questions regarding the website
please contact Web Help.
