Current policies require that proposed projects be subject to "outside
evaluation" before approval by a Commission and by the Division President.
In some instances rigorous critiques are obtained, analogous to those given
by a referee of a journal article, while in other instances only a cursory
analysis is undertaken. Past practice has often limited the description
of a new project to a short paragraph, which is often inadequate for critical
review by someone outside IUPAC. Since the "priority" accorded
a project has little or no relation to funding, there has been little incentive
to improve either the project description or the rigor of evaluation.
The SDIC believes that a system for evaluating proposals prior to approval
is essential. It must be seen within IUPAC as fair and must be viewed outside
IUPAC as rigorous. Moreover, at some time after completion of a project
there should be systematic evaluation of the accomplishments.
Prospective evaluation. A prospective evaluation has two purposes.
First, it should weed out any proposal that IUPAC does not want to endorse
- for example, something that is irrelevant to IUPAC's mission; that might
be embarrassing to the Union; or that might be widely perceived as being
undertaken solely to give credibility to an individual's "pet project".
Although there may be few such projects proposed, it is important to the
Union's reputation that the review system impose some check point for such
eventualities.
The second purpose of prospective evaluation is to allocate the Union's
resources in an optimum way. In setting up procedures, IUPAC should be
careful not to impose burdensome requirements. There should be considerable
flexibility tied to the level of resources required. For example, an essentially
one-person project (the product of which would ultimately be reviewed and
approved broadly) might make few demands on IUPAC resources and require
relatively little justification. A project that is funded within a Division's
budget must be justified there but not at higher levels. A project that
is proposed for central funding (the successor to "pool projects")
should have adequate justification for this purpose. A project that requires
outside resources (UNESCO, ICSU, industry, etc.) may require more detailed
justification. For all types of projects the totality of IUPAC resources
should be considered, including Secretariat support and publication requirements,
as well as direct costs for the project. It is likely that the principal
direct costs will continue to be for travel and subsistence; however, even
with the voluntary contribution of time by Task Group members there may
be instances where modest expenditures for specific purposes, such as data
evaluation or purchase of reagents, might be warranted.
The Bureau should develop general criteria to guide reviewers and establish
detailed procedures for prospective evaluation at the Division level and
centrally. The SDIC believes that the criteria should include the following:
|
Inherent scientific quality of the project |
|
Anticipated impact on some aspect of the chemical sciences or interdisciplinary
activities |
|
Relation to IUPAC's goals and strategic thrusts |
|
Provision for publication of results and widespread dissemination |
|
Cost/benefit ratio |
Retrospective evaluation. Retrospective review and evaluation
of individual projects has generally not been done at the IUPAC level,
but some Commissions or Divisions do look at outcomes in a serious way.
The SDIC believes that the Bureau should set criteria for either a central
evaluation or a mandated Division evaluation of the impact of a project
and of the outcome relative to what was anticipated. The nature and timeframe
for such an evaluation must be decided on a case by case basis, but must
involve consultation widely with the relevant scientific community. Where
applicable, quantitative assessments, such as literature citations or adoption
of nomenclature by major journals, should be employed. Over a period of
years, such retrospective evaluations can provide useful guidance on the
selection of future projects. In addition, the retrospective review may
be valuable in identifying material that can be used for public relations
purposes and for future fund-raising.
Executive Summary
Formation of the SDIC
Strategic Plan
Organization and Management of Scientific Work
Responsibilities of Division Committees
Election of Division Committees and Division Officers
Project-Driven System
Conversion to a New Project-Driven System
Operation of a Project-Driven System
Evaluation of Projects
Role of the Secretariat
Financial Considerations
Summary of Recommendations on Organization and
Management
Summary of Formal Actions Required
Concluding Statement
Appendix 1
Appendix 2
Appendix 3
Appendix 4
Home - News and Notices - Symposia/ Conferences - IUPAC Organizations and People
Recommendations
- Provisional
Recommendations - Divisions
- Commissions
Standing Committees
- Publications -
Links - IUPAC
Affiliates
Page last modified 5 October 1998.
Copyright © 1997, 98 International Union of Pure and Applied Chemistry.
Questions or comments about IUPAC
please contact the Secretariat.
Questions regarding the website
please contact Web Help.