Information about Western Hemisphere Hemisferio Occidental
Chapter

Chapter 1 The Independent Evaluation Office: Objectives and Modes of Operation

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
March 2005
Share
  • ShareShare
Information about Western Hemisphere Hemisferio Occidental
Show Summary Details

Independent evaluation is widely regarded as an essential requirement in international financial institutions (IFIs), contributing to increased transparency and accountability and strengthening the process of learning from experience. The IMF had a long tradition of internal evaluations of its operations and policies. These were regularly submitted to the Executive Board and led to new directions on policy and procedure. However, the IMF did not have a mechanism for independent evaluation of its activities until the establishment of the IEO by the Executive Board in the second half of 2001.

This chapter outlines the main objectives of the IEO as set forth in its terms of reference, and describes the operational modalities that have been developed by the IEO to achieve these objectives. The historical background to the creation of the IEO was discussed at greater length in last year’s Annual Report.

Purpose of the IEO 1

The purpose of the IEO, as outlined in its terms of reference, is to systematically conduct objective and independent evaluations “on issues, and on the basis of criteria, of relevance to the mandate of the Fund.” The terms of reference, reproduced in Appendix 1, further elaborate that the IEO is intended to:

  • Serve as a means of enhancing the learning culture of the IMF;
  • Strengthen the IMF’s external credibility;
  • Promote greater understanding of the work of the IMF throughout its membership; and
  • Provide independent feedback to the Executive Board in its governance and oversight responsibilities over the IMF.

The work of the IEO is envisaged as complementing the review and evaluation work being conducted within the IMF and is expected to improve the IMF’s ability to draw lessons from its experience and to more quickly integrate improvements into its future work.

Independence

Independence of evaluation is critical if it is to be credible. This aspect of evaluation was emphasized in the Executive Board discussions that led to the establishment of the Independent Evaluation Office. The terms of reference explicitly state that the “IEO will be independent of Fund management and staff and will operate at arm’s length from the Fund’s Executive Board.” The following provisions are designed to achieve this objective:

  • The Director of the IEO is appointed solely by the Executive Board; IMF management, while it may be consulted in the selection process, is not involved in making the selection.2 The Director is specifically precluded from appointment or reappointment to an IMF regular staff position at the end of the term of office.
  • With a view to ensuring that the IEO is staffed with independent and highly qualified individuals, the Director of the IEO is solely responsible for the selection of IEO personnel, a majority of whom must come from outside the IMF.3 IEO staff reports exclusively to the Director of the IEO and not to IMF management.
  • The budget of the IEO is prepared by the Director and submitted directly to the Executive Board for approval. IMF management is not involved at any stage of the process.
  • The IEO’s work program is determined by the Director in light of consultations with members of the Executive Board and other interested stakeholders, from both inside and outside the IMF. The work program determined by the Director is presented to the Executive Board for review, but is not subject to the Board’s approval.

The IEO’s terms of reference provide that within three years of the launch of IEO operations, the Executive Board should initiate an external evaluation of the IEO to assess its effectiveness and to consider possible improvements to its structure, mandate, operational responsibilities, or terms of reference. While the precise timing and modalities of this external evaluation will be determined by the Executive Board, the evaluation is expected to take place toward the end of 2005. The review is expected to solicit broad-based input from both within and outside the official community.

Budget and Staffing

The IEO’s current budget is calibrated to allow the office to achieve an output equivalent to approximately four standard-size evaluations a year. The approved budget for FY20044 was US$3.9 million and estimated actual expenditure was US$3.4 million (Appendix 2). The approved budget for FY2005 is US$4.2 million. These amounts include staff costs, consultants, travel, outreach, and other miscellaneous costs. The FY2005 budget represents a real increase of 2 percent over the previous fiscal year, reflecting the phased buildup in the number of evaluation projects to four a year. The IEO’s budget is equivalent to about 0.5 percent of the IMF’s total administrative budget.

The IEO currently has 13 full-time staff positions, including the Director, the Deputy Director, nine professionals, and two administrative assistants. The majority of the staff has been recruited from outside the IMF and has wide experience in relevant areas. Since the IEO’s evaluation work is expected to involve constantly changing topics, there is a shifting need for expertise of different types. This requires a greater use of consultants than in the IMF in general, a practice that also helps to achieve independence and credibility. The budget for consultants is about a quarter of the IEO’s full-time staff budget.

The IEO Work Program and Future Menu of Topics

The terms of reference provide a very broad mandate for the work program of the IEO. It specifies that the IEO “should focus on issues of importance to the Fund’s membership and of relevance to the mandate of the Fund taking account of current institutional priorities.” Choosing three to four studies a year from the very wide range of issues potentially eligible under the IEO’s mandate necessarily requires careful prioritization. This has been achieved through extensive consultations with stakeholders inside and outside the IMF, and by using transparent criteria.

In choosing its annual work programs, the IEO has consulted with internal and external interested groups, including representatives of civil society and academics, on the basis of a note that identifies a list of potential issues for evaluation. The IEO’s first three annual work programs were guided by a core set of 15 possible topics, which had been chosen (again on the basis of extensive consultation with various stakeholders) from the initial list of over 30 topics prepared immediately after the office’s establishment in July 2001.5 The status of completed and ongoing evaluation projects is given in Table 1.1.

Table 1.1.Completed and Ongoing IEO Work Programs
ProjectExpected Completion1
Initial round of evaluation projects
Prolonged Use of IMF ResourcesCompleted
The IMF and Recent Capital Account Crises: Indonesia, Korea, BrazilCompleted
Fiscal Adjustment in IMF-Supported ProgramsCompleted
FY2004 work program
The IMF and ArgentinaCompleted
Poverty Reduction Strategy Papers and the Poverty Reduction and Growth FacilityCompleted
IMF Technical AssistanceCompleted January 2005
FY2005 work program
IMF’s Approach to Capital Account LiberalizationFebruary 2005
IMF Assistance to JordanApril 2005
Financial Sector Assessment ProgramSeptember/October 2005
Multilateral SurveillanceLast quarter of 2005
FY2006 work program2
IMF Structural ConditionalityFirst quarter of 2006

The date refers to the time the completed report (and management/staff comments) is expected to be circulated to the Board. The timing of the actual Board discussion of each project is set as part of the overall schedule of the Executive Board and is not under the IEO’s control. Publication is decided by the Board and takes place after the Board discussion.

The remainder of the work program for FY2006 will be announced in the first quarter of 2005.

The date refers to the time the completed report (and management/staff comments) is expected to be circulated to the Board. The timing of the actual Board discussion of each project is set as part of the overall schedule of the Executive Board and is not under the IEO’s control. Publication is decided by the Board and takes place after the Board discussion.

The remainder of the work program for FY2006 will be announced in the first quarter of 2005.

Now that 11 out of the initial set of 15 issues have been selected for evaluation—and priorities and pressing issues for the IMF may well have changed after three years—the IEO is initiating a new round of consultation to determine a menu of topics to guide its work programs over the coming years. For this purpose, the IEO has prepared a preliminary list of candidate topics, from which the work programs for the coming three years may be selected. As in the past, the criteria used for selection gave priority to topics that (i) had the greatest learning potential, (ii) had the greatest interest among the wide range of the IMF membership, and (iii) had been the subject of controversy and criticism while bearing in mind the IEO’s comparative advantage. In addition, priority was also given to topics with a potential for producing recommendations that could assist the IMF as it embarks on the process of developing medium-and long-term strategies. The list is being posted on the IEO website and is reproduced in Appendix 3. Comments are invited and may be directed to ieo@imf.org. The actual work program will be announced annually, and it is expected that the program for FY2006 will be announced in the first quarter of 2005.

Transparency and Accountability

For IEO evaluations to have credibility, it is important that they be conducted in a transparent manner, with adequate opportunity given to different stakeholders, especially those outside the IMF, to provide relevant inputs. To meet these objectives, the IEO has developed procedures that allow for extensive consultations in designing the evaluation project and also for receiving substantive inputs during implementation.

To ensure consultation at the design stage, each evaluation begins with the preparation of an issues paper that identifies the questions to be addressed and, to the extent possible, the methodology to be followed. The IEO seeks comments on this document from Executive Directors, IMF staff and management, member country governments (especially in the case of evaluations involving individual countries), and other interested observers. The issues paper is posted on the IEO website (www.imf.org/ieo) to elicit comments from a wider set of interested external observers. The comments received are taken into account in determining the final terms of reference for the study, which are also posted on the website.

The responsibility for the research undertaken and the conclusions reached must necessarily rest with the IEO. However, in conducting its evaluations the IEO interacts extensively with concerned parties both inside and outside the IMF. A unique feature of IEO evaluations, which distinguishes them from other external analyses of IMF activities, is that the IEO has access to internal IMF documents not normally made public and can also interview IMF staff concerned with the subject of the evaluation. For evaluations involving individual countries, consultations are held in the country concerned with both the authorities and a broad range of other interested parties, including civil society. Furthermore, the issues paper posted on the IEO website specifically invites interested parties to make submissions to the IEO on topics covered by the issues paper.

An important aspect of transparency and credibility is the assurance that IEO reports will be published and disseminated to a wide audience. The IEO’s terms of reference provide that the reports, once they have been considered by the Executive Board, will be promptly published “unless in exceptional circumstances the Executive Board were to decide otherwise.” All five evaluation reports prepared in the course of the first two years of the IEO’s operations have been published.

To ensure full transparency, IEO reports are published in the form in which they were submitted to the Executive Board, without being changed in any way in light of comments received from management.6 Comments received on the evaluation report from IMF management and staff, along with the IEO’s reactions to those comments, if any, are submitted to the Executive Board as separate documents for the Board meeting at which the evaluation report is discussed. These documents are published together with the evaluation report and a summary of the Executive Board discussion. Once released to the public, the report is immediately posted on the IEO’s website, followed by publication of the print version.

Outreach Activities, Opportunities for Feedback, and Follow-Up

One of the objectives of the IEO is to promote greater understanding of the work of the IMF. Accordingly, once an evaluation report is made public, the IEO engages in external outreach to make the evaluation report and the Board’s decisions on it available to a wider audience. To promote this objective, various outreach events are organized to discuss each report after publication. To increase accessibility of the evaluation messages, a number of the country case studies have been translated into relevant local languages.7

During the past 18 months, the IEO participated in a number of outreach seminars and mid-pipeline workshops, generally at the invitation of third-party organizers. These are listed in Appendix 4.8

Following the approach begun in the Annual Report 2003, this report describes the status of follow-up on the five IEO evaluations completed to date.

The matrices contained in Appendixes 59 track specific recommendations for each evaluation, noting the Executive Board’s response to, and degree of support for, each recommendation; management’s or staff’s intended follow-up; and any actions to implement the recommendations that have been taken to date. Staff inputs are sought on the descriptions of the follow-up, but the final judgment on the language used is that of the IEO. This has proven to be a useful mechanism to track the extent to which IEO evaluations have elicited a response from the IMF to change aspects of its policies and operations. That said, the matrices are essentially of a descriptive nature and are not intended to make judgments on the effectiveness of follow-up actions in achieving their intended objectives.9

Relations with Other Evaluation Offices

Since independent evaluation is now a feature of all IFIs and there are evaluation offices in all bilateral donor agencies, there are networks of such offices that exchange information on issues of mutual interest, including methodological approaches. The IEO is a member of the Evaluation Cooperation Group, which comprises the evaluation offices of multilateral development banks and the IMF and aims to strengthen the use of evaluation for greater effectiveness and accountability as well as to share lessons and harmonize approaches (www.ecgnet.org). The IEO also participates in the activities of the Development Assistance Committee Working Party on Aid Evaluation, an international network for development evaluation experts and managers under the auspices of the Organization for Economic Cooperation and Development (OECD), which seeks to improve evaluation practice by sharing methods and experience and by elaborating technical guidance.

1Additional background to the thinking behind the goals and modes of operation of the office is given in “Making the IMF’s Independent Evaluation Office Operational: A Background Paper,” prepared by the Evaluation Group of Executive Directors, August 7, 2000 (www.imf.org/external/np/eval/evo/2000/Eng/evo.htm).
2The initial terms of reference provided for a four-year term, renewable for a second term of up to three years. The Executive Board, in July 2001, appointed Montek Singh Ahluwalia as the first Director of the IEO. Mr. Ahluwalia resigned on June 30, 2004 to become Deputy Chairman of the Indian Planning Commission. When a search began for a new Director, the terms of the appointment were changed to provide for a nonrenewable term of six years.
3The maximum length of appointment for full-time staff in the IEO is six years.
4The IMF’s fiscal year begins May 1 and ends April 30.
6The only exception would be for purely factual corrections, for which an errata page, identifying the specific corrections made, would be issued.
7A full list of IEO publications is available on the IEO website at www.imf.org/external/np/ieo/pap.asp.
8We would especially like to express our gratitude to InWent, Capacity Building International of Germany for hosting an annual workshop on IEO evaluation projects that has allowed us to receive feedback on completed and ongoing projects from a wide range of participants from Europe as well as from developing countries most directly associated with the various evaluations.
9Where subsequent internal assessments—for example, in the context of the regular biennial reviews of IMF surveillance and conditionality—assess progress on particular issues, the results are reported in the matrices, but without implying any IEO endorsement of these internal assessments. The role of the IEO in following up on evaluation recommendations may be revisited following the evaluation of the IEO itself.

    Other Resources Citing This Publication