Staff Response to the Independent Evaluation Office Report on Research at the IMF—Relevance and Utilization
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- March 2013
Executive Board Meeting
June 13, 2011
1. The report provides an informative stock-taking on the quality, utilization, and perceptions of Fund research, as well as constructive recommendations. Conducting high-quality, policy-relevant research at the Fund is essential for our credibility—both in interactions with country authorities and with the international community more generally. We therefore welcome the overall finding that much of Fund research has been highly relevant to the membership and has benefited from interactions with academia and national authorities. On advice, suggestions for periodic strategic reviews of research, better allocation of resources to research projects, enhanced review processes, and vigilance against the risk of message-driven research warrant further consideration. We look forward to hearing the Board’s views on these and other issues, including taking into consideration potential additional resource costs.
2. Nonetheless, we have concerns about some aspects, notably on the targeting, neutrality, and coordination of research. The report could have been stronger if it better accounted in its analysis for the different purposes and audiences for various Fund research products and could have delved more deeply into how to avoid message-driven research. Also, while it is important to avoid any unnecessary duplication, the IEO recommendation to coordinate IMF research could result in the stifling of individual research efforts.
3. We agree on the need to narrow gaps in IMF research—efforts have been underway and will continue. Since 2008, the end of the period covered by the evaluation, the Fund has increased research in areas cited by the IEO. Topics covered include work on capital controls and macro-prudential policies, fiscal policy and debt sustainability, exchange rate regimes and stability of the international monetary system, monetary and exchange rate policies, optimal reserve holdings, reserves adequacy and country insurance, policy responses to the global financial crisis, fiscal multipliers and counter-cyclical policies, and various aspects of macro-financial linkages.
4. The report makes an important point about message-driven research, and could have delved more into this critical issue. Fund-relevant research will almost inevitably carry policy messages—that is what makes it relevant. But it is troubling if researchers feel that they need to toe a line or tilt empirical results. We support the call for vigilance against these risks and believe the report could do more in proposing concrete solutions.
5. The methodology employed to judge different research product lines would have benefited from taking explicit account of their intended purposes and audiences. For instance, in the discussion of technical quality, the report appears to have misunderstood the purpose of Regional Economic Outlooks (REOs), which are not simply research products but are also used as outreach tools, including as a means to engage policymakers. While the REO style may be narrative, policy recommendations are underpinned by rigorous research and provide analysis and messaging as well as policy advice that is grounded in cross-country and intra-regional analysis. To judge REOs by traditional academic metrics and benchmarks alone is therefore inappropriate. However, the report’s finding on the relevance and utilization of some REOs is well taken, and further efforts need to be made to improve these. The report could have used a more systematic framework for assessing the quality and relevance of Fund research, asking if the Board or Management had identified the goals of different lines of Fund research, and then evaluating outputs against those objectives.
6. Periodic strategic reviews of the functions and uses of research products merit further consideration. Such reviews could also consider how best to allocate time and resources among various research product lines and how best to improve the review process, taking into account budget constraints. If the Board agrees, staff will consider these issues and present proposals in the Management Implementation Plan (MIP). The recommendation to consult with member countries and the Executive Board on an indicative medium-term research agenda, however, raises concerns as it may limit flexibility and may be seen as a bureaucratic step. Notably, there is already consultation with the Board, for example, through periodic discussions of the Managing Director’s Work Program.
7. Staff generally support the need to consult more with country authorities on research topics prepared for surveillance. At the same time, staff and authorities will not always agree on which issues are most pressing, and staff will need to remain free to research the issues that they judge to be most important. The report should also recognize that the various research products differ with respect to the desirable degree of involvement of country authorities in the choice of topics. Selected Issues Papers (SIPs), for instance, are designed to provide background for bilateral surveillance, and so teams should normally be encouraged to consult with the authorities in advance on SIPs on a regular basis. This is not necessarily the case for all Fund research, especially on multilateral, theoretical, and/or cross-country work.
8. The recommendation to increase staff tenure is well taken, and knowledge of country specifics for research is clearly important. However, as noted in responses to previous evaluations, striking a balance between mission team stability on the one hand and the desire for cross-country perspectives, along with staff career development needs and department flexibility, on the other remains a complex challenge. An update on this important issue is provided in the Fourth Periodic Monitoring Report of Board-Endorsed IEO Recommendations.
9. We agree on the need to improve the ability of stakeholders to distill relevant findings and policy implications. Several initiatives have been undertaken since 2008 to begin to address this issue and more can be done. For example, the “Research at the IMF” website was created in 2009 precisely to promote the dissemination of IMF research, and there has been increased investment in other access channels such as the eLibrary and Google Book Search. The IEO also highlights the creation of the Staff Position Note/ Staff Discussion Note series since 2008 to help address this problem, to which we would add enhancements to publications such as the IMF Survey and Finance and Development.
10. As noted in paragraph 4, we agree on the need to promote openness to alternative perspectives. It is particularly important that researchers not feel that they need to toe a line or tilt empirical results. We will reflect on this issue and present proposals in the MIP.
11. While coordination of research could help avoid duplication—certainly necessary in a constrained budgetary environment—it is also important that individual research efforts are not stifled. As the report itself notes, there have been numerous attempts at coordination of research over the years. We believe there is merit in healthy competition of research efforts across departments and would not want to stifle this. By the same token, departments should not seek to force collaboration for individual pieces of research (e.g., SDNs, and other research outputs that represent the views of the individual authors). Nonetheless, collaboration with the Research Department and/or other functional departments having expertise in the topic should otherwise be expected, especially in Board papers with substantial analytical content. To help researchers across the Fund identify research priorities while avoiding duplication of effort, better information sharing and dissemination is needed within the institution. To achieve this, one possibility would be to resurrect the Committee on Research Priorities (CRP), with the Managing Director or First Deputy Managing Director chairing, and the Director of Research as secretary. The Director of Research could also report to the Executive Board once per year, reviewing major research accomplishments over the past year and laying out identified priorities for the following year.