Under the Microscope: Some Findings from the 2011 Triennial Surveillance Review
by Martin S. Edwards, Member of New Rules and Associate Professor at Seton Hall University
New Analysis on the review of IMF Surveillance.
The Fund’s recent Triennial Surveillance Review (hereafter TSR) was released at end of October of this year. A series of fourteen documents, it deepens a trend stared in the 2008 TSR of allowing external commentary.  Whereas the 2008 Review had a paper written by an external consultant on the Article IV process in
Of course, these additional efforts are a sensible response to the global economic crisis, but also to the Fund’s role in it. The Independent Evaluation Office’s 2011 review of Fund Activities in the Run-up to the Crisis was a bold statement that criticized not only the content of IMF surveillance, but also a lack of accountability for creating a system of surveillance that is not sensitive to assessing risks. Some of these problems were presaged in the 2008 Triennial Surveillance Review, which recommended more attention to the financial sector and greater attention to the problem of spillovers between countries.
Rather than discuss all fourteen papers in detail, I address some cross-cutting issues in the 2011 TSR. These include an assessment of findings at a country level, concerns about the content of Article IV consultations, and the critical issue of differential effects of IMF surveillance across Fund members. I conclude with some recommendations moving forward.
First, the TSR sought input from member countries on the effects of surveillance at two points: anonymous surveys were sent to country authorities (both the ministries of treasury or finance and the central bank or financial supervision agency of each member country). In addition, a separate document summarized the findings of twenty in-depth interviews with country authorities. I will focus on the survey findings below.
The country authority survey had a response rate of 33 percent. There are two key issues in this survey that bear discussing. First is the issue of whether Fund advice fits with the context of the member country. Respondents were asked to indicate their level of agreement or disagreement with the following statement:
During the aftermath of the global financial crisis Fund staff has provided policy advice that has been appropriately informed by the political and social context.
Overall, 7% of respondents strongly agreed with this statement, 60% of respondents agreed with this statement, 31% percent of respondents were neutral, and 3% percent of respondents disagreed. The percentage of respondents with positive views of this statement (summing up the “strongly agree” and “agree” responses) differed more by region than by income level. 100% of respondents from African countries had positive views of this statement, while only 50% of the respondents from the
A second issue is country authority views of the usefulness of the Article IV process. If country authorities are given advice that they cannot use, then it is difficult to talk about surveillance being influential or consequential.
The country authority survey asked two relevant questions:
To what extent did Fund policy advice given in the context of the most recent Article IV consultation help generate policy debate?
To what extent did Fund policy advice given in the context of the most recent Article IV consultation help foster appropriate policy change?
As with the question on context above, respondents had five options (to a very great extent, to a great extent, to some extent, to a little extent, and not at all). Unlike the previous question, the real differences here lie across income categories rather than across regions. Overall, 28% of respondents felt that Fund advice helped generate debate to a very great or great extent, 45% of respondents felt that Fund advice helped generate debate to some extent, and 27% of respondents felt that Fund advice helped generate debate to little extent or not at all.
The survey responses were broken down by income level, in which countries were classified as advanced economies, emerging economies, or low income countries. 43% of respondents from advanced economies felt that Fund advice helped generate policy debate to either a little extent or not at all. 28% of respondents from emerging economies felt that Fund advice helped generate policy debate to either a little extent or not at all. Significantly, none of the respondents from low income countries felt that Fund advice helped generate debate to a little extent or not at all. Thus, there seems to be an inverse relationship between the usefulness of Fund policy advice and country income level.
Similar findings hold for the question on whether or not Fund policy advice helped to foster appropriate policy change. Respondents here had the same five options. Overall, 39% of respondents felt that Fund advice helped foster policy change to little extent or not at all. 64% of respondents from advanced economies had a negative view of Fund advice, whereas only 34% of respondents from emerging economies and 4% of respondents from low income countries held similar negative views.
In terms of the content of country surveillance, there are three big findings here dealing with spillovers, follow-up, and omissions. National economic policies don’t exist in a vacuum, yet the idea of surveillance is based on the Fund member’s economy as a unit of analysis. Addressing the spillover problem has been a priority since 2008, and it remains a challenge moving forward. Another component of the review was a review of consultations that were concluded between January 1, 2010 and April 30, 2011. Out of the 148 consultations completed during this time period, these reports were stratified by income level and region, and a random sample of 50 staff reports was drawn. Two Fund staffers read each of the staff reports, and then the staffers answered a survey question about the staff report.
The responses to one survey question in the staff report survey demonstrate how much more work is necessary to make spillovers a mainstream issue in Article IV surveillance.
Are the impacts of global economic and financial developments or policies of other countries (systemic countries or trading partners) discussed in the report?
Respondents had four possible options: No, Yes (Passing reference), Yes (More than a passing reference, with some analysis) and Yes (Extensive substantive analysis). Overall, one staff report in five had no discussion of external impacts, and sixty percent of staff reports had either a passing reference or some analysis. The one region with the highest incidence of extensive analysis of external impacts was
A second challenge for surveillance is the lack of follow-up. One question in the staff report survey addressed this as a yes or no question:
Does the staff report include a self-contained comprehensive statement assessing the effectiveness of past surveillance in the country?
Only 44% of the Article IV reports in the overall sample addressed the effectiveness of past surveillance. The percentage of staff reports in which prior surveillance was not addressed was highest in Europe (67% of the sample) and Middle East and
Finally, it is worth stressing that the Article IV process is largely concerned with the appropriateness of country fiscal and monetary policy. Even now under the constraints of the global economic crisis, the tradeoffs between fiscal stimulus to fight unemployment and deficit reduction are the stuff of newspapers and political debate, but are largely absent in Fund surveillance documents. Concerns about the potential implications of an exit from fiscal and monetary loosening are more focused on the financial system than on unemployment. For example, the following question was posed in the staff report survey:
If fiscal or monetary exit strategies are elaborated in the Article IV report (including from extraordinary financial sector support measures), does the report discuss potential implications of these on financial stability, growth, or employment: (Check all that apply)
Despite the fact that the question allowed multiple options, concerns about the effect of exit strategies on employment were only a salient issue in 10% of the cases in which there was a discussion of exit strategies. Financial stability and growth were more important considerations than employment by a factor of three or more.
This finding points to a larger issue. The effect of exit strategies on employment rarely came up in Article IV reports. By way of contrast, it has been much more of a concern in the Fund’s dealings with the
One finding from the country authority survey makes the fairness of treatment problem visibly apparent. Respondents were asked whether they strongly disagreed, disagreed, had a neutral view, agreed, or strongly agreed with the following statement:
The IMF is evenhanded in its policy advice.
Overall, 3% of respondents strongly agreed with this statement, 35% of respondents agreed with this statement, 39% of respondents were neutral, 21% disagreed, and 2% strongly disagreed with this statement. There are large differences in answers to this question between G-20 members and non-members. Only 12% of countries that were NOT in the G-20 had a negative response to this question (summing up strongly disagree and disagree responses). By contrast, 58% of G-20 members had negative responses to this question (4% strongly disagree; 54% disagree). A central 8 challenge for the Lagarde years at the IMF will be to develop the means to reduce this perception.
Finally, what are some recommendations for the Fund moving forward as it attempts to create an organizational culture that is more surveillance-centered? First, more transparency is essential. Staff reports from the Article IV process are now released by almost every country in the world, but the survey data from this TSR may never be released. This is a missed opportunity for crowd sourcing, as additional analysis from academics and NGOs might produce some clear recommendations moving forward. A better understanding of how member countries view surveillance can only help to improve the finished product. Second, the famous line from Volcker and Gyohten (1992:143) remains clear today: “When the Fund consults with a poor and weak country, the country gets in line. When it consults with a big and strong country, the Fund gets in line.” The strategy moving forward for advanced economies needs to be based on a clear-eyed appraisal of how IMF advice can make a difference for these countries. It is only by providing resources that politicians find useful that the Fund can seek to strengthen its own influence. Finally, while there is a great deal of soul-searching at the Fund, there may not be learning. The Fund should look to other international organizations that are active in the area of surveillance to find some solutions to the problems it faces.
Momani, Bessma. 2006. Assessing the Utility of, and Measuring Learning from,
Volcker, Paul A. and Toyoo Gyohten. 1992. Changing Fortunes: The World’s Money and the Threat to American Leadership.
Hyperlinks to TSR Documents:
IMF Staff Reports
- Overview Paper
- Review of 2007 Surveillance Decision
- Staff Background Studies
- Health Check and Statistical Information
- Evaluation of IMF Surveillance of the Euro Area
- Coverage, Consistency and Coherence
- IMF and Global Financial Stability
- External Report on Interviews with Country Authorities