Thursday, October 23, 2014

Under the Microscope: Some Findings from the 2011 Triennial Surveillance Review

by Martin S. Edwards, Member of New Rules and Associate Professor at Seton Hall University

New Analysis on the review of IMF Surveillance.

Download a pdf of this report

The Fund’s recent Triennial Surveillance Review (hereafter TSR) was released at end of October of this year. A series of fourteen documents, it deepens a trend stared in the 2008 TSR of allowing external commentary. [1] Whereas the 2008 Review had a paper written by an external consultant on the Article IV process in Europe, the 2011 review had three additional external studies (one on consistency and coherence of surveillance, one on the interaction between IMF surveillance and other international financial institutions, and one summarizing interviews with country authorities) plus another one on Europe. The 2011 review also had comments by Joseph Stiglitz and Martin Wolf, a review of the entire package by a team of external advisors, and an action plan to address the findings by Managing Director Lagarde.

Of course, these additional efforts are a sensible response to the global economic crisis, but also to the Fund’s role in it. The Independent Evaluation Office’s 2011 review of Fund Activities in the Run-up to the Crisis was a bold statement that criticized not only the content of IMF surveillance, but also a lack of accountability for creating a system of surveillance that is not sensitive to assessing risks. Some of these problems were presaged in the 2008 Triennial Surveillance Review, which recommended more attention to the financial sector and greater attention to the problem of spillovers between countries.

Rather than discuss all fourteen papers in detail, I address some cross-cutting issues in the 2011 TSR. These include an assessment of findings at a country level, concerns about the content of Article IV consultations, and the critical issue of differential effects of IMF surveillance across Fund members. I conclude with some recommendations moving forward.

First, the TSR sought input from member countries on the effects of surveillance at two points: anonymous surveys were sent to country authorities (both the ministries of treasury or finance and the central bank or financial supervision agency of each member country). In addition, a separate document summarized the findings of twenty in-depth interviews with country authorities. I will focus on the survey findings below.

The country authority survey had a response rate of 33 percent. There are two key issues in this survey that bear discussing. First is the issue of whether Fund advice fits with the context of the member country. Respondents were asked to indicate their level of agreement or disagreement with the following statement:

During the aftermath of the global financial crisis Fund staff has provided policy advice that has been appropriately informed by the political and social context.

Overall, 7% of respondents strongly agreed with this statement, 60% of respondents agreed with this statement, 31% percent of respondents were neutral, and 3% percent of respondents disagreed. The percentage of respondents with positive views of this statement (summing up the “strongly agree” and “agree” responses) differed more by region than by income level. 100% of respondents from African countries had positive views of this statement, while only 50% of the respondents from the Americas had positive views of this statement. Percentages were roughly equivalent in other regions: 43% of respondents in Asia viewed this statement positively, and 53% of respondents from the Middle East and Central Asia viewed this statement positively.

A second issue is country authority views of the usefulness of the Article IV process. If country authorities are given advice that they cannot use, then it is difficult to talk about surveillance being influential or consequential.[2]

The country authority survey asked two relevant questions:

To what extent did Fund policy advice given in the context of the most recent Article IV consultation help generate policy debate?

To what extent did Fund policy advice given in the context of the most recent Article IV consultation help foster appropriate policy change?

As with the question on context above, respondents had five options (to a very great extent, to a great extent, to some extent, to a little extent, and not at all). Unlike the previous question, the real differences here lie across income categories rather than across regions. Overall, 28% of respondents felt that Fund advice helped generate debate to a very great or great extent, 45% of respondents felt that Fund advice helped generate debate to some extent, and 27% of respondents felt that Fund advice helped generate debate to little extent or not at all.

The survey responses were broken down by income level, in which countries were classified as advanced economies, emerging economies, or low income countries. 43% of respondents from advanced economies felt that Fund advice helped generate policy debate to either a little extent or not at all. 28% of respondents from emerging economies felt that Fund advice helped generate policy debate to either a little extent or not at all. Significantly, none of the respondents from low income countries felt that Fund advice helped generate debate to a little extent or not at all. Thus, there seems to be an inverse relationship between the usefulness of Fund policy advice and country income level.

Similar findings hold for the question on whether or not Fund policy advice helped to foster appropriate policy change. Respondents here had the same five options. Overall, 39% of respondents felt that Fund advice helped foster policy change to little extent or not at all. 64% of respondents from advanced economies had a negative view of Fund advice, whereas only 34% of respondents from emerging economies and 4% of respondents from low income countries held similar negative views.

In terms of the content of country surveillance, there are three big findings here dealing with spillovers, follow-up, and omissions. National economic policies don’t exist in a vacuum, yet the idea of surveillance is based on the Fund member’s economy as a unit of analysis. Addressing the spillover problem has been a priority since 2008, and it remains a challenge moving forward. Another component of the review was a review of consultations that were concluded between January 1, 2010 and April 30, 2011. Out of the 148 consultations completed during this time period, these reports were stratified by income level and region, and a random sample of 50 staff reports was drawn. Two Fund staffers read each of the staff reports, and then the staffers answered a survey question about the staff report.

The responses to one survey question in the staff report survey demonstrate how much more work is necessary to make spillovers a mainstream issue in Article IV surveillance.

Are the impacts of global economic and financial developments or policies of other countries (systemic countries or trading partners) discussed in the report?

Respondents had four possible options: No, Yes (Passing reference), Yes (More than a passing reference, with some analysis) and Yes (Extensive substantive analysis). Overall, one staff report in five had no discussion of external impacts, and sixty percent of staff reports had either a passing reference or some analysis. The one region with the highest incidence of extensive analysis of external impacts was Europe, as this level of discussion appeared in half of the staff reports in the sample.

A second challenge for surveillance is the lack of follow-up. One question in the staff report survey addressed this as a yes or no question:

Does the staff report include a self-contained comprehensive statement assessing the effectiveness of past surveillance in the country?

Only 44% of the Article IV reports in the overall sample addressed the effectiveness of past surveillance. The percentage of staff reports in which prior surveillance was not addressed was highest in Europe (67% of the sample) and Middle East and Central Asia (75% of the sample). If surveillance is not framed as a cumulative enterprise, then it can be difficult to suggest that it has meaningful impact on countries.

Finally, it is worth stressing that the Article IV process is largely concerned with the appropriateness of country fiscal and monetary policy. Even now under the constraints of the global economic crisis, the tradeoffs between fiscal stimulus to fight unemployment and deficit reduction are the stuff of newspapers and political debate, but are largely absent in Fund surveillance documents. Concerns about the potential implications of an exit from fiscal and monetary loosening are more focused on the financial system than on unemployment. For example, the following question was posed in the staff report survey:

If fiscal or monetary exit strategies are elaborated in the Article IV report (including from extraordinary financial sector support measures), does the report discuss potential implications of these on financial stability, growth, or employment: (Check all that apply)

Despite the fact that the question allowed multiple options, concerns about the effect of exit strategies on employment were only a salient issue in 10% of the cases in which there was a discussion of exit strategies. Financial stability and growth were more important considerations than employment by a factor of three or more.

This finding points to a larger issue. The effect of exit strategies on employment rarely came up in Article IV reports. By way of contrast, it has been much more of a concern in the Fund’s dealings with the US. The Fund’s document for the Paris G-20 summit effectively endorsed the Obama Administration’s American Jobs Act as a way to address the need to deal with unemployment and the need to develop fiscal consolidation. As suggested above in the discussion of the effects of Fund policy advice, the IMF does not treat all countries similarly. A central challenge for the Fund in the years to come is to develop a strategy for making its message clearer and more influential in the advanced economies. The report of the External Advisory Group evaluating the 2011 TSR recommended a greater focus on employment and social elements in surveillance as a means to increase IMF influence in the advanced economies. This is an instance in which deeper surveillance might become more influential surveillance by providing more fine-grained advice to policy makers.

One finding from the country authority survey makes the fairness of treatment problem visibly apparent. Respondents were asked whether they strongly disagreed, disagreed, had a neutral view, agreed, or strongly agreed with the following statement:

The IMF is evenhanded in its policy advice.

Overall, 3% of respondents strongly agreed with this statement, 35% of respondents agreed with this statement, 39% of respondents were neutral, 21% disagreed, and 2% strongly disagreed with this statement. There are large differences in answers to this question between G-20 members and non-members. Only 12% of countries that were NOT in the G-20 had a negative response to this question (summing up strongly disagree and disagree responses). By contrast, 58% of G-20 members had negative responses to this question (4% strongly disagree; 54% disagree). A central 8 challenge for the Lagarde years at the IMF will be to develop the means to reduce this perception.

Finally, what are some recommendations for the Fund moving forward as it attempts to create an organizational culture that is more surveillance-centered? First, more transparency is essential. Staff reports from the Article IV process are now released by almost every country in the world, but the survey data from this TSR may never be released. This is a missed opportunity for crowd sourcing, as additional analysis from academics and NGOs might produce some clear recommendations moving forward. A better understanding of how member countries view surveillance can only help to improve the finished product. Second, the famous line from Volcker and Gyohten (1992:143) remains clear today: “When the Fund consults with a poor and weak country, the country gets in line. When it consults with a big and strong country, the Fund gets in line.” The strategy moving forward for advanced economies needs to be based on a clear-eyed appraisal of how IMF advice can make a difference for these countries. It is only by providing resources that politicians find useful that the Fund can seek to strengthen its own influence. Finally, while there is a great deal of soul-searching at the Fund, there may not be learning. The Fund should look to other international organizations that are active in the area of surveillance to find some solutions to the problems it faces.

Sources Cited:

Momani, Bessma. 2006. Assessing the Utility of, and Measuring Learning from, Canada’s IMF Article IV Consultations. Canadian Journal of Political Science 39(2):249–269.

Volcker, Paul A. and Toyoo Gyohten. 1992. Changing Fortunes: The World’s Money and the Threat to American Leadership. New York: Times Books

Hyperlinks to TSR Documents:

Managing Director’s Action Plan

IMF Staff Reports

External Commentaries

External Studies

Report of the External Advisory Group

[1] Readers interested in accessing all of the reports in the review should refer to the table of hyperlinks on the last page.

[2] Momani (2006) found this in her study of the effectiveness of the Article IV process in Canada

Comments (0)

    Follow Us



    • Brooking's Event on Natural Resource Transparency
      June 11, 2014 
    • Achieving Inclusive Capitalism: Views from Exclusive Institutions
      May 29, 2014 
    • BLOG CALL: Inequality & Minimum Wage
      May 20, 2014 
    • Is there an IMF Governance Problem?
      May 02, 2014 
    • How to win the new debate about the IMF
      March 07, 2014 
    • The IMF and Reversing the Trend of Rising Inequality
      February 06, 2014 
    • Comments on Governance and Impact Report - Paul Blustein
      November 07, 2013 
    • Insight to CSO Views on Fiscal Transparency
      October 16, 2013 
    • The Wall Street Alchemist - How does financial innovation impact the real economy?
      April 23, 2013 
    • World Economic Forum Meetings Key Points and Briefings
      February 01, 2013 
    • Overexposed and Under-reported: Bank Disclosure of Risk
      January 18, 2013 
    • Sovereign Debt: Solving Insolvencies
      January 09, 2013 
    • Why is a Human Rights Approach Needed in Financial Regulation?
      December 11, 2012 
    • IMF's Re-engagement with Egypt: A New Economic Plan
      August 22, 2012 
    • Note on Quota Formula Review: Initial Considerations
      May 03, 2012 

    Share this page

    Join Our Mailing List
    For Email Newsletters you can trust