Chapter

CHAPTER 9 Institutions, Program Implementation, and Macroeconomic Performance

Author(s):
Alessandro Rebucci, and Ashoka Mody
Published Date:
April 2006
Share
  • ShareShare
Show Summary Details
Author(s)
Saleh M. Nsouli, Ruben Atoyan, and Alex Mourmouras1

This paper assesses empirically the links among a country’s institutions and political environment, its implementation of IMF-supported programs, and macroeconomic performance in a sample of 197 programs approved between 1992 and 2002. We find that a stronger institutional and political environment is associated with better macroeconomic outcomes, especially at longer time horizons. This direct beneficial impact of institutions on macroeconomic outcomes is in addition to their indirect impact through better program implementation. We also find that program implementation exerts an independent influence on macroeconomic outcomes, especially over shorter time horizons of up to two years. Better-implemented programs are associated with lower inflation and with initially weaker but ultimately stronger external and fiscal outcomes, but with a statistically insignificant impact on economic growth.

Introduction

The quality of a country’s institutions—broadly de fined to include the formal and informal rules of economic and political interactions—is a key determinant of sustainable economic progress. Weak institutions are behind many development failures (IMF, 2003; Rodrik, Subramanian, and Trebbi, 2002; and North, 1997). Many developing and transition economies participating in IMF-supported programs are bedeviled by corruption, weak and uneven enforcement of property rights, and other institutional failings. These institutional drawbacks have increasingly become the focus of concern for international financial and development institutions, which have made structural reform the subject of their conditionality.

Despite the significance of a country’s institutions for its macroeconomic performance, the empirical literature on IMF-supported reforms lacks a systematic quantitative assessment of their importance. Most evaluations of IMF-supported programs simply ignore the effect on macroeconomic performance of variation in institutions, either across countries or over time within a country. These evaluations are also generally inconclusive. Although inflation, the balance of payments, and the public finances seem to improve in countries that adopt IMF-supported programs, the impact on economic growth is ambiguous.2

The quality of a country’s institutions also shapes the extent to which it succeeds in implementing its IMF-supported programs. Program interruptions and the uneven record of implementation of some IMF-supported programs are rooted, at least in part, in weak institutions in the countries making use of IMF resources. Until recently, detailed data on program implementation were lacking. Most studies simply captured countries’ decision to participate in IMF-supported programs and did not consider how variation in program performance affected macroeconomic outcomes. This is an important omission, because a large proportion of IMF-supported programs are known to suffer major interruptions (see Ivanova and others, 2003 and Mecagni, 1999). Recently, the literature has begun to investigate quantitatively the links between institutions, program implementation, and macroeconomic performance (Ivanova and others, 2003; Joyce, 2003; and Dreher, 2004).

This paper sets out to measure the effect of variation in institutional quality on the macroeconomic performance of countries implementing IMF-supported programs. Building on the recent literature, it develops a statistical framework to assess empirically the links among a country’s institutions and political environment, its implementation of IMF-supported programs, and macroeconomic performance. We first update the results on program implementation reported in Ivanova and others, 2003 by adding the outcomes of 25 recent programs to the sample of that paper. The qualitative results regarding program implementation remain qualitatively similar: the rate of program interruption continues to be high, exceeding 40 percent. We then assess the effect of variation in institutions on program implementation and macroeconomic performance using four indicators of performance: inflation, economic growth, the balance of payments, and the fiscal balance. Our measures of program implementation come from the IMF’s Monitoring of IMF Arrangements (MONA) database, which contains detailed information on a large number of IMF-supported programs approved since the early 1990s. Information on borrowing countries’ institutions and domestic politics comes from the International Country Risk Guide (ICRG).

Our empirical framework is flexible, designed to take into account both the time-series properties of macroeconomic variables and the endogeneity of program implementation. In the data, inflation, growth, and most other macroeconomic indicators tend to be highly serially correlated and mean-reverting. We therefore examine the impact of institutions, politics, and program implementation, taking into account the autoregressive structure of the main macroeconomic and institutional variables, using a methodology suggested by the literature on the error correction mechanism. Instrumental variables are used to handle the endogeneity bias that exists because macroeconomic shocks also impact program performance.

Our findings are mixed. When the endogeneity of program implementation is properly accounted for, we find that institutions and program implementation both matter for macroeconomic performance. The response of macroeconomic variables to programs is often nonmonotonic, however. For example, although better-implemented programs are associated with lower inflation rates, the fiscal and external current account balances typically deteriorate for two years after program approval before they turn around. And, as in previous work, we could not detect statistically significant associations between program implementation and economic growth at any time during the three years following program approval.

The paper is structured as follows. The second section sets the stage for the empirical analysis. It describes the measures of program implementation and institutional development used and presents descriptive statistics. The third section describes the econometric methodology and main results. The fourth section provides a conclusion.

Measuring Program Implementation and Institutional Development

Overview

Disbursements of IMF loans are tied to prior actions by the recipient country, the observance of performance criteria, and the completion of program reviews—and thus to fulfillment of conditionality. Breaches of conditionality, if not followed by waivers because the breach was judged minor or temporary, or by required corrective action to keep the program on track, can lead to program interruptions. Following Ivanova and others (2003), we use two complementary measures of program implementation. The first measure captures the premature “cancellation” of an IMF-supported program. This index takes the form of a binary variable indicating whether a program experienced a major and irreversible interruption. An “irreversible” interruption occurs when either the last scheduled program review was not completed, or all scheduled reviews were completed but the subsequent annual arrangement was not approved. The second implementation measure is the ratio of disbursements to commitments. It is a continuous variable indicating the share of available IMF credit actually drawn. This measure contains information on actual program duration and the extent to which the IMF’s financial commitments under the program were fulfilled.

A variety of indicators can be used to assess the institutional and political setting in countries participating in IMF-supported programs. Based on country and time coverage and the ability to capture various aspects of governance, we choose to focus on the ICRG political risk indicators, which allow us to ascertain the short- and medium-term impacts of the political and institutional environment on economic performance and program implementation. Somewhat arbitrarily, we divide the 12 ICRG components into two groups. The first group proxies for basic institutional quality, protection of property rights, and contract enforcement. It includes indices for the Investment Profile, Corruption in Government, Law and Order, and the Quality of the Bureaucracy. The second group serves as a proxy for political outcomes. It is captured by the following variables: Government Stability, Socioeconomic Conditions, Internal Conflict, External Conflict, Military in Politics, Religious Tensions, Ethnic Tensions, and Democratic Accountability. These variables provide useful information about the internal and external political factors influencing program implementation and economic performance.3

Descriptive Statistics

Table 1 updates the results on program implementation presented in Ivanova and others (2003). In our sample of 197 IMF programs approved between 1992 and 2002, 41 percent of all programs (including precautionary arrangements) experienced an irreversible interruption, compared with about 44 percent reported in Ivanova and others (2003).

Table 1Program Implementation by Type of Arrangement
Including

Precautionary Arrangements
Excluding

Precautionary Arrangements
AllEFFESAF/

PRGF
SBAAllEFFESAF/

PRGF
SBA
Programs having irreversible interruptions141.1240.0045.3138.8942.7734.7845.3143.06
Number of observations1972564108159236472
Quantitative implementation index279.1887.2177.0978.5279.3686.9577.0978.85
Number of observations182246296151236266
Structural implementation index366.3773.9870.9760.5468.4176.5470.9762.44
Number of observations168246381142226357
Overall implementation index474.2983.2772.9172.8174.8183.7172.9173.45
Number of observations166236281141226257
Share of committed funds disbursed162.0572.5680.0248.4774.5478.8780.0268.02
Number of observations1932564104156236469
Source: IMF, Monitoring of IMF Arrangements (MONA) database.Notes: This table updates Table 1 in Ivanova and others (2003). Multiyear arrangements are treated as one program. Each cell contains the average percentage value of the implementation index that is based on a sample of programs approved between 1992 and 2002. SBA stands for Stand-By Arrangement; EFF stands for Extended Fund Facility; ESAF stands for Enhanced Structural Adjustment Facility; and PRGF stands for Poverty Reduction and Growth Facility.

The irreversible interruption index and the share of committed funds disbursed were computed as defined in the text.

The quantitative implementation index for a given macro performance criterion is equal to 100 percent if macro performance criterion was met or met after modification; and it is equal to zero if macro performance criterion was not met, not met after modification, waived, or waived after modification. The quantitative implementation index for a program is then computed as the average of those indices across all macro per formance criteria for this program.

The structural implementation index for a given structural condition is equal to 100 percent if structural condition was met or met with a small delay for structural benchmarks; it is equal to 50 percent if the structural condition was partially met or delayed for performance criteria; and it is equal to zero if the structural condition was not met. The structural implementation index for a program is then computed as the aver age of those indices across all structural conditions for this program.

The overall implementation index for a given program is the average of quantitative and structural implementation indices over all conditions in this program.

Source: IMF, Monitoring of IMF Arrangements (MONA) database.Notes: This table updates Table 1 in Ivanova and others (2003). Multiyear arrangements are treated as one program. Each cell contains the average percentage value of the implementation index that is based on a sample of programs approved between 1992 and 2002. SBA stands for Stand-By Arrangement; EFF stands for Extended Fund Facility; ESAF stands for Enhanced Structural Adjustment Facility; and PRGF stands for Poverty Reduction and Growth Facility.

The irreversible interruption index and the share of committed funds disbursed were computed as defined in the text.

The quantitative implementation index for a given macro performance criterion is equal to 100 percent if macro performance criterion was met or met after modification; and it is equal to zero if macro performance criterion was not met, not met after modification, waived, or waived after modification. The quantitative implementation index for a program is then computed as the average of those indices across all macro per formance criteria for this program.

The structural implementation index for a given structural condition is equal to 100 percent if structural condition was met or met with a small delay for structural benchmarks; it is equal to 50 percent if the structural condition was partially met or delayed for performance criteria; and it is equal to zero if the structural condition was not met. The structural implementation index for a program is then computed as the aver age of those indices across all structural conditions for this program.

The overall implementation index for a given program is the average of quantitative and structural implementation indices over all conditions in this program.

Countries with fewer program interruptions tend to have higher disbursement rates: the correlation co efficient between program interruptions and the disbursement share is −0.7. When precautionary arrangements are excluded, the average disbursement share is approximately 75 percent, compared with 71 percent in the sample examined by Ivanova and others (2003). The improvement in implementation reflects the fact that our sample contains more Stand-By Arrangements, which tend to have fewer interruptions than programs supported under the Extended Fund Facility and the Poverty Reduction and Growth Facility.

Improvements in the institutional climate, as reflected in higher ICRG indicators, are generally positively correlated with better program implementation, as measured by higher disbursement shares and fewer program interruptions. As in earlier studies (Dollar and Svensson, 2000; and Ivanova and others, 2003), greater government stability and a stronger investment profile in the year immediately preceding program approval are both associated with fewer interruptions (Table 2). On average, the risk of program interruption is much lower in environments in which governments are friendlier to inward foreign investment and are better able to carry out their programs. Improvements in the investment profile over a horizon of two years after the beginning of a program lead to significantly fewer interruptions. Lower corruption (a higher ICRG score) and improvements in socioeconomic conditions in the year after program approval are also associated with better implementation of conditionality as measured by the quantitative implementation index.

Table 2Correlation of Program Implementation Indices with ICRG Risk Ratings at Different Horizons
Panel A.

Correlation with

T−1 Risk Ratings
Panel B.

Correlation with

T+1 Risk Ratings
Panel C.

Correlation with

Change in Risk Ratings

from T−1 to T+1
Irreversible

interruption

index
Disbursement

share1
Irreversible

interruption

index
Disbursement

share1
Irreversible

interruption

index
Disbursement

share1
Bureaucracy quality−0.026−0.127−0.016−0.072−0.0260.042
Corruption−0.0900.027−0.006−0.0280.119−0.078
Democratic accountability−0.045−0.089−0.038−0.062−0.014−0.034
Ethnic tensions−0.0040.050−0.0240.1010.0080.038
External conflict0.047−0.183*0.025−0.032−0.0200.143
Government stability−0.191**−0.033−0.0670.0110.0760.008
Internal conflict0.093−0.1600.0250.006−0.0980.142
Investment profile−0.199**0.045−0.274***0.167*−0.185**0.058
Law and order−0.025−0.167*0.034−0.0540.0720.102
Military in politics−0.065−0.059−0.019−0.0350.067−0.047
Religious tensions−0.0900.028−0.1150.097−0.0680.102
Socioeconomic conditions−0.0320.162*−0.1240.187−0.0930.049
Notes: The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively.

Correlation coefficients with the share of committed funds disbursed are computed excluding precautionary arrangements.

Notes: The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively.

Correlation coefficients with the share of committed funds disbursed are computed excluding precautionary arrangements.

The political and institutional climate in countries in which programs are interrupted varies systematically from that in countries in which programs are completed. Figure 1 plots the average change in each of the ICRG variables during the program relative to the last preprogram year, distinguishing between interrupted and uninterrupted programs. Program interruptions are associated with less progress in improving the investment climate and in improving the quality of the bureaucracy, and with intensified internal conflict. Successful program implementation is associated with greater initial influence of the military, followed by a significant reduction in subsequent years. Program interruptions tend to be accompanied by sharp increases in the military’s involvement in the third and fourth years after program approval.

Figure 1Quality of Institutions in Interrupted and Completed Programs

Countries completing IMF programs appear to be more successful in reducing inflation than countries that experience program interruptions, as reflected in mean changes in macroeconomic outcomes between period T− 1 and five different horizons (Figure 2). Uninterrupted programs are also associated with sharp improvements in fiscal balances in the first year of the program, followed by a gradual deterioration in subsequent years. On the other hand, countries whose programs are interrupted register very modest improvements in fiscal balances initially but then catch up with the others. The external current account balance improves in countries whose programs do not get interrupted. Interrupted programs are associated with slight improvements in the current account in the year immediately following program approval, followed by steady deterioration.

Figure 2Macroeconomic Performance in Interrupted and Completed Programs

Correlation analysis and comparisons of macroeconomic performance in completed versus interrupted programs, although suggestive, mask a great deal of variability in the data. Each country starts from different initial economic, institutional, and political conditions. While they are engaged in IMF-supported programs, countries are subject to a variety of external and internal shocks that influence macroeconomic outcomes and program implementation. We now turn to a more rigorous econometric methodology to properly take into account this broad spectrum of country-specific effects.

Econometric Methodology and Results

Methodology

Most empirical research on the macroeconomic impacts of IMF-supported programs relies on panel data. Our framework, by contrast, relies on a pooled dataset in which each program is treated as an independent observation in the context of a statistical model that takes into account the short-run autoregressive, mean-reverting nature of macroeconomic variables. We assess the impact of program implementation on growth, inflation, and the fiscal and external balances once country-specific institutional and political effects are taken into account. Similar to the “before-and-after approach,” our approach compares macroeconomic outcomes under the program with those in the last preprogram year for a sample of countries that chose to participate in such programs. We do not address issues related to sample selection bias, i.e., the systematic differences between countries that agree to participate in IMF-supported programs and those that do not. Our focus is on a narrower question. Given that certain countries do self-select into the “treatment” of IMF-supported programs, we ascertain the relative impacts of program implementation and institutions on macroeconomic outcomes.

Consider Mi,T, a macroeconomic variable observed at time T in country i. Since we consider macroeconomic development only in countries implementing IMF-supported programs, the index i is also a unique country-program identifier. Following Atoyan and others (2004), the evolution of Mi,T can be represented by

where Xi,T −1 is a vector of noninstitutional forcing variables at time T−1 that also includes a random term in time T, IMPLi is the measure of program implementation in country i, INSTi,T−1 is a vector of domestic political and institutional initial conditions, ΔINSTi,T is a vector of contemporaneous changes in country i’s political and institutional environment, and f(∙) is the reduced-form data-generating process.

Hypothesizing that macroeconomic variables are influenced by their own values in previous periods, because of institutional or psychological inertia, we assume that growth, inflation, and the current account and fiscal balances follow a finite-order autoregressive process. Assuming second-order autoregression, the reduced-form model can be conveniently written in the following form:

where εi,TM is a stochastic disturbance to M. Equation (2) is the “autoregressive and mean-reversion form,” as it includes both lagged differences and the lagged level as the regressors for the current first difference of the variable M. It captures the autoregressive structure of M via the first-difference term ΔMi, T−1. The adjustment of M in response to deviations from its “normal” historical value is captured via the mean-reversion term Mi, T−1. The coefficient γ2 is a partial adjustment coefficient. It shows what percentage of the deviation from the long-run equilibrium will be covered each year following the deviation. Note that because γ2 =(β1 + β2 − 1) with β1 and β2 representing the autoregressive (AR) parameters in the underlying AR(2) process, small negative values of the coefficient are consistent with M being highly persistent.

Equation (2) holds for all T and i, including periods in which IMF-supported programs are in effect. As these programs are designed to improve macroeconomic performance, equation (2) incorporates their impacts into the model. Equation (2) also captures the impact of institutional and political conditions on the macroeconomy. In implementing equation (2), we treat institutional and political developments during the IMF-supported program as exogenous and mean-reverting. Including first differences and lagged levels of institutional variables in the regressions assumes that there are long-run levels of institutional development and that deviations from these levels are temporary.4 This view of institutions is certainly valid in analyzing short-term programs such as Stand-By Arrangements. It is probably less appropriate for programs with greater structural orientations, such as those supported under the Extended Fund Facility and the Poverty Reduction and Growth Facility, which aim to improve the supply response of the economy. The nature of institutional change that takes place in the context of IMF-supported programs is ultimately an empirical question. In the event, there is little correlation in the data between program implementation and institutional development (see Table 2, Panel C).5

We estimate the following system of four equations, one for each macroeconomic outcome variable:

Note that, in equations (3), monetary and fiscal policies are kept in the background. Program implementation serves as a proxy for the impact of macroeconomic policies on macroeconomic outcomes. As in Ivanova and others (2003) and Dollar and Svensson (2000), the probability of implementation of an IMF-supported program is related to the underlying political and institutional factors in the borrowing country, to the IMF financial and human resource effort in the program, and to initial economic conditions in the country. Although the probability of program implementation is unobservable, it is related to the observable implementation index:

In equation (4), the 8s are vectors of coefficients. INITIALi is a vector of initial conditions represented by the preprogram values of real GDP per capita, inflation, the GDP growth rate, the current account balance, and the fiscal balance. FUNDi is a vector of program-specific variables that are important in determining program outcomes. These variables are either directly under IMF control or provide information about the nature of the relationship between the country and the IMF. Our regression approach in equation (4) is similar to that used in Ivanova and others (2003).

Since we are interested in several potentially mean-reverting macroeconomic indicators, a vector error correction model (VECM) could be considered. In equation (2), M would then represent a 4x1 vector of variables (inflation, growth rate, fiscal balance, and current account balance). We pursued this approach by estimating the augmented version of the VECM and comparing results with the ones obtained from estimating equations (3). The results confirm the existence of a long-run relationship among some of the macroeconomic variables.6 On the other hand, the marginal benefit of incorporating this information into our analysis, which focuses on the relative importance of program implementation and institutional factors for macroeconomic outcomes, seems small. If a VECM representation is adopted, the testing down approach on the institutional and political factors yields identical model specification to the one we already have. The estimated coefficients and their significance levels change only marginally relative to those obtained by considering only a variable’s own autoregressive and mean-reversion terms. To simplify the presentation and economize on degrees of freedom, we do not present VECM results.7 These are available on request from the authors.

The properties of the ordinary-least-squares (OLS) estimator in equations (3) depend on the stochastic properties of the explanatory variables, and in particular on whether or not they are distributed independently of the disturbance term. In addition, shocks to macroeconomic outcomes are likely to impact program implementation, implying that Corr(IMPLi,εi,TM)0. Consequently, the OLS estimator is likely to be biased.8

We employ two related instrumental variable (IV) techniques to correct for potential endogeneity bias. One is the two-stage-least-squares (2SLS) procedure, where we first regress the program implementation measure on the exogenous variables and a set of instruments that are correlated with the implementation measure but are not related to the error terms in equations (3). In the second stage, we estimate the system of equations (3) by OLS using the predicted values of the implementation measures instead of the actual ones.

In general, it is difficult to find instruments that are related to program implementation but do not systematically affect economic performance. The best candidates are variables that describe the nature of the relationship between member countries and the IMF: a country’s quota in the IMF; the cumulative time spent in an IMF-supported program (number of months in program mode since 1980); the amount approved in relation to the country’s IMF quota; and the dollar cost of the program starting six months before program approval.9

Our second IV procedure is 3SLS. This has the advantage of incorporating information from the cross-correlations of the error terms in equations (3) and producing sharper (more efficient) parameter estimates.10 To arrive at 3SLS estimates, we use the 2SLS estimates to obtain an estimate of the contemporaneous variance-covariance matrix of the errors in equations (3). Applying the generalized-least-squares method to the transformed single-equation representation of the system yields 3SLS estimates, which are consistent and asymptotically more efficient than 2SLS estimates.

Model Specification

There is a broad consensus that domestic institutions and politics are key determinants of economic performance in countries borrowing from international financial institutions. There is less agreement on precisely which aspects of the institutional and political environment are especially important. Although all the ICRG indices could be included in the regression analysis, this would lead to collinearity problems and a loss of precision. On the other hand, omitting relevant institutional and political variables would lead to biased estimates.

This dilemma dictates a parsimonious approach to model specification. We use changes in macroeconomic variables over a one-year horizon following program approval as a testing horizon. This implicitly assumes that if a certain institution or political feature is important at high frequencies, it will also be influential over longer horizons. This strategy produces results that are robust with respect to the choice of program implementation measure.

Our model specification technology is described as a “testing down” approach. We start with an unrestricted model that includes all ICRG indices as regressors and then simplify it in light of sample evidence. Specifically, we estimate each of the equations in (3) separately while systematically dropping regressors with low t-statistics. The adjusted R2 is used as an additional consideration in model selection.

Our results indicate that inflation in program countries is influenced considerably by such institutional factors as the prevalance of law and order, the quality of the bureaucracy, and the country’s investment climate. On the political side, only variations in ethnic tension and internal conflict appear to matter for inflation. Economic growth is affected by the investment profile, government stability, and initial socioeconomic conditions.11 Corruption in the political system, democratic accountability, ethnic tension, external conflict, and military involvement in politics are important for the evolution of fiscal balances. Finally, corruption, ethnic tension and external conflict, government stability, the in vestment climate, and military participation in the country’s political life have significant impacts on the evolution of the current account.

Main Findings

What Determines Program Implementation?

Table 3 presents first-stage regressions of the implementation measures on initial economic conditions, ICRG indicators during the year preceding program approval, and our instruments. To obtain the predicted values used in the second-stage regressions, we employ the complete model (columns 1 and 3). However, to overcome the collinearity problem discussed above, the discussion in the rest of this subsection relies on estimates (columns 2 and 4 of Table 3) that drop some of the ICRG indices that appear to be insignificant.

Table 3Determinants of Program Implementation: First-Stage Regressions
Dependent VariableDisbursement

Share1,3,4
Irreversible Interruption

Index2,3,4
Regression number(1)(2)(3)(4)
Intercept0.680**0.848***−0.397−0.085
Initial per capita real GDP−0.012−0.013−0.066−0.072
INFL (T−1)0.00020.00020.0020.002
GCBY (T−1)0.1160.209−3.509−3.230
BCAY (T−1)0.0910.049−0.753−1.198
GROWTH (T−1)−0.002−0.0030.0460.056*
Bureaucracy quality (T−1)−0.017−0.070
Corruption (T−1)0.0060.322*0.358**
Democratic accountability (T−1)−0.019−0.104
Ethnic tensions (T−1)0.073**0.076**0.097
External conflict (T−1)0.006−0.034
Government stability (T−1)0.071*0.078**0.044
Internal conflict (T−1)−0.061**−0.062***−0.221*−0.217**
Investment profile (T−1)0.0210.090
Military in politics (T−1)0.0410.044*0.101
Religion tensions (T−1)0.0190.1440.189
Socioeconomic (T−1)−0.0080.040
Law and order (T−1)−0.066−0.069*−0.092
Fund effort per program year0.0410.043*0.0360.050
Fund quota (log)−0.034−0.0460.0950.052
Number of months spent in IMF programs0.002**0.002**−0.002−0.0003
Amount approved as a fraction of quota0.024**0.025**0.0880.094
Dummy for precautionary arrangement−1.121***−1.109***
Observations115115115115
Log likelihood−14.695−15.676−61.998−63.251
Correlation coefficient/correctly predicted (percent)0.8070.80375.6675.66

Notes: The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively.

Results are obtained using the tobit model: y=max (X′ β+ε,0).

Results are obtained using the probit model. Parameter estimates are computed to reflect the probability of no irreversible interruption: Pr(Interruption = 0) = F(X′β), where F is normal cumulative distribution function.

The chi-square statistics for the estimated parameters are available from the authors upon request.

All regressions include year dummies.

Notes: The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively.

Results are obtained using the tobit model: y=max (X′ β+ε,0).

Results are obtained using the probit model. Parameter estimates are computed to reflect the probability of no irreversible interruption: Pr(Interruption = 0) = F(X′β), where F is normal cumulative distribution function.

The chi-square statistics for the estimated parameters are available from the authors upon request.

All regressions include year dummies.

When the share of committed funds disbursed is used as a measure of program implementation, none of the variables reflecting initial economic conditions is significantly different from zero. This could suggest that programs are tailored to participating countries’ circumstances or that their outcomes are independent of initial economic conditions (see Ivanova and others, 2003).

Reduced ethnic tension and greater government stability before program approval improve program implementation. Coefficients on the ICRG ratings of ethnic tensions and government stability in the year preceding program approval are positive and significant. In addition to a larger proportion of funds being disbursed in countries where racial and ethnic tensions are less pronounced, better program implementation is positively correlated with the general public’s perception of a government’s ability to carry out its declared programs. Other factors remaining the same, a one-point increase in either rating raises disbursements by about 8 percent.

Reductions in internal conflict and improvements in law and order in the year before program approval are associated with lower disbursements. The coefficient on the initial level of the internal conflict index is negative and highly significant. The magnitude of the effect is rather large: a one-point increase in the rating would lower disbursements by just over 6 percent. The coefficient on the initial level of the law and order rating is also negative and significant. These results may reflect the IMF’s financial involvement in countries where observance of the law is not very good initially, often because the countries are recovering from conflict.

There is some evidence that greater initial involvement by the military in politics is associated with lower disbursements of IMF financing. The coefficient on the corresponding ICRG index is positive and significant.

Countries with a history of IMF-supported programs seem to have higher disbursement shares. Every additional month spent in IMF-supported programs translates into 0.2 percent more funds disbursed. Taken literally, higher disbursement ratios could manifest better program design and implementation, and the length of IMF engagement simply reflects the long-term nature of the needs of these low-and middle-income countries. But the reasons for—and results of—prolonged financial association between member countries and the IMF are complex (see IMF, 2002 for a recent evaluation).

The size of programs, as measured by the amount of IMF financing committed in relation to a country’s quota, appears to be important in determining program outcomes. Countries with larger programs tend to have higher disbursement shares. These packages are often provided in response to capital account emergencies. They require not only more financing but also greater front-loading of assistance than suggested by usual IMF phasing rules.

The IMF’s effort at program design and implementation, as measured by staff hours and the dollar cost of staff resources, is only marginally important in raising a program’s prospects of success. Although larger quotas have an ambiguous net effect on program implementation a priori (see Box 1), the coefficient on the country’s IMF quota is negative, suggesting that the implementation of IMF-supported programs could be weaker in countries with larger IMF quotas.

Our findings are broadly similar when the interruption index is used as the measure of program implementation. Almost all the variables describing the initial economic conditions of participating countries have insignificant coefficients. The only exception is the lagged level of a country’s growth rate, which has a marginally significant coefficient. This can be interpreted as evidence that countries that were growing relatively fast before program initiation are less likely to have an irreversible interruption of the program.

Reduced government corruption has a strikingly positive impact on the probability of successful program implementation. The coefficient on the preprogram level of corruption is positive and significant, and its magnitude is impressive. On average, a one-point improvement in the ICRG corruption index, all other determinants of program success held constant, coincides with a 35.8 percent better chance of having no program interruption.

Box 1.List of Instrumental Variables

The outcomes of IMF-supported programs are endogenous. Instrumental variables (IVs) help us obtain unbiased estimates of the impact of IMF-supported programs on the economic performance of participating countries. The instruments must be correlated with program implementation (lack of program interruptions and the share of committed funds disbursed) and not be direct determinants of the economic policy outcomes (inflation, economic growth, fiscal balance, and current account). The following IVs are used in the analysis:

IMF quota (log). A country’s quota determines the member’s voting power in the IMF. Countries with larger quotas have more bargaining power and systemic importance in the world economy. Greater bargaining power could allow countries to extract more concessions from the IMF, leading to less conditionality and more lenient IMF treatment. The coefficient on the IMF quota in the implementation measure regressions would then be positive. On the other hand, the size of the quota also reflects a country’s systemic importance in the world economy and its access to international capital markets. Governments of large countries might be less cooperative with IMF conditionality if the perceived political costs were too high. In that case, the parameter estimate could have a negative sign.

Number of months spent in IMF programs since 1980. This variable captures the extent of a country’s financial involvement with the IMF. The length of the country’s history under IMF-supported programs could lead, through learning-by-doing, to better program design and higher implementation rates as government officials and IMF staff gain more experience and knowledge of country-specific factors and IMF procedures.

Amount approved as a fraction of IMF quota. This variable is expected to capture the financial importance of a particular program. Large values would be positively correlated with the severity of crises and the willingness of the authorities to implement IMF-supported reforms.

IMF effort per program year, including six months prior to program approval. This is a direct measure of the dollar cost of IMF programs. It is computed from the IMF’s Budget Reporting System data on hours spent by staff on program implementation and estimated average staff salaries by grade. More effort invested in program implementation is expected to be positively correlated with program implementation.

As in the regressions using the disbursement share as the measure of program implementation, the co efficient on the initial level of internal conflict is negative and significant, and for similar reasons. The coefficient on the preprogram level of political violence is negative: an improvement in this rating by one point is associated with a 21.7 percent higher chance of an irreversible interruption.

With the exception of the coefficient on the number of months spent in program mode, the variables characterizing the relationship between a country and the IMF enter the regression with the expected signs. However, none of the coefficients is significantly different from zero. As in Ivanova and others (2003), this result suggests that the implementation of IMF-supported programs is largely determined by the country’s domestic political economy and institutions. Variables under IMF control have only a marginal impact on program outcomes.

What Determines Macroeconomic Outcomes?

This subsection summarizes the empirical links among macroeconomic performance, the institutional and political environment, and program implementation (Tables 4 through 11). In all the regressions in these tables, the dependent variable is the change in the macroeconomic outcome between period T−1 (the preprogram year) and the end of the first, second, or third years after program approval (T, T+1, or T+2). Each table reports OLS, 2SLS, and 3SLS estimates, using the disbursement share or lack of program interruptions as the measure of program performance. Unless otherwise noted, in what follows we will refer to results obtained using the 3SLS procedure and the disbursement share as the measure of program implementation.

Table 4Program Implementation, Institutions, and Inflation: Regressions Using the Share of Committed Funds Disbursed
Dependent VariableΔINFL (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Disbursement share13.862−55.6592.778−140.87−90.624−9.000−150.828*−103.740−12.621
RGDPPC (T−1)−8.361**−6.406*−0.277−10.622***−7.222*−0.643−10.602***−7.013*−0.474
ΔINFL (T−1)−0.1580.0003−0.001−0.214**−0.002−0.001−0.180*−0.004−0.002
INFL (T−1)−0.731***−0.958***−0.976***−0.682***−0.950***−0.977***−0.739***−0.954***−0.975***
Bureaucracy quality (T−1)−26.488**−12.571−4.100−27.744**−13.581−4.317*−27.964**−12.012−5.537***
Change in bureaucracy quality (T−1 to horizon)−46.700−11.681−2.909−73.494*−14.381−4.211−74.746**−14.256−3.954
Ethnic conflict (T−1)−15.834−5.164−1.969−6.632−4.066−1.299−4.275−2.152−1.034
Change in ethnic conflict (T−1 to horizon)−5.246−12.083−3.455−11.568−12.110−4.060−12.868−12.904−4.606*
Internal conflict (T−1)13.151**7.4781.0336.9746.1490.0616.6057.6580.330
Change in internal conflict (T1 to horizon)−17.336*−11.734−0.425−23.794**−12.419−0.154−24.206**−13.982*−0.461
Investment profile (T−1)−9.8343.493−0.809−2.8543.715−0.110−2.7843.4560.487
Change in investment profile (T−1 to horizon)−17.426−3.723−1.623−19.212−7.275−1.851−19.558−9.327−1.566
Law and order (T−1)12.7385.8625.700**14.4708.7307.348***12.3425.6396.603***
Change in law and order (T−1 to horizon)−87.059***−34.2771.430−86.529***−30.4692.516−84.579***−22.8594.800*
Observations123122116115115109115115109
R20.5860.5960.9820.6260.5920.9830.6250.5900.983

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Table 5Program Implementation, Institutions, and Inflation: Regressions Using Irreversible Interruption Index
Dependent VariableΔINFL (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Non-interruption dummy1.364−18.0193.400−38.40928.53216.575*−32.15229.72613.730
RGDPPC (T−1)−7.688**−6.026*−0.262−10.512***−6.447−0.329−10.182***−6.241−0.195
ΔINFL (T−1)−0.165−0.002−0.001−0.207*−0.0010.001−0.172−0.003−0.001
INFL (T−1)−0.724***−0.957***−0.977***−0.706***−0.986***−0.991***−0.769***−0.992***−0.984***
Bureaucracy quality (T−1)−25.676**−14.196−4.103−25.351*−13.776−4.125*−25.928**−12.195−5.463**
Change in bureaucracy quality (T−1 to horizon)−38.449−13.225−2.957−63.094*−13.866−3.911−66.295*−14.974−4.020
Ethnic conflict (T−1)−12.467−8.963−1.694−10.522−11.185−2.043−7.869−10.586−2.102
Change in ethnic conflict (T−1 to horizon)−5.445−12.690−3.344−7.393−10.723−3.092−8.472−12.180−3.885
Internal conflict (T−1)12.615**8.9281.0828.37510.6391.2118.67712.8221.279
Change in internal conflict (T−1 to horizon)17.151*12.357*−0.48119.309*12.131−0.50120.607**14.480*−0.637
Investment profile (T−1)−8.5691.037−0.842−4.825−1.951−1.231−6.345−2.885−0.315
Change in investment profile (T−1 to horizon)−16.780−4.559−1.737−17.045−7.490−1.706−18.050−9.764−1.355
Law and order (T−1)11.0338.0052.480**18.1719.7676.592**15.3866.7726.285***
Change in law and order (T−1 to horizon)−85.731***−36.9791.589−83.905***−34.7452.187−82.277***−28.3454.010
Observations126123116115115109115115109
R20.5820.5890.9820.6170.5890.9840.6140.5870.983

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Table 6Program Implementation, Institutions, and Growth: Regressions Using the Share of Committed Funds Disbursed
Dependent VariableΔGROWTH (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Disbursement share2.814*2.331*1.4122.6070.4842.9212.5790.6392.309
RGDPPC (T−1)0.2090.1690.551***0.277*0.0770.515***0.266*0.0720.534***
ΔGROWTH (T−1)−0.205**−0.203***−0.017−0.249***−0.248***−0.023−0.210***−0.212***−0.033
GROWTH (T−1)−0.850***−0.785***−0.946***−0.829***−0.758***−0.931***−0.840***−0.753***−0.896***
Government stability (T−1)0.4710.4630.0490.1630.3690.097−0.0250.299−0.076
Change in government stability (T−1 to horizon)1.185**0.3640.0251.022*0.3390.0621.117**0.305−0.250
Investment profile (T−1)−0.5780.2160.634−0.623−0.0490.463−0.568−0.0420.462
Change in investment profile (T−1 to horizon)0.963*1.079***0.841***1.228**0.764**0.849**1.029**0.678**0.857**
Socioeconomic (T−1)−0.536−0.607−0.013−0.525−0.4670.019−0.420−0.529−0.036
Observations125124118115115109115115109
R20.6870.7440.6600.7040.7440.6410.7010.7430.638

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Table 7Program Implementation, Institutions, and Growth: Regressions Using Irreversible Interruption Index
Dependent VariableΔGROWTH (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Non-interruption dummy0.5550.8250.195−0.443−0.388−0.321−0.544−0.529−0.268
RGDPPC (T−1)0.1440.1770.566***0.2040.0810.536***0.1900.0700.525***
ΔGROWTH (T−1)−0.199**−0.208***−0.039−0.250***−0.242***−0.035−0.211**−0.208***−0.050
GROWTH (T−1)−0.857***−0.789***−0.931***−0.829***−0.757***−0.920***−0.842***−0.759***−0.898***
Government stability (T−1)0.3540.4830.0950.0940.3690.091−0.1030.300−0.058
Change in government stability (T−1 to horizon)0.894*0.4000.0800.8730.3500.0630.992*0.326−0.210
Investment profile (T−1)−0.3780.1370.480−0.335−0.1270.374−0.306−0.0850.334
Change in investment profile (T−1 to horizon)1.120**1.013***0.784***1.372**0.700**0.736**1.143**0.620**0.738**
Socioeconomic (T−1)−0.398−0.639*−0.078−0.481−0.494−0.049−0.370−0.543−0.107
Observations128125118115115109115115109
R20.6770.7360.6490.6920.7420.6310.6900.7400.629

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Table 8Program Implementation, Institutions, and Fiscal Balance (Ratio to GDP): Regressions Using the Share of Committed Funds Disbursed
Dependent VariableΔGCB_Y (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Disbursement share0.024*0.0200.013−0.013−0.039**−0.021−0.024−0.038*−0.019
RGDPPC (T−1)0.0020.0003−0.00060.002**−0.001−0.0010.002*−0.001−0.001
ΔGCB_Y (T−1)0.156*−0.349***−0.1310.161**−0.323***−0.0960.100−0.321***−0.059
GCB_Y (T−1)−0.623***−0.523***−0.434***−0.651***−0.497***−0.527***−0.451***−0.472***−0.558***
Corruption (T−1)−0.010***−0.005−0.006−0.009**−0.006−0.006−0.006*−0.006*−0.006
Change in corruption (T−1 to horizon)−0.0080.0050.002−0.0080.0030.003−0.0030.002−0.001
Democratic accountability (T−1)0.0010.0020.0070.002−0.003−0.001−0.002−0.0020.002
Change in democratic accountability (T−1 to horizon)−0.008−0.0070.0007−0.009*−0.008**−0.004−0.008*−0.008**0.0004
Ethnic conflict (T−1)0.001−0.005−0.0040.001−0.002−0.0010.001−0.002−0.002
Change in ethnic conflict (T−1 to horizon)−0.015**0.0006−0.005−0.016**−0.003−0.006−0.015**−0.004−0.007
External conflict (T−1)0.004*0.005*0.0040.0020.0030.0030.0030.0030.004
Change in external conflict (T−1 to horizon)0.009***0.0040.0050.006*0.0020.0010.006**0.0030.003
Military in politics (T−1)0.004**0.0030.0020.0030.005**0.0030.0030.004*0.003
Change in military in politics (T−1 to horizon)0.0060.0007−0.0050.008−0.001−0.0060.001−0.001−0.005
Observations119118112115115109115115109
R20.5100.6030.3920.5050.6390.4560.4720.6370.443

Notes: OLS denotes ordinary least squares; 2SLS denotes two–stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the esti mated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two–stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the esti mated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Table 9Program Implementation, Institutions, and Fiscal Balance (Ratio to GDP): Regressions Using the Irreversible Interruption Index
Dependent VariableΔGCB_Y (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Non-interruption dummy0.016***0.014**0.008−0.010−0.023−0.035*−0.022−0.020−0.032*
RGDPPC (T−1)0.002*0.001−0.0010.002*−0.001−0.0010.002−0.001−0.001
ΔGCB_Y (T−1)0.190**−0.354***−0.1410.168**−0.308***−0.0890.099−0.310***−0.057
GCB_Y (T−1)−0.576***−0.534***−0.433***−0.632***−0.508***−0.522***−0.436***−0.484***−0.558***
Corruption (T−1)−0.012***−0.007*−0.007−0.008*−0.003−0.002−0.004−0.004−0.004
Change in corruption (T−1 to horizon)−0.0080.0030.001−0.0070.0030.004−0.0020.001−0.0003
Democratic accountability (T−1)0.0020.0030.0070.002−0.003−0.002−0.002−0.0020.002
Change in democratic accountability (T−1 to horizon)−0.006−0.008*0.0002−0.008−0.008**−0.003−0.007*−0.008**0.00003
Ethnic conflict (T−1)0.002−0.004−0.0040.002−0.003−0.0020.001−0.003−0.002
Change in ethnic conflict (T−1 to horizon)−0.014**0.001−0.005−0.016**−0.004−0.006−0.015**−0.004−0.007
External conflict (T−1)0.003*0.005**0.0050.0020.0030.0020.0030.0030.004
Change in external conflict (T−1 to horizon)0.008***0.005*0.0050.007**0.0020.0010.007**0.0030.003
Military in politics (T−1)0.005**0.0030.0020.0030.004**0.0030.004*0.004*0.002
Change in military in politics (T−1 to horizon)0.0080.003−0.0040.008−0.002−0.0060.001−0.001−0.005
Number of observations122119112115115109115115109
R20.4990.6090.3920.4920.6340.4710.4580.6310.459

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Table 10Program Implementation, Institutions, and Current Account (Ratio to GDP): Regressions Using the Share of Committed Funds Disbursed
Dependent VariableΔBCA_Y (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Disbursement share0.0080.0220.060***−0.082**−0.0390.024−0.080**−0.0380.020
RGDPPC (T−1)−0.0008−0.0010.0006−0.002−0.003−0.0001−0.002−0.003−0.0001
ΔBCA_Y (T−1)−0.145−0.172*−0.118−0.215**−0.195*−0.155−0.160*−0.190*−0.157
BCA_Y (T−1)−0.157**−0.217**−0.371***−0.149**−0.125−0.394***−0.114*−0.143*−0.435***
Corruption (T−1)−0.011−0.005−0.014*−0.007−0.002−0.016**−0.006−0.002−0.017**
Change in corruption (T−1 to horizon)−0.002−0.0050.0090.002−0.001−0.0020.002−0.003−0.002
Ethnic conflict (T−1)−0.008*−0.008−0.002−0.006−0.0070.001−0.006−0.0070.001
Change in ethnic conflict (T−1 to horizon)−0.010−0.002−0.008−0.016*−0.016*−0.006−0.016*−0.015−0.006
External conflict (T−1)0.0030.0001−0.00010.001−0.003−0.0030.001−0.002−0.003
Change in external conflict (T−1 to Horizon)0.008−0.0008−0.0050.003−0.002−0.010**0.004−0.002−0.012**
Government stability (T−1)0.014***0.013*0.0100.009*0.0100.0020.008*0.0100.002
Change in government stability (T−1 to horizon)−0.0040.005−0.002−0.0050.005−0.007−0.0030.004−0.007
Investment profile (T−1)−0.014***−0.013**−0.009*−0.009**−0.010*−0.006−0.008**−0.011**−0.006**
Change in investment profile (T−1 to horizon)−0.016***−0.018***−0.007*−0.016***−0.015***−0.003−0.019***−0.016***−0.003
Military in politics (T−1)0.011***0.012***0.010**0.007**0.010**0.008*0.007**0.010**0.008*
Change in military in politics (T−1 to horizon)0.0040.003−0.0020.0060.003−0.0010.0050.004−0.001
Number of observations124123117115115109115115109
R20.3920.3790.4430.4600.4070.4510.4600.4060.449

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept, precautionary arrangement, and year dummies.

Table 11Program Implementation, Institutions, and Current Account (Ratio to GDP): Regressions Using Irreversible Interruption Index
Dependent VariableΔBCA_Y (T−1 to horizon)
Regression number(1)(2)(3)(4)(5)(6)(7)(8)(9)
OLS2SLS3SLS
HorizonTT+1T+2TT+1T+2TT+1T+2
Non-interruption dummy0.0020.0100.024**−0.037−0.0240.032−0.032−0.0240.030
RGDPPC (T−1)−0.001−0.0010.0008−0.002−0.0030.001−0.002−0.0030.001
ΔBCA_Y (T−1)−0.133−0.172*−0.105−0.235**−0.197*−0.180−0.173*−0.192*−0.180*
BCA_Y (T−1)−0.158**−0.208**−0.366***−0.136**−0.115−0.365***−0.105*−0.135*−0.399***
Corruption (T−1)−0.012*−0.007−0.019**−0.0040.0004−0.019**−0.003−0.0001−0.020**
Change in corruption (T−1 to horizon)0.0002−0.0050.0070.0004−0.00040.0001−0.0005−0.0020.0002
Ethnic conflict (T−1)−0.006−0.0050.002−0.006−0.0060.003−0.006*−0.0060.003
Change in ethnic conflict (T−1 to horizon)−0.011−0.002−0.007−0.015−0.016*−0.004−0.015−0.014−0.004
External conflict (T−1)0.0030.00030.00060.002−0.002−0.0020.002−0.002−0.002
Change in external conflict (T−1 to horizon)0.008*0.0004−0.0030.005−0.001−0.009**0.006−0.001−0.010**
Government stability (T−1)0.013**0.014**0.0100.009*0.0100.0010.008*0.0100.001
Change in government stability (T−1 to horizon)−0.0030.006−0.003−0.0060.005−0.007−0.0030.005−0.008
Investment profile (T−1)−0.011***−0.011**−0.005−0.009**−0.008*−0.005−0.009***−0.009*−0.005
Change in investment profile (T−1 to horizon)−0.016***−0.018***−0.006−0.014***−0.014***−0.002−0.019***−0.015***−0.002
Military in politics (T−1)0.011***0.012***0.010**0.006*0.010**0.008*0.007**0.010**0.008**
Change in military in politics (T−1 to horizon)0.0040.0050.00010.0060.003−0.0010.0060.004−0.0003
Number of observations127124117115115109115115109
R20.3850.3680.4160.4430.3950.4480.4300.3930.447

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Notes: OLS denotes ordinary least squares; 2SLS denotes two-stage least squares; and 3SLS denotes three-stage least squares. The symbols *, **, and *** denote significance at the 90, 95, and 99 percent confidence levels, respectively. Standard deviations and t-statistics for the estimated parameters are available from the authors upon request. All regressions include intercept and year dummies.

Inflation

Inflation is highly persistent in program countries. The coefficients on lagged inflation are highly significant for all horizons (Tables 4 and 5). For the average program, about three quarters of any deviation from “normal” inflation after a program is approved is reversed within a year. Deviations of inflation from its long-run equilibrium are erased almost completely in three years.

In contrast to many other studies, which were unable to link IMF-supported programs with price stability, our findings represent reasonable evidence that better program implementation leads to lower inflation. After correcting for endogeneity bias, the coefficients on the disbursement share have a negative sign and decline in absolute value for each of the three years following program approval, although only the result for the first year is statistically significant.12 A similar pattern is observed when the lack of program interruptions is considered as the measure of program implementation. The absence of program interruptions is correlated with greater price stability in the year following program approval, followed by slightly higher inflation rates over longer horizons. However, these results are only marginally significant.

Better institutions also lead to lower inflation in countries implementing IMF-supported programs.13 Inflation is lower, the better is the government bureaucracy at the start of the program and the more it improves subsequently. The importance of the quality of the bureaucracy index is highest in the first year of the program and declines afterward. Inflation is also lower, the more the legal system improves and the more the public observes the law. Interestingly, a higher degree of law and order before the start of the program and improvements in this regard during the program are associated with slightly higher inflation at horizon T+2.

The role of political factors in inflation performance in countries under IMF-supported programs is more difficult to interpret. Lower inflation is associated with increased political violence in the first two years of the program. Tight demand-side policies that succeed in reducing inflation could also trigger public protests against austerity, as has occasionally been the case in countries implementing IMF-supported programs.14

Recognizing that cross-country inflation regressions are dominated by outliers, we also examine whether ethnic tensions and internal conflicts are still the primary determinants of inflation when such observations are excluded from the sample. We reestimate the model on a sample that excludes all observations with annual change in inflation greater than 50 percent, which cuts the sample size by approximately 30 percent. The results are somewhat reassuring. In the inflation equation, ethnic tensions still play an important role in determining the evolution of inflation. Internal conflicts become in significant but, by contrast, government stability turns out to be significant. This is not very surprising since the two indices are highly correlated in our sample.

Growth

Economic growth is highly serially correlated and mean-reverting during the course of IMF-supported programs (Tables 6 and 7). As in the case of inflation, deviations of the growth rate from long-run equilibrium are very short lived. Approximately 90 percent of any deviation in growth rates from the country’s “normal” growth pattern is made up within three years. The largest adjustment, 83–84 percent, occurs within one year after the realization of the shock.

At first glance, better program implementation appears to be associated with more rapid economic growth, as suggested by positive and significant estimated coefficients in OLS regressions of the disbursement share. Unfortunately, this result is not robust—it appears to be driven by the endogeneity of program implementation. The corresponding 2SLS and 3SLS estimates are positive at all horizons, but the parameters are not significantly different from zero. In addition, the impact of program performance on economic growth is fragile to the choice of implementation measure. Although fewer program interruptions appear to be associated with higher growth rates, the OLS results are not significant, and the coefficients turn negative when IV techniques are used.

These mixed findings are consistent with those of the existing literature. Recovery of growth rates from the initial drop (a V-shaped response of output) was reported by Conway (1994). Khan (1990) and Przeworski and Vreeland (2000) find significantly negative effects of IMF program participation on economic growth. At the same time, Killick (1995), Bagci and Perraudin (1997), and Dicks-Mireaux, Mecagni, and Schadler (2000) report positive and significant effects. One possibility is that the extent of program implementation does matter for economic growth, but that the leads are greater than three years and therefore we have been unable to capture them. Certainly the structural reforms of many programs in the 1990s took a long time to come to fruition. Many countries—including transition economies—began to experience faster growth only in the late 1990s; such a delayed response would not be captured in our methodology.

Not surprisingly, improvements in institutions during the course of program implementation are associated with better growth performance. This is most evident in the case of the investment profile, which measures the risk to foreign business operations in the country, including risk of repatriation of profits. A onecroughly a 1 percent increase in the growth rate, and this result is robust to the length of the horizon and the choice of estimation technique. Improvements in the ability of the government to stay in office, which are influenced by the cohesion of the government and by the extent of the public’s approval of its policies, appear to have a significant positive impact on growth, at least in the first year of a program. These findings are robust to the choice of implementation measure and to omitting outliers.15

Public Finances

The fiscal balance (in relation to GDP) is persistent and mean-reverting, but less so than inflation and growth. Improvements in the fiscal balance persist for two years but are then reversed (Tables 8 and 9). This pattern could be consistent with governments implementing IMF-supported reforms aiming to balance their budgets over a four-year horizon. The mean-reversion term is highly significant. Approximately 45 percent of any deviation of the fiscal balance from its long-run average is offset within a year. The speed of adjustment is much slower than for inflation or growth.

As in the regressions explaining growth, program implementation appears to be associated with improvements in the public finances when simultaneity bias issues are ignored, but these results are reversed in the regressions using IV approaches. Regardless of the choice of implementation measure, the OLS estimates are positive and significant for the first two years, whereas the 2SLS and 3SLS estimates are negative. If anything, better program implementation seems to be associated with larger fiscal deficits: IV estimates of the coefficient on the disbursement share two years after program approval are significant.16 The results are similar when the lack of program interruptions measure is considered. They suggest that fiscal deficits in countries with completed programs are about 3 percent larger than in countries whose programs were interrupted.

This finding and our similar finding for the current account balance (see the subsequent discussion) likely reflect the impact of additional financial resources flowing into countries that are successful in implementing IMF-supported reforms. Better program implementation makes more financing available to countries participating in IMF-supported programs, which allows more gradual adjustment and larger fiscal and external deficits.

The most important institutional factor influencing fiscal outcomes is the initial level of corruption, but its effect is anomalous. Lower corruption is associated with weaker fiscal outcomes over time. We do not have a good explanation for this result.

Several aspects of the political environment play an important role in determining fiscal outcomes in countries with IMF-supported programs. First, improvements in the government’s responsiveness to its people are associated with larger deficits. This could be evidence that democratic incumbents tend to postpone fiscal consolidation. Second, declines in ethnic tension are contemporaneously correlated with improved fiscal balances. This could reflect a country’s return to normalcy, which is associated with improved revenue collection and lower military spending. Third, less military involvement in politics in the preprogram year, as well as declines in the risk of external conflict (ranging from trade restrictions to full-scale warfare) are positively and significantly associated with lower fiscal deficits.

Current Account

Shocks to the current account are longer lived and have larger permanent components than other macroeconomic outcomes (Tables 10 and 11). Only about 10 percent of any deviation from a country’s “normal” ratio of the current account to GDP is made up for in one year.

Most studies find that participation in IMF-supported programs helps improve the current account. Our results on the impact of program implementation on the current account are more nuanced. Countries that do a better job at implementing programs experience a deterioration of the current account for about two years, but this is followed by a sharp improvement in the trade balance for the third year. Disbursement of 100 percent of committed funds is accompanied by an 8 percent deterioration of the current account in the first year (relative to the preprogram year), followed by a numerically noticeable but statistically insignificant 2 percent improvement in the third year. Our mixed results are similar to Barro and Lee’s (2002). By contrast, Conway (1994) finds evidence of improvement in the current account in countries participating in IMF-supported programs, but it does not correct for the extent of program implementation.

The only institutional variables that matter for the current account are the initial investment profile and its change during the program period. Both are highly significant and enter the regressions with negative signs. Not surprisingly, the better a government’s attitude toward inward investment, the larger the current account deterioration during the period considered.

Of the political variables, the ones relevant for the evolution of the external current account are external conflict, government stability, and military involvement in politics. The coefficient on the change in the external conflict index is negative and highly significant for the T+2 horizon. Improvements in the index are associated with the elimination of embargoes and of trade restrictions and are correlated with a worsening of the current account. A one-point increase in this rating is correlated with a 1.2 percent deterioration of the current account over three years. Governments that are more stable in the preprogram year tend to have better current account performance. Similar positive effects on the current account appear to result from less military involvement in politics before the program initiation. These results are robust to the choice of program implementation measure.

Conclusion

This paper has examined the nexus among institutions, policy implementation, and economic performance in countries undertaking IMF-supported reforms. We employed a short-run statistical model that treats institutions and politics as exogenous and mean-reverting, that takes into account the autoregressive and mean-reverting nature of macroeconomic outcomes, and that corrects for the endogeneity of program implementation with respect to macroeconomic performance.

Our main findings are fourfold. First, the quality of institutions and the domestic political environment matter for macroeconomic outcomes in countries implementing IMF-supported programs, especially at longer horizons of up to three years. This direct beneficial impact of institutions on the macroeconomic variables is in addition to their indirect impact through better program implementation. As expected, improvements in the government bureaucracy and better enforcement of law and order are associated with lower inflation. However, declines in internal conflict are associated with higher inflation. Improvements in a program country’s investment profile and in government stability lead to faster economic growth. Easing of external conflict and lower military involvement in politics before program approval are associated with stronger fiscal outcomes as military expenditure declines. On the other hand, reductions in ethnic tension and improvements in government accountability are associated with weaker fiscal outcomes, perhaps because programs may provide for higher targeted expenditure. Greater government stability and reductions in the military’s involvement in politics before the program starts are associated with a strengthening of the external current account. However, lower ethnic tension and improvements in a program country’s investment profile lead to a deterioration of the current account.

Second, the institutional and political environment is quantitatively important for the implementation of IMF-supported programs. Rates of disbursement of IMF loans are higher and program interruptions less frequent in countries where ethnic tensions are low, where governments are stable and less corrupt, and where the military is less involved in politics. In addition, more IMF loans are disbursed and fewer interruptions are experienced in countries in which internal conflict was intense and law enforcement weak before program approval. Arguably, this reflects the IMF’s role, as lender and policy adviser, in facilitating the return to normalcy of countries experiencing natural or political shocks.

Third, program implementation varies systematically with the duration of a country’s financial engagement with the IMF and the size of its quota. More funds are disbursed and fewer program interruptions are experienced in countries that have spent more time in previous IMF-supported programs. Implementation is also better for larger programs (as measured by the amount of program financing approved in relation to the country’s IMF quota).

And fourth, after the impact of institutions on the macroeconomic situation is taken into account, the extent of program implementation exerts an independent influence on macroeconomic outcomes, especially over shorter horizons of up to two years. Better-implemented programs are associated with lower inflation, with initially weaker but ultimately stronger external and fiscal outcomes, and with a statistically insignificant impact on economic growth. These results are to be contrasted with those of studies that do not correct for program implementation; these studies conclude that program participation has ambiguous effects on inflation. Correcting for differences in implementation thus provides some evidence linking successful implementation of IMF-supported reforms to more progress in achieving price stability.

What, then, are the policy implications of this analysis for the IMF? The first issue is the lack of clear-cut results linking program implementation to the resumption of economic growth in countries implementing IMF-supported reforms. One possibility is that successful program implementation has favorable impacts on growth that are only felt beyond the three-year horizon captured in our model. The length of lags in the operation of IMF-supported structural reforms should be a topic of future research. Further, the lack of conclusive links between program implementation and growth suggests that it might be useful for the IMF to seek to identify structural reforms that could pay off quickly in terms of economic growth, both at the program design stage and at the implementation stage. At the program design stage, the IMF could monitor regularly published institutional and political indicators relevant to economic growth—such as the ICRG ratings of the level of ethnic tension, government accountability, and the investment climate. These indicators would also need to be carefully monitored during program implementation to ascertain whether IMF-supported reforms are on track toward meeting their growth objectives. Information on the determinants of the investment profile—viability of contracts, threat of expropriation, ease of profit repatriation, and payment delays—could provide high-frequency feedback concerning the extent to which programs are on track in implementing investment-friendly reforms.

Second, paying due attention to relevant political and institutional developments is critical to the successful design and implementation of IMF-supported programs. Quantitative information and analysis could be a useful complement to information from IMF missions and resident representatives in assessing rapidly changing political environments, indicating the potential for successful program implementation. A decline in political indicators below thresholds historically associated with inadequate program implementation could give the IMF an early warning signal, much as financial vulnerability indicators provide useful signals of impending financial crisis. The IMF has on occasion responded to heightened political uncertainty by requiring the major political forces in a country—the government and the main opposition parties in parliament—to endorse a program at an early stage. Systematizing these efforts, as the IMF has been doing by increasing the emphasis on ownership, could yield dividends in terms of improved program design, implementation, and macroeconomic performance. It would enable the IMF to avoid situations in which, having designed and implemented first-best programs that failed to fully take into account relevant political and institutional factors, one ends up in a third-best world when these “ideal” programs are not properly executed. In econometric terms, one would ideally want initial (T–1) institutional and political variables to enter implementation regressions (such as those in Table 3) with in significant coefficients. That would provide evidence that IMF-supported programs are well tailored to the specifics of each country’s politico-institutional climate and that the success or failure of a program is independent of initial political conditions. Unfortunately, we have such neutrality only for the initial economic conditions. More generally, it might be useful to consider incorporating quantitative political and institutional indicators and analysis in IMF surveillance work.

Third, we have treated institutions as exogenous and mean-reverting processes, yet institutional development is an important objective of IMF-supported programs with a structural orientation. It would be useful to assess systematically the impact of better implementation of IMF-supported programs on the dynamics of institutional and political factors. In such a model, the evolution of formal and informal institutions would be endogenous to the politico-economic process, including participation in IMF-supported programs. To the extent that IMF-supported programs promote welfare-improving institutional change, their beneficial effects are going to be larger than suggested by models, such as ours, that treat institutions as exogenous.

References

    Atoyan, Ruben, PatrickConway, MarceloSelowsky, and TsidiTsikata,2004,“Macroeconomic Adjustment in IMF-Supported Programs: Projections and Reality,”IMF Independent Evaluation Office BP/04/2(Washington:International Monetary Fund). This paper also appears, in somewhat different form, as Chapter 5 in this volume.

    • Search Google Scholar
    • Export Citation

    Bagci, Pinar, and WilliamPerraudin,1997,“The Impact of IMF Programmes,” Working Paper No. 24(London:Global Economic Institutions Research Program, Center for Economic Policy Research).

    • Search Google Scholar
    • Export Citation

    Barro, Robert, and J.Lee,2002,“IMF Programs: Who Is Chosen and What Are the Effects?” NBER Working Paper No. 8951(Cambridge, Massachusetts:National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation

    Conway, Patrick,1994,“IMF Lending Programs: Participation and Impact,”Journal of Development Economics,Vol. 45(December),pp. 365–91.

    • Search Google Scholar
    • Export Citation

    Dicks-Mireaux, Louis, MauroMecagni, and SusanSchadler,2000,“Evaluating the Effect of IMF Lending to Low-Income Countries,”Journal of Development Economics,Vol. 61,pp. 495–526.

    • Search Google Scholar
    • Export Citation

    Dollar, David, and JakobSvensson,2000,“What Explains the Success or Failure of Bank-Supported Adjustment Programs?”Economic Journal,Vol. 110(October),pp. 894–917.

    • Search Google Scholar
    • Export Citation

    Dreher, Axel,2004,“IMF and Economic Growth: The Effects of Programs, Loans, and Compliance with Conditionality,”Working Paper, University of Konstanz(July).Available on the Web at http://econwpa.wustl.edu:8089/eps/if/papers/0404/0404004.pdf.

    • Search Google Scholar
    • Export Citation

    Haque, Nadeem Ul, and MohsinKhan,1998,“Do IMF-Supported Programs Work? A Survey of the Cross-Country Empirical Evidence,” IMF Working Paper 98/169(Washington:International Monetary Fund).

    • Search Google Scholar
    • Export Citation

    International Country Risk Guide (ICRG),2003,Brief Guide to the Ratings System(East Syracuse, New York:Political Risk Services Group, Inc.).

    • Search Google Scholar
    • Export Citation

    International Monetary Fund (IMF),2002,Evaluation of the Prolonged Use of IMF Resources (2 vols.).(Washington:Independent Evaluation Office, International Monetary Fund).

    • Search Google Scholar
    • Export Citation

    International Monetary Fund (IMF),2003,“Growth and Institutions,”Chapter III of World Economic Outlook(April)(Washington:International Monetary Fund).

    • Search Google Scholar
    • Export Citation

    Ivanova, Anna, WolfgangMayer, AlexMourmouras, and GeorgeAnayiotos,2003,“What Determines the Implementation of IMF-Supported Programs?” IMF Working Paper 03/08(Washington:International Monetary Fund). This paper also appears, in somewhat different form, as Chapter 10 in this volume.

    • Search Google Scholar
    • Export Citation

    Joyce, Joseph,2003,“Promises Made, Promises Broken: A Model of IMF Program Implementation,” Working Paper No. 2003–03(Wellesley, Massachusetts:Department of Economics, Wellesley College).Available on the Web at http://www.wellesley.edu/Economics/joyce/PROM4.pdf.

    • Search Google Scholar
    • Export Citation

    Khan, Mohsin,1990,“The Macroeconomic Effects of Fund-Supported Adjustment Programs,”Staff Papers, International Monetary Fund,Vol. 37(June),pp. 195–231.

    • Search Google Scholar
    • Export Citation

    Killick, Tony,1995,IMF Programmes in Developing Countries: Design and Impact(London:Routledge).

    Mecagni, Mauro,1999,“The Causes of Program Interruptions,” inEconomic Adjustment and Reform in Low-Income Countries, ed. byHughBredenkamp and SusanSchadler(Washington:International Monetary Fund).

    • Search Google Scholar
    • Export Citation

    North, Douglass,1997,“Introduction,” inThe Frontiers of the New Institutional Economics, ed. byJohnDrobak and JohnNye(New York:Academic Press),pp. 3–13.

    • Search Google Scholar
    • Export Citation

    Przeworski, Adam, and James R.Vreeland,2000,“The Effect of IMF Programs on Economic Growth,”Journal of Development Economics,Vol. 62,pp. 385–421.

    • Search Google Scholar
    • Export Citation

    Rodrik, Dani, ArvindSubramanian, and FrancescoTrebbi,2002,“Institutions Rule: The Primacy of Institutions over Geography and Integration in Economic Development,” NBER Working Paper No. 9305(Cambridge, Massachusetts:National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation

    Schadler, Susan, FranekRozwadowski, SiddharthTiwari, and DavidRobinson,1993,Economic Adjustment in Low-Income Countries—Experience Under the Enhanced Structural Adjustment Facility, IMF Occasional Paper No. 106(Washington:International Monetary Fund).

    • Search Google Scholar
    • Export Citation
1

We thank our colleagues at the IMF and the University of North Carolina at Chapel Hill for useful input when the project was under way. We are especially indebted to Alex Dreher, Shigeru Iwata, Timothy Lane, and Gene Leon for many helpful suggestions on an earlier draft. We alone are responsible for any remaining errors.

2

This is understandable, given the econometric difficulties of constructing counterfactuals and the long and variable lags between progress in microeconomic and structural policies and improvements in economic performance. Haque and Khan (1998) discuss the problem of the counterfactual and the stylized facts of the macroeconomic impacts of IMF-supported programs.

3

See the ICRG Guide to the Rating System for details (http://www.prsonline.com).

4

Note that excluding contemporaneous changes in institutions from the model addresses a potential endogeneity problem that is present if the error term affecting macroeconomic variables also affects institutional developments during the program period.

5

An alternative approach would be to run regressions with only first differences of institutional variables. The qualitative results remained unchanged when we reestimated our regressions in this manner. This makes us confident that our results are robust.

6

The coefficient on the lagged fiscal balance term is significant in the inflation equation. The lagged first difference and the lagged level of inflation are significant in the growth equation. The lagged first difference and the lagged level of the fiscal balance significantly influence the evolution of the current account.

7

We formally test a set of restrictions that turns the VECM into an autoregressive model. With an exception of the growth equation, we cannot reject the null hypothesis that the data-generating process was indeed just an autoregression (the p-values are 0.15, 0.02, 0.15, and 0.40 for inflation, growth, the fiscal balance, and the current account balance, respectively).

8

The OLS estimator is still useful in model selection because it is less sensitive than the alternatives to the presence of multicollinearity, errors in variables, or misspecification, particularly in small samples. After relying on OLS to choose an appropriate model, we compare its predictions with those from the same model estimated by alternative means.

9

An overidentifying restrictions test could not reject the null hypothesis of overidentified restrictions for either implementation measure.

10

A shock that affects economic growth has informational content for inflation, the fiscal deficit, and the external current account.

11

Contemporaneous changes in socioeconomic conditions are excluded from the analysis. This avoids the problems associated with dependent variables appearing on the right-hand side of the equation.

12

The inflation dynamics reported here are similar to those in Conway (1994). Killick (1995) finds reduction in inflation to be significant. Barro and Lee (2002) reports coefficients on contem poraneous and lagged IMF loans that are similar in sign but in significant.

13

Our findings on the impact of institutions on macroeconomic performance in program countries are robust to the choice of implementation measure.

14

This cannot be formally tested in our model, because we treat political variables as exogenous.

15

We define outliers as countries growing or shrinking by more than 10 percent a year.

16

Schadler and others (1993) also find some evidence of negative effects of IMF lending on the fiscal balance. By contrast, Conway (1994) finds significant fiscal deficit reduction.

    Other Resources Citing This Publication