Ex post Evaluation of Macro Financial Assistance Operations in Georgia
Description:
The main objective of this ex-post evaluation is to evaluate the EC's macro-financial assistance (MFA) operations in Georgia in order to learn key lessons that can be applied to future interventions and/or to determine whether a reorientation of the present approach is needed. The evaluation questions broadly focus on three core areas of effects: macroeconomic stabilization, sustainability of the external financial situation, and structural reforms.
For the attribution of effects to the MFA operation, we follow a five step approach. The first step is to describe the MFA operation in terms of its design, objectives and its relevance. This step has for a large part been concluded and results are incorporated in this inception report.
The second step is to identify the gross macroeconomic and structural effects. These gross effects are equal to the observed developments after the disbursement of the grant. To a large extent, this second step is reflected in this inception report. The third step is to establish a counterfactual situation. This refers to the most likely scenario in the domain of macroeconomic stabilisation and structural reform when no MFA would have been granted to Georgia.
In determining the counterfactual situation, we will also include the analysis of the unexpected and indirect effects of the MFA operation. This third step will be based on interviews with key stakeholders of the IFI's and the Georgian government. This will be part of the next phase of this evaluation.
The fourth step involves the determination of the net effects of the operation. The net effects follow out of the comparison between observed developments and the counterfactual situation.
In the final step, conclusions are drawn on the design of the MFA approach
In determining the counterfactual situation, we will also include the analysis of the unexpected and indirect effects of the MFA operation. This third step will be based on interviews with key stakeholders of the IFI's and the Georgian government. This will be part of the next phase of this evaluation (see chapter 9 for the planning of this next phase).
To conduct the five steps of the evaluation approach, we will collect data by means of a variety of evaluation methods.
Document analysis
Analysis of the documents available in the files of the European Commission has been concluded. Also, the review of documents from the main IFI's (World Bank and IMF) has been done and results are included in this Inception Report. Further analysis and search for additional documents will continue in the next phase.Interviews with key officials
In addition, the first round of interviews with EC officials in Brussels has been conducted (see Annex I). The findings are incorporated in this report. In the next phase, we will interview stakeholders of the Georgian authorities and IFI's.
Macroeconomic modelling
A quantitative modelling approach will be used to estimate the macroeconomic effects in the counterfactual situation. This model will be similar to those employed in previous evaluations. We note that the MFA operation in Georgia has some similarity with the EFA operation in Tajikistan which also focussed on reducing debt obligations to the EC. As such we will take into consideration the modelling approach used in the Tajikistan evaluation of EFA and adapt it to the Georgian situation.
Case studies
To explore cause-and-effect relations between the MFA and actual reforms in Georgia, we will conduct case studies of the structural reform.
Delphi Questionnaire
To collect and report the perceptions of the key stakeholders on the counterfactual in a structured and transparent manner, we will conduct a two-round Delphi questionnaire.
Stakeholder workshop
At the end of the evaluation, we will discuss our findings and preliminary conclusions and recommendations in a workshop with the main stakeholders in Tbilisi.
Sponsor:
European Commission, Directorate for Economic and Financial Affairs
Partner:
ECORYS Netherlands BV
