Causal Inference Methods for Policy Evaluation

7-Synthetic Control Methods

Jacopo Mazza

Utrecht School of Economics

2026

The Policy: The Costs and Benefits of Migration

Migration is Increasing

Migration Slows Down Aging

Migration and the Economy

  • Benefits:

    • Economic growth,
    • Innovation,
    • Cultural diversity

  • Costs:

    • Pressure on public services,
    • social cohesion,
    • wages

How would you assess the effect of immigration on local wages, with data?

The Mariel Boatlift

A Natural Experiment

  • In 1980 Fidel Castro announced that any Cuban who wanted to leave the country could do so.

  • Cubans in the US helped to organize the Mariel Boatlift, which brought 125,000 Cubans to Miami in a few months.

  • Miami’s labor force increased by 7% in a few months.

  • Card (1990) used this event to study the effect of an inflow of low-skilled workers on local wages.

Miami: the Treatment Group

The Control Group

Card’s Analysis

  • The analysis compares the evolution of wages in Miami with the evolution of wages in other cities.

  • Data from the Current Population Survey, on wages and employment before and after the Mariel Boatlift in Miami and in a set of control cities.

The Difference-in-Differences Results

  • Card finds no evidence of a negative effect of the Mariel Boatlift on wages in Miami.

  • Neither for white nor for African-American workers.

The Choice of Control Cities

Card (1990, p. 249)

Does this method remind you of something we have seen before?

→ Difference-in-Differences! DiD also compares a treated unit to a control group before and after an intervention. SCM generalises this: rather than choosing the control group ad hoc, it constructs one as an optimally weighted average of donor units.

The Synthetic Control Method

The Idea

  • Card’s choice of control cities was arbitrary.

  • The synthetic control method removes this arbitrariness.

  • The method constructs a synthetic control group for the treated unit that is a weighted average of the donor pool.

  • The weights are chosen to minimize the difference between the treated unit and the synthetic control group in the pre-treatment period.

  • Synthetic control methods are usually used for a single treated unit (a single school, firm, country, etc.).

The Objective

  • Our goal: reproduce the counterfactual of a treated unit by finding the combination of untreated units that best resembles the treated unit before the intervention in terms of the values of \(k\) relevant covariates (predictors of the outcome of interest)

  • Method selects weighted average of all potential comparison units that best resembles the characteristics of the treated unit(s) - called the synthetic control

Synthetic control method: advantages

  • Convex hull means synth is a weighted average of units which means the counterfactual is a collection of comparison units that on average track the treatment group over time.

  • Constraints on the model use non-negative weights which does not allow for extrapolation

  • Makes explicit the contribution of each comparison unit to the counterfactual

  • Formalizing the way comparison units are chosen has direct implications for inference

Synthetic control method: disadvantages

  1. Subjective researcher bias kicked down to the model selection stage

  2. Significant diversity at the moment as to how to principally select models - from machine learning to modifications - as well as estimation and software

  3. Part of the purpose of this procedure is to reduce subjective researcher bias

  4. Ferman, Pinto and Possebom (2020) suggest specific specifications and report all of them

Formalization

Suppose that we observe \(J + 1\) units in periods \(1, 2, . . . , T\)

  • Unit “one” is exposed to the intervention of interest (that is, “treated”) during periods \(T_0 + 1, . . . , T\)

  • The remaining \(J\) are an untreated reservoir of potential controls (a “donor pool”)

  • Key assumptions: no anticipation of the treatment before \(T_0\); no spillovers from the treated unit to the donor pool (SUTVA)

Optimal Weights

  • The synthetic control is a weighted average of the \(J\) potential controls

  • Let \(W = (w_2, \ldots, w_{J+1})′\) with \(w_j \geq 0\) for \(j = 2, \ldots, J + 1\) and \(w_2 + \cdots + w_{J+1} = 1\). Each value of \(W\) represents a potential synthetic control

  • Let \(X_1\) be a \((k \times 1)\) vector of pre-intervention characteristics for the treated unit. Similarly, let \(X_0\) be a \((k \times J)\) matrix which contains the same variables for the unaffected units.

  • Step 1: For a given predictor-weight matrix \(V\) (diagonal, \(k \times k\)), \(W^*(V) = (w^*_2, \ldots, w^*_{J+1})′\) minimises \(\|X_1 − X_0 W\|_V\), subject to the weight constraints

  • Step 2: \(V\) is then chosen to minimise the pre-treatment mean squared prediction error (MSPE): \[ \sum_{t=1}^{T_0} \bigg (Y_{1t} - \sum_{j=2}^{J+1}w_j^*(V)\,Y_{jt}\bigg )^2 \]

Estimating the \(V\) matrix

Choice of \(v_1, . . . , v_k\) can be based on:

  • Assess the predictive power of the covariates using regression
  • Subjectively assess the predictive power of each of the covariates, or calibration inspecting how different values for \(v_1, . . . , v_k\) affect the discrepancies between the treated unit and the synthetic control
  • Minimize mean square prediction error (MSPE) for the pre-treatment period (default).

The Synthetic Control Method in Practice

California’s Proposition 99

  • In 1988, California first passed comprehensive tobacco control legislation:
    • increased cigarette tax by 25 cents/pack
    • earmarked tax revenues to health and anti-smoking budgets
    • funded anti-smoking media campaigns
    • spurred clean-air ordinances throughout the state
    • produced more than $100 million per year in anti-tobacco projects
  • Other states that subsequently passed control programs are excluded from donor pool of controls (AK, AZ, FL, HI, MA, MD, MI, NJ, OR, WA, DC)

The California Study

Abadie, Diamond, and Hainmueller (2010) use the synthetic control method to estimate the effect of Proposition 99 on per capita cigarette consumption in California.

Cigarette Consumption in CA and the Synthetic CA

  • ADH select 38 states as potential donors
  • They select optimal weights to minimize the difference between California and the synthetic California in the pre-treatment period
  • Pre-treatment California and synthetic California are very similar
  • Post-treatment California and synthetic California diverge

Synthetic California

The Predictors

Variables Real California Synthetic Calif. Avg. of 38 Control States
Ln(GDP per capita) 10.08 9.86 9.86
Percent aged 15–24 17.40 17.40 17.29
Retail price 89.42 89.41 87.27
Beer consumption per capita 24.28 24.20 23.75
Cigarette sales per capita 1988 90.10 91.62 114.20
Cigarette sales per capita 1980 120.20 120.43 136.58
Cigarette sales per capita 1975 127.10 126.99 132.81

All variables except lagged cigarette sales are averaged for the 1980–1988 period. Beer consumption is averaged 1984–1988.

CA and Synthetic CA Gap in Cigarette Sales

Inference in Synthetic Control Methods

The Problem

  • We only observe 2 points per year: Hard to determine whether the observed difference between the treated unit and the synthetic control is due to the treatment or to chance

  • ADH (2010) use randomization inference to test the null hypothesis that the treatment had no effect on the outcome

Randomization inference

To assess significance, we calculate exact p-values under Fisher’s sharp null using a test statistic equal to after to before ratio of RMSPE

  • Exact p-value method
    • Iteratively apply the synthetic method to each country/state in the donor pool and obtain a distribution of placebo effects
    • Compare the gap (RMSPE) for California to the distribution of the placebo gaps. For example the post-Prop. 99 RMSPE is: \[RMSPE = \bigg (\dfrac{1}{T-T_0} \sum_{t=T_0+1}^T \bigg (Y_{1t} - \sum_{j=2}^{J+1} w_j^* Y_{jt} \bigg )^2 \bigg )^{\tfrac{1}{2}}\] and the exact p-value is the treatment unit rank divided by \(J+1\) (the total number of units, including the treated unit)

Is California Treatment Effect Extreme?

Placebo distribution using all units as donor pool

Pre-Proposition 99 RMSPE \(\geq2\) times Pre-Prop 99 RMSPE for CA
Figure 1

Histogram of post/pre RMSPE of all units.

If we were to assign the intervention at random, the probability of obtaining a post/pre-Proposition 99 RMSPE ratio as extreme as the one observed in California is 0.026.

The Synthetic Marielitos

Remember the Mariel Boatlift? Peri and Yasenov (2019) use the synthetic control method to estimate the effect of the Mariel Boatlift on the Miami labor market.

Findings: no evidence of a negative effect of the Mariel Boatlift on wages in Miami.

Preconditions and Data Requirements of Synthetic Control Methods

Preconditions

SCM is a powerful tool, but it is not a panacea. It requires:

  1. Big and non-volatile effects;
  2. Availability of a good comparison group;
  3. No anticipation of the treatment;
  4. No spillover effects (SUTVA);
  5. Convex hull of the treated unit in the donor pool;

Data Requirements

  1. Aggregate data on predictors and outcomes;
  2. Sufficient pre-intervention data;
  3. Sufficient post-intervention data.

Key Takeaways

Key Takeaways

  • Motivation: When a single unit is treated (a state, country, firm), DiD with an ad hoc control group is arbitrary. SCM constructs the counterfactual optimally.

  • The method: A synthetic control is a weighted average of donor units, with weights chosen to minimise the pre-treatment gap in outcomes and predictors.

  • Two-step estimation: Unit weights \(W^*\) and predictor weights \(V\) are jointly optimised — \(V\) governs how predictors are compared, \(W^*\) governs how donors are blended.

  • Inference: Standard asymptotics are unavailable (\(J\) small, \(T\) moderate). Fisher permutation/randomisation inference — applying SCM to each donor as a placebo — yields an exact \(p\)-value.

  • Validity: Requires no anticipation, SUTVA, and that the treated unit lies in the convex hull of the donor pool (no extrapolation).

  • Limitations: Researcher degrees of freedom at the model-selection stage; works best with many pre-treatment periods and a clearly defined single treated unit.