pretest

Thresholded DID Diagnostics for Conditional Extrapolation

Overview

pretest asks whether the observed pre-treatment path is mild enough to carry delta_bar into conditional extrapolation in common-treatment-time block-adoption designs. One run reports pre-treatment severity S_pre, a pass indicator, delta_bar, and, only when pretest_pass = 1, pass-case interval endpoints read as centered on delta_bar and informative about the average ATT only under the conditional extrapolation assumption.

The shipped Prop99 example shows the decision margin. Lowering the threshold from 5 to 1 leaves delta_bar = -14.4450 and S_pre = 2.1959 unchanged but moves pretest_pass from 1 to 0. The threshold changes what the user may claim before it changes the DID summary.

In practice, the command separates three choices that applied DID work often blurs: threshold choice, sample choice, and violation-summary choice.

Read a run in four steps: severity first, gate second, delta_bar third, and pass-case interval endpoints last. When those endpoints appear, read them as centered on delta_bar. Treat them as informative about the average ATT only under the conditional extrapolation assumption and the asymptotic conditions in the method paper.

Users choose the threshold under the conditional extrapolation assumption:

Assumption 3 (Conditional Extrapolation): If S _pre ≤ M, then S _post ≤ S _pre.

A typical run gives users:

The pre-treatment severity S_pre
A pass indicator for the chosen threshold
The DID summary delta_bar
Pass-case interval endpoints for that conditional reading rule

First Run

pretest cigsale, treatment(treated) time(year) treat_time(1989) threshold(5)

Use overall only when the cumulative-violation path is the empirical question.

Requirements

Before using this package, ensure your data meets the following requirements:

Requirement	Description
Minimum 3 time periods	T _pre ≥ 2. At least two pre-treatment periods are required because iterative violations ν̂_t are only defined for t ≥ 2.
Block adoption design	All treated units must receive treatment at the same timet ₀. Staggered adoption designs are not supported.
Binary treatment	Treatment indicator must be coded as 0 (control) or 1 (treated).
Complete time-group cells	Each time period must contain observations in both treatment and control groups.

Data Completeness

When some time periods lack observations for either group, the covariance matrix cannot be computed. In such cases:

e(phi) = . (missing, indicating data issue)
e(data_valid) = 0
e(S_pre), e(f_alpha), e(ci_lower), e(ci_upper) will be missing (.)

Common causes:

Missing values in treatment or outcome variables creating empty cells
Survey data with irregular interview schedules
Sample restrictions that eliminate entire time periods for one group

Solution: Ensure at least one observation per time-treatment cell, or restrict analysis to time periods with complete coverage.

Two-Period Designs

This command cannot be used for canonical 2×2 DID designs with only two time periods.

Installation

From GitHub

net install pretest, from("https://raw.githubusercontent.com/gorgeousfish/pretest/main") replace

From SSC (Coming Soon)

This package will be available on the SSC archive in the near future.

Quick Start

* Load the example data (prefer a local shipped or net-get copy)
capture noisily findfile prop99_smoking.dta
if _rc == 0 {
    use "`r(fn)'", clear
}
else {
    webuse set "https://raw.githubusercontent.com/gorgeousfish/pretest/main/"
    webuse prop99_smoking.dta, clear
}

* Set panel structure
xtset state year

* Run pre-test with threshold M = 5
pretest cigsale, treatment(treated) time(year) treat_time(1989) threshold(5)

* Use overall only when the cumulative-violation path is the question
pretest cigsale, treatment(treated) time(year) treat_time(1989) threshold(5) overall

Warning for clustered runs: Before clustered runs, subset the analysis sample explicitly and rerun pretest on that fixed sample; inline [if] [in] restrictions are not a supported substitute for this workflow.

net get pretest downloads loose working-directory copies of prop99_smoking.dta and example_prop99.do. A clean net install pretest alone does not place discoverable example assets on the installed adopath, so the GitHub webuse path is a fallback rather than the canonical local route.

Syntax

pretest depvar , treatment(varname) time(varname) threshold(#) [options]

Required

Option	Description
`treatment(varname)`	Binary treatment indicator (0/1)
`time(varname)`	Time variable
`threshold(#)`	Acceptable violation thresholdM > 0

Optional

Option	Default	Description
`treat_time(#)`	auto	Common treatment-onset timet ₀; auto-detected only from a common 0→1 switch path
`p(#)`	2	Severity normp ≥ 1
`alpha(#)`	`1-c(level)/100`	Nominal significance level for interval construction
`level(#)`	`c(level)`	Nominal confidence level (%) for interval construction
`cluster(varname)`	—	Cluster variable for robust SE after explicit sample subsetting and missing-row cleanup
`overall`	off	Use cumulative-violation path
`nograph`	off	Suppress event study graph
`simulate(#)`	5000	Monte Carlo simulations
`seed(#)`	12345	Random seed
`diagnose`	off	Display detailed diagnostic information

Graph Customization

Option	Description
`ci_opt_pass(string)`	Override pre-treatment CIs and pass-case post-treatment CIs
`ci_opt_fail(string)`	Override CI style in non-pass cases
`line_opt_m(string)`	Override threshold M line style
`marker_opt_pre(string)`	Override pre-treatment marker style
`marker_opt_post(string)`	Override post-treatment marker style
`scheme()`, `title()`, etc.	Any standard Stata twoway_options

Note: Element-specific options replace the default styling for the layers they control. In particular, ci_opt_pass() controls the pre-treatment CI layer and, when the run passes, the post-treatment pass-case endpoints.

If alpha() and level() are both omitted, pretest inherits Stata's current confidence level. If both are supplied, alpha() takes precedence.

Key Formulas

Pre-test Indicator

At the interface level, use the gate:

φ = 𝟙{Ŝ _pre > M}

Here φ = 0 means the run passes the chosen threshold and φ = 1 means it fails that threshold. The main reporting flag is pretest_pass: it equals 1 when phi = 0 and 0 in all non-pass cases. When phi is missing because the data are invalid, pretest_pass still remains 0, so read 0 as non-pass and use phi together with data_valid to separate a threshold fail from an invalid-data run.

Average DID Estimate

Important: The δ̄̂ reported by this package is not the traditional ATT.

The DID estimand at time t is defined relative to the treatment time t ₀:

δ̂_t = (Ȳ _t,D=1 − Ȳ _t₀,D=1) − (Ȳ _t,D=0 − Ȳ _t₀,D=0)

where Ȳ _t,D=d denotes the sample mean of outcomes for group D = d at time t.

The average DID estimand across post-treatment periods is:

δ̄̂ = (1/T_post) × Σ_t=t₀^T δ̂_t

Key differences from traditional DID:

Aspect	Paper's δ̄̂	Traditional ATT
Reference point	Treatment timet ₀	Pre-treatment average
δ̂_t₀	Always 0 (by construction)	N/A
Interpretation	Incremental change fromt ₀	Total treatment effect

Example: If treatment effect is constant at 2.0 per period:

Traditional ATT ≈ 2.0 (total effect)
Paper's δ̄̂ ≈ 0 (no incremental change after t₀)

Why this definition? The paper's δ̄̂ is designed for the conditional extrapolation framework, where:

In iterative mode, the bias bound uses κ · Ŝ_pre; in overall mode, the command switches to the cumulative-violation object Ŝ^Δ_pre rather than simply reusing the iterative path with κ turned off
The summary tracks incremental change from treatment onset rather than the ATT level

For comparison with a conventional DID interval for delta_bar under exact parallel trends, use e(ci_conv_lower) and e(ci_conv_upper); for ATT-level estimation, use a standard DID estimator.

Pass-Case Conditional Interval (Theorem 2)

1. Iterative mode (Default):

I = δ̄̂ ± {κ · Ŝ _pre + f(α, Σ̂) / √n}

Bias bound includes the multiplier κ ≥ 1.

2. Overall mode:

I^Δ = δ̄̂ ± {Ŝ^Δ_pre + f^Δ(α, Σ̂^Δ) / √n}

Bias bound uses the cumulative-violation object Ŝ^Δ_pre, so overall mode is a different inferential path rather than iterative mode with κ switched off.

κ Constant (Iterative Mode Only)

κ = ((1/T_post) · Σ_t=1^T_post t^q)^1/q

where q is the Hölder conjugate of p. κ captures the worst-case accumulation of iterative violations over time.

For T _post > 1, κ > 1.
For p = 2 and large T _post, κ grows with √T _post.
Overall Mode: κ is not used (effectively κ = 1), so the bias bound and interval construction follow the cumulative-violation path instead.

Stored Results

Scalars

Result	Description
`e(S_pre)`	Estimated pre-treatment severity
`e(S_pre_se)`	Standard error of S_pre (Delta method)
`e(kappa)`	Bias-bound scalar κ (posted as 1 in overall mode)
`e(phi)`	Pre-test result (0 = pass, 1 = fail, . = data issue or invalid)
`e(data_valid)`	Data validity indicator
`e(pretest_pass)`	Pre-test pass indicator (1 = pass, 0 = non-pass; read with `e(phi)` and `e(data_valid)` when needed)
`e(delta_bar)`	Average DID change relative to `t0`
`e(se_delta_bar)`	Standard error of average DID estimate
`e(ci_lower)`	Pass-case interval lower endpoint; read as part of a `delta_bar`-centered interval when `pretest_pass = 1` and as informative about the average ATT only under the conditional extrapolation assumption
`e(ci_upper)`	Pass-case interval upper endpoint; read as part of a `delta_bar`-centered interval when `pretest_pass = 1` and as informative about the average ATT only under the conditional extrapolation assumption
`e(T)`	Total time periods
`e(t0)`	Treatment time on internal consecutive index
`e(t0_orig)`	Treatment time in original calendar units (e.g., the year supplied to `treat_time()`)
`e(T_pre)`	Pre-treatment periods
`e(T_post)`	Post-treatment periods
`e(N)`	Number of observations

Matrices

Result	Description
`e(nu)`	Pre-treatment violations (T _pre−1 × 1); iterative ν̂_t in iterative mode, cumulative ν̄_t in overall mode
`e(delta)`	DID estimates (T _post × 1)
`e(theta)`	Full parameter vector θ̂; pre-treatment block is ν̂_t (iterative mode) or ν̄_t (overall mode)
`e(Sigma)`	Asymptotic covariance matrix; in overall mode stored in overall-frame coordinates (A·Σ·A')

Compatibility returns such as e(b) and e(V) remain for Stata postestimation plumbing. They mirror delta_bar and its variance, so use e(delta_bar) as the substantive DID summary.

Mode Selection: Iterative vs. Overall

The package offers two violation summaries for parallel-trend problems, which have different sensitivities:

Feature	Iterative Mode (Default)	Overall Mode (`overall`)
Assumption	Violations accumulate period-to-period	Violations are bounded by cumulative total
Sensitivity	Sensitive tovolatility/noise (sharp changes)	Sensitive todrift/trend (long-term divergence)
Blind Spot	May pass smooth linear trends (constant small changes)	May fail even if period-to-period changes are small
Bias Bound	Scaled by κ (proportional to √T _post)	Uses the cumulative-violation path; no iterative κ multiplier
CI Width	Depends on the κ-scaled bias bound	Depends on the cumulative-violation path

How to compare the two modes:

Start with iterative mode when period-to-period volatility is the main concern.
Check overall when cumulative drift is the empirical question.
If one mode passes and the other does not, treat that split as evidence that the two violation summaries answer different questions, not as proof that one mode is uniformly more permissive or more informative.

Example

* Simulated panel data
clear
set seed 12345
set obs 500
gen id = ceil(_n/10)
gen time = mod(_n-1, 10) + 1
gen treat = (id <= 25)
gen y = rnormal() + treat*(time >= 6)*0.5

Future Roadmap

The development team is evaluating the following extensions for future versions:

Triple Difference-in-Differences (DDD): Extending the conditional extrapolation framework to DDD designs with an additional grouping dimension.
Staggered Adoption Support: Extending the conditional extrapolation framework to staggered treatment adoption designs via cohort stacking.
Covariate Adjustment: Adding support for control variables in the estimation.
Threshold Sensitivity Analysis: Visualizing how the pass-case conditional interval varies across a continuous range of $M$ values.

References

Mikhaeil, J. M., & Harshaw, C. (2025). In Defense of the Pre-Test: Valid Inference When Testing Violations of Parallel Trends for Difference-in-Differences. arXiv preprint arXiv:2510.26470. Available at: https://arxiv.org/abs/2510.26470

Rambachan, A., & Roth, J. (2023). A More Credible Approach to Parallel Trends. Review of Economic Studies, 90(5), 2555–2591. https://doi.org/10.1093/restud/rdad018

Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. American Economic Review: Insights, 4(3), 305–322. https://doi.org/10.1257/aeri.20210236

Authors

Stata Implementation:

Xuanyu Cai, City University of Macau Email: xuanyuCAI@outlook.com
Wenli Xu, City University of Macau Email: wlxu@cityu.edu.mo

Methodology:

Jonas M. Mikhaeil, Department of Statistics, Columbia University
Christopher Harshaw, Department of Statistics, Columbia University

License

AGPL-3.0. See LICENSE for details.

Citation

If you use this package in your research, please cite both the methodology paper and the Stata implementation:

APA Format:

Cai, X., & Xu, W. (2025). pretest: Thresholded DID diagnostics for conditional extrapolation (Version 0.1.1) [Computer software]. GitHub. https://github.com/gorgeousfish/pretest

Mikhaeil, J. M., & Harshaw, C. (2025). In Defense of the Pre-Test: Valid Inference when Testing Violations of Parallel Trends for Difference-in-Differences. arXiv preprint arXiv:2510.26470. https://arxiv.org/abs/2510.26470

BibTeX:

@software{pretest2025stata,
      title={pretest: Thresholded DID diagnostics for conditional extrapolation},
      author={Xuanyu Cai and Wenli Xu},
      year={2025},
      version={0.1.1},
      url={https://github.com/gorgeousfish/pretest}
}

@misc{mikhaeil2025defensepretestvalidinference,
      title={In Defense of the Pre-Test: Valid Inference when Testing Violations 
             of Parallel Trends for Difference-in-Differences}, 
      author={Jonas M. Mikhaeil and Christopher Harshaw},
      year={2025},
      eprint={2510.26470},
      archivePrefix={arXiv},
      primaryClass={stat.ME},
      url={https://arxiv.org/abs/2510.26470}
}

Recommended Resources

For beginners in causal inference and econometrics:

Causal Inference for the Brave and True - An excellent introductory tutorial on causal inference by Matheus Facure
Causal Inference for the Brave and True (Chinese Edition) - Chinese translation by Wenzhe Huang and Wenli Xu

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
ado		ado
examples		examples
image/README		image/README
mata		mata
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
_pretest_mata.do		_pretest_mata.do
build_mlib.do		build_mlib.do
lpretest.mlib		lpretest.mlib
pretest.ado		pretest.ado
pretest.pkg		pretest.pkg
pretest.sthlp		pretest.sthlp
prop99_smoking.dta		prop99_smoking.dta
rebuild_mlib.log		rebuild_mlib.log
stata.toc		stata.toc

Folders and files

Latest commit

History

Repository files navigation

pretest

Overview

First Run

Requirements

Data Completeness

Two-Period Designs

Installation

From GitHub

From SSC (Coming Soon)

Quick Start

Syntax

Required

Optional

Graph Customization

Key Formulas

Pre-test Indicator

Average DID Estimate

Pass-Case Conditional Interval (Theorem 2)

κ Constant (Iterative Mode Only)

Stored Results

Scalars

Matrices

Mode Selection: Iterative vs. Overall

Example

Future Roadmap

References

Authors

License

Citation

Recommended Resources

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages