Stress validation

Application-layer stress tests for the owned Julia causal stack. Package test/ suites stay lean; broader functionality, edge cases, recovery Monte Carlo, and wall-clock checks live in the CDCS application harness.

The distinctive demonstration is integration along the CDCS spine: Structural (identify / GraphSCM / $do$ / shared $\mathbf{u}$) → Dynamical (sequential / survival) → Observable (LMTP, mediation, real cohorts) → audit, with Turing for small $n$ and RxInfer for larger tables. Capability matrix: ECOSYSTEM_COMPARISON.md.

Methods notebook (Quarto): docs/stress/stress_validation.qmd runs dataset-by-dataset analyses with expected vs actual results, timings, and DAGMakie / plot_mtp_curve / posterior figures. Fixtures: docs/data/.

Canonical harness (data + runners): SimonAB/causal-dynamics-book → scripts/stress_harness/ and data/catalog.toml.

Julia↔R concordance (known-truth LMTP / mediation vs lmtp / crumble) remains under scripts/synthetic_benchmark/ in the same repository. Do not promote Super Learner defaults from a single Monte Carlo batch.

Packages under test

Related third-party packages

Design

Runners (scripts/stress_harness/):

1. Functionality — finite estimates or documented throws per catalog engine
2. Recovery MC — multi-seed absolute error and 95% coverage vs oracle (synthetic)
3. Performance — wall time for lean LMTP on mixed baselines and real microdata

Environment knobs: STRESS_PROFILE, STRESS_N, STRESS_SEEDS, STRESS_FOLDS, STRESS_T.

Methodological source papers

Full bibliographic list: References. Stress design leans on:

Estimation and policies

• Díaz, Williams, Hoffman & Schenck (2023). Nonparametric causal effects based on longitudinal modified treatment policies. JASA. doi:10.1080/01621459.2021.1955691
• Williams & Díaz (2023). lmtp: An R package for estimating the causal effects of modified treatment policies. Observational Studies. muse.jhu.edu/article/883479
• Díaz & Hejazi (2020). Causal mediation analysis for stochastic interventions. JRSS-B. doi:10.1111/rssb.12362
• Liu, Williams, Rudolph & Díaz (2024). General targeted machine learning for modern causal mediation analysis. arXiv:2408.14620
• van der Laan & Rose (2011). Targeted Learning. Springer — IPCW-TMLE and Super Learner practice

Missing data and positivity

• Petersen et al. (2012). Diagnosing and responding to violations in the positivity assumption. Stat Methods Med Res. doi:10.1177/0962280210386207
• Weberpals et al. (2024) — missing-data methods in TMLE (see harness notes / ISSUES)
• Berrevoets et al. (AISTATS 2023) — selective imputation for treatment-effect estimation

Identification (CausalDynamics)

• Pearl (2009). Causality (2nd ed.)
• Shpitser & Pearl (2006). Identification of joint interventional distributions… AAAI

Datasets and literature anchors

Catalog registry: data/catalog.toml in the book repository. Fixtures under data/fixtures/; processed tables under data/processed/ (gitignored raw downloads).

Conservation biology

Ecology (public microdata via Rdatasets)

Causal-inference benchmarks

Methods synthetics (in-package DGPs)

Exported or in-module generators exercised by the harness include simulate_linear_mtp, simulate_mixed_baseline_mtp, simulate_weak_positivity_mtp, simulate_missing_outcome_mtp, simulate_missing_covariate_mtp, simulate_discrete_survival_mtp, simulate_continuous_mtp_mediation, and harness factories for high-cardinality sites, wide baselines, and sequential panels. See src/synthetic.jl.

Smoke-profile freeze (2026-08-13)

Snapshots committed under scripts/stress_harness/results/ in the book repository:

Interpretation notes for recovery:

• linear_mtp / mixed_baseline_mtp / missing_outcome_mtp — small error on this freeze
• weak_positivity_mtp — noisy by design
• missing_covariate_mtp under :impute — soft failure mode until IPCW / imputation paths harden

Open gaps (tracked issues)

Closed in 2026-08-14 cycle: IPCW in LMTP/g-comp (CT#9); sequential missing Y under :drop (CT#10); survival MAR $S_T$ IPCW wiring; mean-only contrast guard (CT#11); fold-stable mediation schema (CT#8 / CM#3); GraphSCM sorted parent order (CD#8); g-comp refitting bootstrap (CT#13); mediation PPL handle_missing (CM#4).

Harness notes: ISSUES.md.

Reproducing the suite

From a checkout of causal-dynamics-book (with owned packages developed under packages/):

julia --project=. --threads=auto scripts/stress_harness/generate_synthetic.jl
julia --project=. --threads=auto scripts/stress_harness/fetch_real.jl --write-hash
STRESS_PROFILE=smoke julia --project=. --threads=auto scripts/stress_harness/run_functionality.jl
STRESS_PROFILE=smoke STRESS_SEEDS=5 julia --project=. --threads=auto scripts/stress_harness/run_recovery_mc.jl
STRESS_PROFILE=smoke julia --project=. --threads=auto scripts/stress_harness/run_performance.jl

Copy refreshed artefacts into scripts/stress_harness/results/ when freezing a new methods snapshot. Harness README: scripts/stress_harness/README.md.

Minimal local checks (no full catalog)

Without the book tree, package unit tests remain the merge gate:

julia --project=. -e 'using Pkg; Pkg.test()'

For a hand smoke of mixed baselines and missing strategies:

using CausalTargeted, StableRNGs
df, t = simulate_mixed_baseline_mtp(120; rng = StableRNG(1))
run_lmtp_grid(df, :A, :Y; baseline = t.baseline, deltas = [0.0, 0.5],
              folds = 2, learners_outcome = DEFAULT_SL_LEARNERS, parallel = false)

See also Small-n checklist and Methods and literature.