34  Reporting Standards for Causal-Dynamical Claims

Status: Draft

v0.2

34.1 Introduction

Causal claims require careful justification. This chapter is part of Part IV’s practice of Reason: a reviewer-facing template for reporting causal-dynamical analyses after the CDM synthesis (Chapter 28), so semantic declarations, estimands, identification, estimators, observation assumptions, and diagnostics stay explicit.

Before reporting a result, record what each variable refers to, its value type and temporal support, the descriptive scale and identity criterion, and whether it is endogenous, exogenous, observed, latent, or a parameter. For each relation, state whether it is causal influence, constitution, participation, measurement, or temporal order. For each intervention, state the target, replacement rule, timing, duration, available information, admissibility conditions, and mechanisms assumed unchanged. A graph or equation without these declarations can be formally neat while leaving the scientific claim ambiguous.

34.2 Workflow provenance

Report enough of the model-development path that a reader can distinguish a successful computation from a supported causal claim. Record:

  • the declared strata, referents, temporal support, graph, intervention, and estimand;
  • the identification result or certificate and the assumptions on which it depends;
  • the components fitted or estimated, including transition laws, observation maps, filters, nuisance functions, and Bayesian heads where applicable;
  • the checks run on simulated and recorded data, including oracle recovery, calibration, posterior predictive checks, trajectory checks, positivity, sensitivity, and out-of-domain validation;
  • any failed check, the component or assumption that was changed, and whether identification was re-established afterwards;
  • the final claim type: predictive, interventional, counterfactual, transport-based, or decision-oriented.

An identification certificate is evidence that an estimand follows from the declared structure and assumptions. It is not evidence that the structure is true. Likewise, predictive calibration supports the relevant observation or forecasting component but does not establish causal validity. Report these claims separately.

34.3 The Reporting Template

34.3.1 1. Estimand

What do we want to estimate?

  • Clear definition: State the causal quantity of interest
  • Notation: Use standard notation (\(do(\cdot)\), counterfactuals)
  • Population: Define the target population
  • Time horizon: Specify the time period of interest

Example: “We estimate the average treatment effect (ATE) of early vaccination on 6-month recovery, defined as \(\mathbb{E}[Y^{do(A=1)}] - \mathbb{E}[Y^{do(A=0)}]\) in the target population.”

Example (host-immunity CDM): “On the confounded biomarker CDM of Chapter 28, we report \(E[Y_T \mid do(A_{1:T}=1)] - E[Y_T \mid do(A_{1:T}=0)]\) from g_computation, then the corresponding sequential LMTP contrast from observational panels under adjustment for \(C\).”

34.3.2 2. Assumptions

What assumptions are we making?

  • Causal structure: State the causal graph or structural assumptions
  • Identifiability: What makes the estimand identifiable?
  • Exchangeability: State and defend the no-unmeasured-confounding assumption; it is not definitively testable from the observed distribution alone
  • Positivity: Treatment assignment probabilities
  • Model assumptions: Functional forms, distributions

Example: “We assume no unmeasured confounding conditional on observed confounders \(L_t\), represented by the causal graph \(L_t \rightarrow A_t \rightarrow Y_t\), \(L_t \rightarrow Y_t\).”

34.3.3 3. Identification

How do we express the estimand in terms of observables?

  • Identification result: Show how \(P^{do(\cdot)}(Y)\) relates to \(P(Y, A, X)\)
  • Graphical criteria: Use backdoor/frontdoor/instrumental variable criteria
  • G-formula: Show the identifying formula

Example: “Under no unmeasured confounding, the ATE is identified as \(\mathbb{E}[Y \mid A=1, L] - \mathbb{E}[Y \mid A=0, L]\) averaged over \(P(L)\).”

34.3.4 4. Estimator

How do we estimate the identified quantity?

  • Method: Regression, matching, IPTW, TMLE, etc.
  • Implementation: Software, packages, algorithms
  • Hyperparameters: Tuning parameters, model selection
  • Code: Provide reproducible code

Example (LMTP): “We use a continuous modified treatment policy estimated with CausalTargeted.jl (run_lmtp_grid), with Super Learner nuisances chosen by recommend_run_options (Díaz et al. 2023; Laan and Rose 2011).”

Example (survival LMTP): “We estimate event-free probability at horizon \(h=3\) under an additive MTP on time-ordered treatments with CausalTargeted.jl (SurvivalPolicy / run_survival_lmtp), after temporal backdoor identification via CausalDynamics TemporalEffectQuery (Díaz et al. 2024, 2023).”

Example (mediation): “We identify a MediationQuery with CausalDynamics.jl, then estimate interventional NDE/NIE with CausalMediation.jl (spec_from_identification → run_mediation) (Díaz and Hejazi 2020; Liu et al. 2024).”

34.3.5 5. Diagnostics

How do we check the model and assumptions?

  • Model criticism: Calibration, PPCs, residuals
  • Assumption probes: Diagnose positivity; use negative controls, triangulation, sensitivity analysis or design information to probe exchangeability without calling these definitive tests
  • Sensitivity analysis: How robust are results?
  • Validation: Out-of-domain, cross-validation

Example: “Posterior predictive checks show good calibration (calibration slope = 0.98), supporting the observation model but not establishing the causal graph. Sensitivity analysis reports robustness to a stated confounding scenario (E-value = 2.3).”

34.3.6 Validation targets

Validation target Appropriate evidence
Observation model Residuals, calibration and posterior predictive checks
Dynamical model Held-out trajectories, perturbation response and parameter recovery
Causal structure Interventional data, invariance tests, negative controls and competing graphs
Transport Target-domain data, selection diagrams and mechanism comparison

For latent CDMs, add simulation-based calibration and parameter-recovery experiments. Qualify or reject a model where observationally equivalent parameterisations yield materially different interventional conclusions.

The validation target determines where a failed check sends the analysis: to the observation model, dynamics, computation, identification assumptions, or transport claim. Do not treat a green check in one row as a certificate for the other rows.

34.3.7 6. Sensitivity

How sensitive are results to assumptions?

  • Sensitivity parameters: Unmeasured confounding strength, model misspecification
  • Sensitivity analysis: Vary assumptions, report how results change
  • Robustness: Worst-case scenarios, bounds

Example: “Under the assumption that unmeasured confounders have odds ratio ≤ 2.0 with both treatment and outcome, the ATE remains positive (95% CI: [0.1, 0.5]).”

34.3.8 7. Limitations

What are the limitations?

  • Assumptions: Which assumptions may not hold?
  • Data: What data limitations affect results?
  • Generalisation: To what populations/domains do results apply?
  • Uncertainty: What uncertainties remain?

Example: “Results assume no unmeasured confounding, which may not hold if unobserved factors (e.g., genetic variants) affect both treatment and outcome. Generalisation to other populations requires transportability assumptions.”

34.4 Mapping the template to the CDCS stack

Template item CausalDynamics CausalTargeted CausalMediation Notes
Estimand CausalQuery / TemporalEffectQuery / MediationQuery InterventionalMean, SequentialPolicy, SurvivalPolicy MediationSpec, effect kinds Prefer typed estimands over ad hoc strings
Identification identify → IdentificationResult identification_certificate, attach_run_metadata! spec_from_identification, plan_mediation Provenance columns on every result table
Positivity — positivity_report, grid positivity=true — (Petersen et al. 2012; Díaz et al. 2023)
Mediation MC error — façade when CM loaded mediation_n_mc_sweep, mediation_stability_summary (Liu et al. 2024)
Omitted confounding E-value prose (Ch. 05, 17) sensitivity_report (tipping-point / partial \(R^2\)) — (Cinelli and Hazlett 2020; VanderWeele and Ding 2017)
Discovery Associations → prepare_from_discovery merge_discovery_sensitivity! — Sensitivity only, never silent DAG replacement
Figures same graph — — DAGMakie dagplot from the same graph / certificate
TipSmall-n checklist (CausalTargeted)

For continuous MTP and mediation with tens to low hundreds of units (Díaz et al. 2023; Laan and Rose 2011):

  1. Start from recommend_run_options(n; engine, n_mediators) (lean Super Learner when \(n < 80\); parallel=false by default).
  2. Pass mixed-type baselines as-is (String / Bool / categorical are encoded with a fold-stable CovariateSchema in LMTP, sequential, survival, and mediation grids).
  3. Inspect positivity / support (positivity_report or grid positivity=true) (Petersen et al. 2012).
  4. For mediation, sweep nested MC (mediation_n_mc_sweep) until signs and SEs stabilise (Liu et al. 2024).
  5. Report tipping-point / partial-\(R^2\) sensitivity (sensitivity_report) (Cinelli and Hazlett 2020).
  6. Treat discovery graphs as sensitivity only (merge_discovery_sensitivity!), never as silent DAG replacement.

Package prose and DOIs: CausalTargeted methods · references. Worked δ-grid: Chapter 23 (Policy Evaluation).

34.5 Worked report: the shared protocol fixture

The following report uses the seeded synthetic fixture shared by the four owned packages. It is an integration check, not evidence about a clinical or ecological population. The estimand is the contrast in outcome at t = 2 under two baseline assignment policies, with baseline adjustment supplied by the temporal identification certificate.

project_root_report = dirname(Base.active_project())
include(joinpath(project_root_report, "scripts", "ensure_packages.jl"))
@auto_using CausalDynamics CausalTargeted CausalMediation DAGMakie StableRNGs
include(joinpath(project_root_report, "scripts", "cdcs_shared_fixture.jl"))

report_fixture = build_cdcs_shared_fixture(
    n = 160,
    horizon = 3,
    seed = 20260930,
)
report_id = report_fixture.temporal_identification
report_est = CausalTargeted.run_discrete_lmtp_contrast(
    report_fixture.data,
    :assignment,
    :outcome;
    arm_hi = "1",
    arm_ref = "0",
    levels = ["0", "1"],
    baseline = [:baseline],
    folds = 2,
    learners_outcome = (:glm, :mean),
    rng = StableRNG(20260932),
)
(
    estimand = "E[Y₂ | do(assignment=1)] − E[Y₂ | do(assignment=0)]",
    identification_status = report_id.identification_status,
    adjustment = report_id.adjustment,
    estimate = report_est.estimate,
    standard_error = report_est.se,
    confidence_interval = (report_est.lower, report_est.upper),
    positivity = report_est.positivity,
)
(estimand = "E[Y₂ | do(assignment=1)] − E[Y₂ | do(assignment=0)]", identification_status = :identified, adjustment = [:baseline], estimate = 1.0906087046196764, standard_error = 0.07970514164387262, confidence_interval = (0.9343894976150222, 1.2468279116243306), positivity = (ok = true, hi = (n_observed_levels = 2, n_policy_levels = 1, empty_support = String[], ok = true), ref = (n_observed_levels = 2, n_policy_levels = 1, empty_support = String[], ok = true)))

The graph declares baseline as a from-onset context, assignment and protocol as later single-node attributes, pointwise implementation states, and an outcome measured at t = 2. The certificate therefore records baseline as the adjustment variable. CausalTargeted supplies the cross-fitted discrete LMTP contrast; the protocol mediator is handled separately by CausalMediation. The result should be interpreted alongside the synthetic data-generating equations and the overlap diagnostic, not as an empirical claim about a target population.

34.6 Checklists

34.6.1 Transportability Checklist

34.6.2 Robustness Checklist

34.7 Common Reporting Pitfalls

34.7.1 Pitfall 1: Missing Assumptions

Problem: Not stating assumptions explicitly.

Solution: Use the template, always state assumptions.

34.7.2 Pitfall 2: Confusing Association and Causation

Problem: Reporting associations as causal effects.

Solution: Use \(do(\cdot)\) notation, state identification assumptions.

34.7.3 Pitfall 3: Ignoring Uncertainty

Problem: Not reporting uncertainty (confidence intervals, sensitivity).

Solution: Always report uncertainty and sensitivity analysis.

34.7.4 Pitfall 4: Overstating Generalisation

Problem: Claiming results generalise without justification.

Solution: State transportability assumptions, validate in target domain.

34.8 Summary

  1. Use the reporting template to ensure all necessary components are included
  2. Always state assumptions explicitly
  3. Use standard notation (\(do(\cdot)\), counterfactuals) for clarity
  4. Report uncertainty and sensitivity analysis
  5. State limitations and generalisation assumptions
  6. Use checklists to avoid common pitfalls

34.9 Further Reading

  • Hernán and Robins (2020): Causal Inference: What If: comprehensive guide to reporting causal analyses
  • VanderWeele (2015): Explanation in Causal Inference: reporting standards for causal mediation and interaction
  • Rothman et al. (2021): Modern Epidemiology (4th ed.), reporting standards for epidemiological studies
  • Díaz et al. (2023): LMTP estimands and software practice
  • Díaz et al. (2024): Survival / event-time LMTP reporting
  • Cinelli and Hazlett (2020): Partial-\(R^2\) sensitivity reporting
  • Software Engineering Patterns: Package stack and small-\(n\) defaults