36  Reporting Standards for Causal-Dynamical Claims

Status: Draft

v0.2

36.1 Learning Objectives

After reading this chapter, you will be able to:

• Use a comprehensive template for reporting causal-dynamical analyses
• Ensure all necessary components (estimand, assumptions, identification, estimator, diagnostics, sensitivity, limitations) are included
• Avoid common reporting pitfalls
• Create “reviewer-proof” causal claims

36.2 Introduction

Causal claims require careful justification. This chapter provides a “reviewer-proof” template for reporting causal-dynamical analyses, ensuring all necessary components are included and clearly communicated.

36.3 The Reporting Template

36.3.1 1. Estimand

What do we want to estimate?

• Clear definition: State the causal quantity of interest
• Notation: Use standard notation (\(do(\cdot)\), counterfactuals)
• Population: Define the target population
• Time horizon: Specify the time period of interest

Example: “We estimate the average treatment effect (ATE) of early vaccination on 6-month recovery, defined as \(\mathbb{E}[Y^{do(A=1)}] - \mathbb{E}[Y^{do(A=0)}]\) in the target population.”

36.3.2 2. Assumptions

What assumptions are we making?

• Causal structure: State the causal graph or structural assumptions
• Identifiability: What makes the estimand identifiable?
• No unmeasured confounding: Explicitly state or test
• Positivity: Treatment assignment probabilities
• Model assumptions: Functional forms, distributions

Example: “We assume no unmeasured confounding conditional on observed confounders \(L_t\), represented by the causal graph \(L_t \rightarrow A_t \rightarrow Y_t\), \(L_t \rightarrow Y_t\).”

36.3.3 3. Identification

How do we express the estimand in terms of observables?

• Identification result: Show how \(P^{do(\cdot)}(Y)\) relates to \(P(Y, A, X)\)
• Graphical criteria: Use backdoor/frontdoor/instrumental variable criteria
• G-formula: Show the identifying formula

Example: “Under no unmeasured confounding, the ATE is identified as \(\mathbb{E}[Y \mid A=1, L] - \mathbb{E}[Y \mid A=0, L]\) averaged over \(P(L)\).”

36.3.4 4. Estimator

How do we estimate the identified quantity?

• Method: Regression, matching, IPTW, TMLE, etc.
• Implementation: Software, packages, algorithms
• Hyperparameters: Tuning parameters, model selection
• Code: Provide reproducible code

Example: “We use targeted maximum likelihood estimation (TMLE) implemented in CausalTargeted.jl (run_lmtp_grid) for a continuous modified treatment policy, with Super Learner nuisances chosen by recommend_run_options (Díaz et al. 2023; Laan and Rose 2011).”

36.3.5 5. Diagnostics

How do we check the model and assumptions?

• Model criticism: Calibration, PPCs, residuals
• Assumption checks: Test for unmeasured confounding, positivity violations
• Sensitivity analysis: How robust are results?
• Validation: Out-of-domain, cross-validation

Example: “Posterior predictive checks show good calibration (calibration slope = 0.98). Sensitivity analysis suggests results are robust to moderate unmeasured confounding (E-value = 2.3).”

36.3.6 6. Sensitivity

How sensitive are results to assumptions?

• Sensitivity parameters: Unmeasured confounding strength, model misspecification
• Sensitivity analysis: Vary assumptions, report how results change
• Robustness: Worst-case scenarios, bounds

Example: “Under the assumption that unmeasured confounders have odds ratio ≤ 2.0 with both treatment and outcome, the ATE remains positive (95% CI: [0.1, 0.5]).”

36.3.7 7. Limitations

What are the limitations?

• Assumptions: Which assumptions may not hold?
• Data: What data limitations affect results?
• Generalisation: To what populations/domains do results apply?
• Uncertainty: What uncertainties remain?

Example: “Results assume no unmeasured confounding, which may not hold if unobserved factors (e.g., genetic variants) affect both treatment and outcome. Generalisation to other populations requires transportability assumptions.”

36.4 Mapping the template to the CDCS stack

Template item	CausalDynamics	CausalTargeted	Notes
Estimand	CausalQuery / TemporalEffectQuery	InterventionalMean, MediationContrast, SequentialPolicy	Prefer typed estimands over ad hoc strings
Identification	identify → IdentificationResult	identification_certificate, attach_run_metadata!	Provenance columns on every result table
Positivity	—	positivity_report, grid positivity=true	(Petersen et al. 2012; Díaz et al. 2023)
Mediation MC error	—	mediation_n_mc_sweep, mediation_stability_summary	(Liu et al. 2024)
Omitted confounding	E-value prose (Ch. 05, 17)	sensitivity_report (tipping-point / partial \(R^2\))	(Cinelli and Hazlett 2020; VanderWeele and Ding 2017)
Discovery	Associations → prepare_from_discovery	merge_discovery_sensitivity!	Sensitivity only—never silent DAG replacement

TipSmall-n checklist (CausalTargeted)

For continuous MTP and mediation with tens to low hundreds of units (Díaz et al. 2023; Laan and Rose 2011):

1. Start from recommend_run_options(n; engine, n_mediators) (lean Super Learner when \(n < 80\); parallel=false by default).
2. Inspect positivity / support (positivity_report or grid positivity=true) (Petersen et al. 2012).
3. For mediation, sweep nested MC (mediation_n_mc_sweep) until signs and SEs stabilise (Liu et al. 2024).
4. Report tipping-point / partial-\(R^2\) sensitivity (sensitivity_report) (Cinelli and Hazlett 2020).
5. Treat discovery graphs as sensitivity only (merge_discovery_sensitivity!), never as silent DAG replacement.

Package prose and DOIs: CausalTargeted methods · references. Worked δ-grid: Chapter 23 (Policy Evaluation).

36.5 Checklists

36.5.1 Transportability Checklist

• Domain variables: Are domain variables (\(C\)) explicitly included?
• Invariance: Which mechanisms are assumed invariant?
• Domain-specific: Which mechanisms vary across domains?
• Validation: Has the model been validated in target domain?
• Sensitivity: How do results change with domain assumptions?

36.5.2 Robustness Checklist

• Parameter robustness: How sensitive are results to parameter values?
• Structural robustness: How sensitive are results to structural assumptions?
• Model robustness: How sensitive are results to model choice?
• Design robustness: How sensitive are results to experimental design?
• Bounds: What are worst-case scenarios?

36.6 Common Reporting Pitfalls

36.6.1 Pitfall 1: Missing Assumptions

Problem: Not stating assumptions explicitly.

Solution: Use the template—always state assumptions.

36.6.2 Pitfall 2: Confusing Association and Causation

Problem: Reporting associations as causal effects.

Solution: Use \(do(\cdot)\) notation, state identification assumptions.

36.6.3 Pitfall 3: Ignoring Uncertainty

Problem: Not reporting uncertainty (confidence intervals, sensitivity).

Solution: Always report uncertainty and sensitivity analysis.

36.6.4 Pitfall 4: Overstating Generalisation

Problem: Claiming results generalise without justification.

Solution: State transportability assumptions, validate in target domain.

36.7 Key Takeaways

1. Use the reporting template to ensure all necessary components are included
2. Always state assumptions explicitly
3. Use standard notation (\(do(\cdot)\), counterfactuals) for clarity
4. Report uncertainty and sensitivity analysis
5. State limitations and generalisation assumptions
6. Use checklists to avoid common pitfalls

36.8 Further Reading

• Hernán and Robins (2020): Causal Inference: What If — comprehensive guide to reporting causal analyses
• VanderWeele (2015): Explanation in Causal Inference — reporting standards for causal mediation and interaction
• Rothman et al. (2021): Modern Epidemiology (4th ed.) — reporting standards for epidemiological studies
• Díaz et al. (2023): LMTP estimands and software practice
• Cinelli and Hazlett (2020): Partial-\(R^2\) sensitivity reporting
• Software Engineering Patterns: Package stack and small-\(n\) defaults

Cinelli, Carlos, and Chad Hazlett. 2020. “Making Sense of Sensitivity: Extending Omitted Variable Bias.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82 (1): 39–67. https://doi.org/10.1111/rssb.12348.

Díaz, Iván, Nicholas Williams, Katherine L. Hoffman, and Edward J. Schenck. 2023. “Nonparametric Causal Effects Based on Longitudinal Modified Treatment Policies.” Journal of the American Statistical Association 118 (542): 846–57. https://doi.org/10.1080/01621459.2021.1955691.

Hernán, Miguel A., and James M. Robins. 2020. Causal Inference: What If. Chapman & Hall/CRC.

Laan, Mark J. van der, and Sherri Rose. 2011. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer Series in Statistics. Springer.

Liu, Richard, Nicholas T. Williams, Kara E. Rudolph, and Iván Díaz. 2024. General Targeted Machine Learning for Modern Causal Mediation Analysis. https://doi.org/10.48550/arXiv.2408.14620.

Petersen, Maya L., Kristin E. Porter, Susan Gruber, Yue Wang, and Mark J. van der Laan. 2012. “Diagnosing and Responding to Violations in the Positivity Assumption.” Statistical Methods in Medical Research 21 (1): 31–54. https://doi.org/10.1177/0962280210386207.

Rothman, Kenneth J., Sander Greenland, and Timothy L. Lash. 2021. Modern Epidemiology. 4th ed. Lippincott Williams & Wilkins.

VanderWeele, Tyler J. 2015. Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford University Press.

VanderWeele, Tyler J., and Peng Ding. 2017. “Sensitivity Analysis in Observational Research: Introducing the e-Value.” Annals of Internal Medicine 167 (4): 268–74.