30  Hypothesis Generation from Counterfactuals

Status: Draft

v0.4

30.1 Introduction

“What would have happened?” can become a proposal for what to study next. This chapter develops that move within Imagining from the Observable role, after unit-level counterfactuals (Chapter 25) and before study design and reporting (Chapter 29). A generated hypothesis is a proposal for a discriminating observation or intervention, not confirmation of the counterfactual model.

In Peirce’s sense, this is abduction: the disciplined proposal of an explanatory candidate for further inquiry (Peirce 1934). It differs from SCM abduction, which conditions an already specified model on evidence to infer exogenous context. CDCS uses the former to propose a model or experiment, then uses identification, estimation, and criticism to assess the proposal.

On the host-immunity spine, a natural counterfactual is timing: what would this host’s assay trajectory have been under earlier vaccination? The knowledge gap (effect of early versus late \(do(A)\) under confounding by \(C\)) becomes a testable hypothesis and a trial or observational protocol; the full time-indexed CDM is in Chapter 28.

30.2 Counterfactuals → Hypotheses

30.2.1 The Connection

Counterfactual questions naturally lead to hypotheses:

  • Counterfactual: “What would have happened if we had treated earlier?”
  • Hypothesis: “Early treatment improves outcomes”
  • Study design: “Randomised trial comparing early vs late treatment”

30.2.2 Generating Hypotheses from Counterfactuals

Process:

  1. Ask counterfactual question: “What would have happened if X had been different?”
  2. Identify knowledge gap: What do we not know that prevents answering the counterfactual?
  3. Formulate hypothesis: State what we expect to find
  4. Design study: Create experiment to test the hypothesis

30.2.3 Example: Treatment Timing

Counterfactual question: “What would have happened if we had vaccinated this host earlier?”

Knowledge gap: We do not know the effect of early versus late treatment under confounding by nutrition or infection intensity (\(C\)), as in the host-immunity CDM.

Hypothesis: “Early treatment (within the first third of the observation window) raises mean assay \(Y_T\) relative to late treatment, after adjusting for \(C\).”

Study design: Randomised trial or sequential observational design comparing early versus late \(do(A)\), with confounder measurement aligned to the CDM lag structure.

30.2.4 Implementation: Counterfactual → Hypothesis Workflow

The following workflow carries a counterfactual question through to a testable hypothesis:

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(dirname(Base.active_project()), "scripts", "book_bootstrap.jl"))

@auto_using CausalDynamics Graphs

# Step 1: Counterfactual question
counterfactual_question = "What would have happened if we had treated this patient earlier?"

println("Step 1: Counterfactual Question")
println("  ", counterfactual_question)

# Step 2: Check identifiability
# Example graph: L → A → Y, L → Y (confounding)
g = SimpleDiGraph(3)
add_edge!(g, 1, 2)  # L → A
add_edge!(g, 2, 3)  # A → Y
add_edge!(g, 1, 3)  # L → Y

# Check if treatment effect is identifiable
adj_set = backdoor_adjustment_set(g, 2, 3)  # A → Y
is_identifiable = !isempty(adj_set)

println("\nStep 2: Check Identifiability")
if is_identifiable
    println("  ✓ Treatment effect is identifiable (adjust for L)")
    println("  → Can answer counterfactual from available data")
else
    println("  ✗ Treatment effect is NOT identifiable")
    println("  → Knowledge gap: cannot answer counterfactual")
end

# Step 3: If not identifiable, identify knowledge gap
if !is_identifiable
    println("\nStep 3: Identify Knowledge Gap")
    println("  Missing: Unmeasured confounder or insufficient data")
    println("  Need: Additional measurements or experimental design")
end

# Step 4: Formulate hypothesis
hypothesis = "Early treatment (within 24 hours) improves recovery compared to late treatment (after 48 hours)"
println("\nStep 4: Formulate Hypothesis")
println("  ", hypothesis)

# Step 5: Design study
study_design = "Randomised trial: Early treatment (t < 24h) vs Late treatment (t > 48h), controlling for baseline severity"
println("\nStep 5: Study Design")
println("  ", study_design)
Step 1: Counterfactual Question
  What would have happened if we had treated this patient earlier?

Step 2: Check Identifiability
  ✓ Treatment effect is identifiable (adjust for L)
  → Can answer counterfactual from available data

Step 4: Formulate Hypothesis
  Early treatment (within 24 hours) improves recovery compared to late treatment (after 48 hours)

Step 5: Study Design
  Randomised trial: Early treatment (t < 24h) vs Late treatment (t > 48h), controlling for baseline severity

30.3 Using Counterfactuals to Identify Knowledge Gaps

30.3.1 What We Know vs What We Don’t Know

Counterfactual reasoning reveals:

  • What we can answer: Counterfactuals that are identified from available data
  • What we cannot answer: Counterfactuals that are not identified (knowledge gaps)

30.3.2 Knowledge Gaps → Research Questions

When counterfactuals are not identified, this reveals knowledge gaps:

  • Unmeasured confounders: Need to measure additional variables
  • Unidentified mechanisms: Need to design experiments that identify mechanisms
  • Missing data: Need to collect data on specific variables

30.3.3 Example: Unmeasured Confounder

Counterfactual question: “What would have happened if treatment had been different?”

Problem: Unmeasured confounder \(U\) prevents identification.

Knowledge gap: We don’t know \(U\)’s distribution or its relationship to treatment and outcome.

Hypothesis: “If we measure \(U\), we can identify the treatment effect.”

Study design: Collect data on \(U\) in new study.

30.4 Bounds and Partial Identification → Hypotheses

When counterfactuals are only partially identified, bounds can guide hypothesis generation:

  • Wide bounds: Large uncertainty → hypothesis about reducing uncertainty
  • Narrow bounds: Small uncertainty → hypothesis about the effect direction
  • Sensitivity analysis: How bounds change → hypothesis about what variables matter

30.4.1 Example: Sensitivity Analysis

Counterfactual question: “What would have happened if treatment had been different?”

Result: Partial identification with bounds \([0.1, 0.9]\) for treatment effect.

Hypothesis: “Treatment effect is positive (lower bound > 0), but magnitude is uncertain.”

Study design: Design study to narrow the bounds (measure suspected confounders, increase sample size).

30.4.2 Implementation: Bounds → Hypothesis

We can generate hypotheses from partial identification bounds:

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(project_root, "scripts", "ensure_packages.jl"))

@auto_using CairoMakie

# Example: Partial identification with bounds
# Treatment effect bounds: [0.1, 0.9]
lower_bound = 0.1
upper_bound = 0.9
point_estimate = 0.5  # Midpoint (uncertain)

# Generate hypotheses from bounds
if lower_bound > 0
    hypothesis_direction = "Treatment effect is positive (lower bound > 0)"
elseif upper_bound < 0
    hypothesis_direction = "Treatment effect is negative (upper bound < 0)"
else
    hypothesis_direction = "Treatment effect direction is uncertain (bounds span zero)"
end

bound_width = upper_bound - lower_bound
if bound_width > 0.5
    hypothesis_uncertainty = "Large uncertainty → hypothesis: reduce uncertainty by measuring confounders"
else
    hypothesis_uncertainty = "Small uncertainty → hypothesis: effect direction is clear"
end

println("Bounds → Hypothesis:")
println("  Bounds: [", lower_bound, ", ", upper_bound, "]")
println("  Width: ", round(bound_width, digits=2))
println("  Hypothesis (direction): ", hypothesis_direction)
println("  Hypothesis (uncertainty): ", hypothesis_uncertainty)
Bounds → Hypothesis:
  Bounds: [0.1, 0.9]
  Width: 0.8
  Hypothesis (direction): Treatment effect is positive (lower bound > 0)
  Hypothesis (uncertainty): Large uncertainty → hypothesis: reduce uncertainty by measuring confounders
Figure 30.1: Generating hypotheses from partial identification bounds

30.5 From Hypotheses to Study Design

30.5.1 Hypothesis → Research Question

Once we have a hypothesis, we can formulate a research question:

  • Hypothesis: “Early treatment improves outcomes”
  • Research question: “Does early treatment (within 24 hours) improve recovery compared to late treatment (after 48 hours)?”

30.5.2 Research Question → Study Design

The research question guides study design:

  • What to measure: Outcome, treatment, confounders
  • When to measure: Timing of measurements
  • What interventions: Treatment strategies to compare
  • Design type: Randomised trial, observational study, etc.

30.6 Stratum context

Counterfactual reasoning can generate alternative observable outcomes and suggest what to study next. The resulting hypothesis is a proposal for a discriminating observation or intervention, not confirmation of the counterfactual model.

30.7 Summary

  1. Counterfactuals → hypotheses: “What would have happened” questions lead to testable hypotheses
  2. Knowledge gaps: Counterfactuals reveal what we don’t know
  3. Bounds guide hypotheses: Partial identification suggests what to study
  4. Hypotheses → study design: Counterfactual reasoning guides experimental design

30.8 Further Reading