22  From Dynamical to Observable: Records and Actualisation

Status: Draft

v0.5

22.1 Introduction

This chapter bridges the Dynamical and Observable strata. A realised process does not wait for an assay to become actual: an unobserved infection, cellular state or ecological transition is already part of the world. Observation is a further causal interaction that produces a record in a specified design. This is the transition from modelling latent process to modelling the data-generating interaction.

Process refrain. \(X_t\) represents an already realised process; \(Y_t\) is a partial empirical registration of it. Within the philosophical vocabulary of the book, actualisation concerns the passage from structured possibilities to realised process; a trajectory \(X_{0:T}\) represents that process at the chosen scale rather than exhausting its becoming.

22.2 How Dynamic Processes Are Measured

22.2.1 The Observation Model

For non-perturbative measurement, the familiar observation model is:

\[ Y_t = h(X_t, C, U^y_t) \]

where:

• \(X_t\) is the realised latent process
• \(Y_t\) is the recorded observation
• \(h\) is the record-generating map
• \(U^y_t\) is the measurement-side exogenous variable; its realised contribution may include instrument error, sampling variation, detection limits, or other declared observation assumptions

22.2.2 Measurement Interaction and Record

When the measurement apparatus can perturb the process, make that interaction explicit:

\[ M_{t+1} \coloneqq g(M_t, X_t, D_t, U^m_{t+1}), \qquad Y_t \coloneqq h(M_t, U^y_t), \]

where \(D_t\) specifies assay timing, instrument, sampling intensity or censoring protocol. \(M_t\) may be omitted when the direct observation model is adequate. The distinction prevents measurement from being mistaken for the physical existence of \(X_t\).

22.2.3 Implementation: Observation Models

The following comparison shows how different observation models record the same latent process (Figure 22.1):

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(dirname(Base.active_project()), "scripts", "book_bootstrap.jl"))

@auto_using Random Distributions OrdinaryDiffEq CairoMakie

Random.seed!(42)

# Example: Latent state X_t evolving over time
function latent_dynamics!(du, u, p, t)
    """Latent state dynamics: exponential decay with rate 0.1."""
    X = u[1]
    du[1] = -0.1 * X  # Decay process
end

u0 = [1.0]
tspan = (0.0, 20.0)
prob = ODEProblem(latent_dynamics!, u0, tspan)
sol = solve(prob, Tsit5())

X_t = [u[1] for u in sol.u]  # Latent state

# Type 1: Direct measurement Y_t = X_t + noise
σ_direct = 0.1
Y_direct = X_t .+ rand(Normal(0, σ_direct), length(X_t))

# Type 2: Indirect measurement Y_t = h(X_t) + noise
# Example: h(X) = X² (nonlinear observation)
Y_indirect = X_t.^2 .+ rand(Normal(0, 0.05), length(X_t))

# Type 3: Aggregate measurement (if we had multiple states)
# Y_t = Σ X_i + noise (simplified here)
Y_aggregate = X_t .+ rand(Normal(0, 0.1), length(X_t))  # Simplified

println("Observation models:")
println("  Direct: Y_t = X_t + noise (σ = ", σ_direct, ")")
println("  Indirect: Y_t = X_t² + noise (nonlinear)")
println("  Inference: Must recover X_t from Y_t (addressed in state-space models)")

Observation models:
  Direct: Y_t = X_t + noise (σ = 0.1)
  Indirect: Y_t = X_t² + noise (nonlinear)
  Inference: Must recover X_t from Y_t (addressed in state-space models)

Figure 22.1: Different observation models: how latent states become observable

22.3 Partial Observability

22.3.1 The Problem

In most real systems, we cannot directly observe the latent state \(X_t\):

• Partial observability: We only observe \(Y_t\), not \(X_t\)
• Measurement noise: Observations are noisy
• Missing data: Some observations may be missing; the missingness mechanism is itself a causal story (next subsection)

22.3.2 Missingness mechanisms as DAGs

Classical labels MCAR / MAR / MNAR are claims about what causes a recording to fail (Little and Rubin 2019). A DAG makes those claims explicit. Book notation (Concept Reference): complete cover \(C\), response \(R_C\) (\(R_C=1\) means recorded), recorded assay \(C^{\mathrm{rec}}\) (figure alias \(C^*\)). Arrows into \(R_C\) decide whether imputation from covariates is justified. Older teaching labels sometimes write failure \(F=1-R_C\); we keep \(R_C\) so figures match CausalDynamics certificates and IPCW.

Rename the usual rainfall–richness teaching graph to a forest understorey survey: soil moisture \(W\), understorey cover \(C\) (latent when missing), response \(R_C\), and (in one scenario) management intensity \(Z\).

Scenario	Causal claim	Practical consequence
(a) MCAR	\(R_C\) unrelated to \(W,C\)	Complete-case analysis unbiased for \(W\to C\)
(b) MAR	\(W \to R_C\)	Impute \(C\) using \(W\); fills \(\tilde{C}\) remain an Observable policy
(c) Confounded	\(Z\to C\), \(Z\to R_C\)	Must include \(Z\) in the imputation model
(d) MNAR	\(C\to R_C\)	Missingness depends on the missing value; needs stronger assumptions

include(joinpath(dirname(Base.active_project()), "scripts", "book_bootstrap.jl"))
@auto_using DAGMakie CairoMakie Graphs

# Nodes: 1=W, 2=C, 3=C* (≡ C^rec), 4=R_C [, 5=Z]
g_mcar = SimpleDiGraph(4)
add_edge!(g_mcar, 1, 2); add_edge!(g_mcar, 2, 3); add_edge!(g_mcar, 4, 3)

g_mar = SimpleDiGraph(4)
add_edge!(g_mar, 1, 2); add_edge!(g_mar, 2, 3); add_edge!(g_mar, 4, 3); add_edge!(g_mar, 1, 4)

g_conf = SimpleDiGraph(5)
add_edge!(g_conf, 1, 2); add_edge!(g_conf, 2, 3); add_edge!(g_conf, 4, 3)
add_edge!(g_conf, 5, 2); add_edge!(g_conf, 5, 4)

g_mnar = SimpleDiGraph(4)
add_edge!(g_mnar, 1, 2); add_edge!(g_mnar, 2, 3); add_edge!(g_mnar, 4, 3); add_edge!(g_mnar, 2, 4)

# Planar grid: W→C on top, R_C→C* on bottom. For MNAR, place R_C under C
# so C→R_C is vertical rather than a diagonal across the square. Z sits to the left.
layout4 = Point2f[
    Point2f(0.0, 1.0), Point2f(1.6, 1.0), Point2f(1.6, 0.0), Point2f(0.0, 0.0),
]
layout4_mnar = Point2f[
    Point2f(0.0, 1.0), Point2f(1.6, 1.0), Point2f(2.8, 1.0), Point2f(1.6, 0.0),
]
layout5 = Point2f[
    Point2f(1.2, 1.0), Point2f(2.8, 1.0), Point2f(2.8, 0.0), Point2f(1.2, 0.0),
    Point2f(0.0, 0.5),
]
const _miss_lab4 = ["W", "C", "C*", "R_C"]
const _miss_lab5 = ["W", "C", "C*", "R_C", "Z"]
# Observed / latent-complete / recorded / response / confounder
const _col_obs = RGBf(0.30, 0.55, 0.70)
const _col_lat = RGBf(0.85, 0.65, 0.05)
const _col_R = RGBf(0.55, 0.55, 0.55)

Four missingness stories for a forest cover survey. \(W\): soil moisture; \(C\): understorey cover (latent when missing); \(C^*\equiv C^{\mathrm{rec}}\): recorded cover; \(R_C\): response (\(R_C=1\) observed); \(Z\): management intensity. Gold marks the complete cover; grey marks the recording process.

The Observable stratum inherits these graphs: \(h\) says how cover becomes a number and when that number is written down. Imputation models that ignore arrows into \(R_C\) are misspecified for (c) and (d). Under MAR (\(W \to R_C\)), a Bayesian regression of cover on moisture fitted on observed rows, then predicting missing rows, yields fills \(\tilde{C}\) under a declared observation policy. Treat the fit as part of the same criticism loop as elsewhere: if you can, recover known \((\beta_0,\beta_w)\) from simulated MAR data before trusting imputations on the field plot (Gelman et al. 2026); posterior predictive and residual checks for observation heads are in Chapter 21.

using Turing: Turing, @model, sample, NUTS
using Distributions
using Statistics
using Random

rng = Random.Xoshiro(11)
n = 200
W = randn(rng, n)
C_true = 0.2 .+ 0.8 .* W .+ 0.3 .* randn(rng, n)
miss = W .> 0.5                          # MAR: failure depends on moisture
W_obs, C_obs = W[.!miss], C_true[.!miss]

Turing.@model function cover_on_moisture(W, C)
    β0 ~ Normal(0, 2)
    βw ~ Normal(0, 2)
    σ ~ truncated(Normal(1, 1); lower = 0.05)
    for i in eachindex(C)
        C[i] ~ Normal(β0 + βw * W[i], σ)
    end
end

t_mar = @elapsed begin
    global ch_mar = sample(cover_on_moisture(W_obs, C_obs), NUTS(0.8), 400; progress = false)
end
β0̂, βŵ = mean(ch_mar[:β0]), mean(ch_mar[:βw])
C_imp = β0̂ .+ βŵ .* W
mae_miss = mean(abs.(C_imp[miss] .- C_true[miss]))
println("MAR imputer: βw≈", round(βŵ; digits = 3),
    "  MAE on missing = ", round(mae_miss; digits = 3),
    "  (", round(t_mar; digits = 3), " s)")

MAR imputer: βw≈0.801  MAE on missing = 0.212  (4.563 s)

MAR imputation for understorey cover: true \(C\) versus posterior-mean predictions. Orange points are units with survey failure (\(W>0.5\)); blue are observed.

MNAR (\(C\to R_C\)) needs a joint model for \((C,R_C)\); the MAR imputer above is then misspecified. Full Bayesian imputation belongs with estimation tooling; the structural point remains that missingness is part of the measurement DAG.

The graph also determines which quantities can be recovered from incomplete records. MNAR does not by itself settle that question: some missingness graphs permit recovery of a particular target even when the complete joint distribution cannot be recovered (Mohan and Pearl 2018). Chapter 19 connects this structural question to imputation and causal estimation.

22.3.3 Inference Problem

Given observations \(Y_{1:T}\), we must infer the latent state \(X_{1:T}\):

\[ P(X_{1:T} \mid Y_{1:T}) \]

This is the inference problem addressed in State-Space Models: Inferring Structure from Observations.

22.3.4 Implementation: Partial Observability

The next example makes the inference problem explicit: \(Y_t\) is observed, but \(X_t\) is not (Figure 22.2):

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(project_root, "scripts", "ensure_packages.jl"))

@auto_using Random Distributions CairoMakie

Random.seed!(42)

# Example: We observe Y_t but not X_t
# Latent state: X_t (unknown)
# Observation: Y_t = X_t + noise

# Simulate true latent state
T = 50
X_true = zeros(T)
X_true[1] = 1.0
for t in 2:T
    X_true[t] = 0.9 * X_true[t-1] + rand(Normal(0, 0.1))  # AR(1) process
end

# Observations with noise
σ_obs = 0.2
Y_obs = X_true .+ rand(Normal(0, σ_obs), T)

# Inference problem: Given Y_obs, infer X_true
# Simple approach: Use observations directly (ignoring dynamics)
X_inferred_naive = Y_obs

# Better approach: Use dynamics + observations (Kalman filter - simplified)
# Here we show a simple smoothing approach
X_inferred_smooth = zeros(T)
α = 0.5  # Smoothing parameter
for t in 1:T
    if t == 1
        X_inferred_smooth[t] = Y_obs[t]
    else
        # Combine observation with prediction from dynamics
        X_pred = 0.9 * X_inferred_smooth[t-1]
        X_inferred_smooth[t] = α * Y_obs[t] + (1 - α) * X_pred
    end
end

println("Partial observability:")
println("  True latent state: X_t (unknown)")
println("  Observations: Y_t = X_t + noise")
println("  Inference problem: Recover X_t from Y_t")
println("  Naive approach: Use Y_t directly (ignores dynamics)")
println("  Better approach: Combine observations with dynamics (state-space inference)")

Partial observability:
  True latent state: X_t (unknown)
  Observations: Y_t = X_t + noise
  Inference problem: Recover X_t from Y_t
  Naive approach: Use Y_t directly (ignores dynamics)
  Better approach: Combine observations with dynamics (state-space inference)

Figure 22.2: Partial observability: inferring latent states from noisy observations

22.4 Measurement Processes

22.4.1 Types of Measurement

• Direct measurement: \(Y_t = X_t + \text{noise}\) (observe state directly with noise)
• Indirect measurement: \(Y_t = h(X_t) + \text{noise}\) (observe function of state)
• Aggregate measurement: \(Y_t = \sum_i X^i_t + \text{noise}\) (observe sum of states)
• Delayed measurement: \(Y_t = X_{t-k} + \text{noise}\) (observe past state)

22.4.2 Measurement Design

Question: What should we measure to maximise information about latent states?

Answer: Design measurements that maximise mutual information \(I(X_t; Y_t)\).

22.4.3 Implementation: Measurement Design

The final example shows how measurement choices affect the information available about the process (Figure 22.3):

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(project_root, "scripts", "ensure_packages.jl"))

@auto_using Random Distributions CairoMakie

Random.seed!(42)

# Example: Two possible measurement functions
# Option 1: Y₁ = X + noise (direct, high information)
# Option 2: Y₂ = sign(X) + noise (binary, lower information)

# Simulate latent state
n = 1000
X = rand(Normal(0, 1), n)

# Measurement option 1: Direct (high information)
σ1 = 0.1
Y1 = X .+ rand(Normal(0, σ1), n)

# Measurement option 2: Binary (lower information)
Y2 = sign.(X) .+ rand(Normal(0, 0.1), n)

# Approximate mutual information I(X; Y) ≈ H(X) - H(X|Y)
# Using correlation as proxy (higher correlation → higher mutual information)
cor1 = abs(cor(X, Y1))
cor2 = abs(cor(X, Y2))

println("Measurement design comparison:")
println("  Option 1 (direct): Correlation = ", round(cor1, digits=3), " (high information)")
println("  Option 2 (binary): Correlation = ", round(cor2, digits=3), " (lower information)")
println("  → Option 1 provides more information about X")
println("  → Design measurements to maximize I(X; Y)")

Measurement design comparison:
  Option 1 (direct): Correlation = 0.995 (high information)
  Option 2 (binary): Correlation = 0.793 (lower information)
  → Option 1 provides more information about X
  → Design measurements to maximize I(X; Y)

Figure 22.3: Measurement design: choosing observations to maximise information

22.5 Information-Theoretic Causal Measures

Beyond mutual information, information theory provides measures of directed causal influence between time series. These measures quantify how information flows from one process to another over time, complementing the observation and measurement frameworks discussed above.

22.5.1 Transfer Entropy

Transfer entropy measures directed information flow between time series, answering: “How much does the past of \(X\) help predict the future of \(Y\) beyond what \(Y\)’s own past already provides?”

The mathematical definition is:

\[ TE_{X \to Y} = \sum p(y_{t+1}, y_t^{(k)}, x_t^{(\ell)}) \log \frac{p(y_{t+1} \mid y_t^{(k)}, x_t^{(\ell)})}{p(y_{t+1} \mid y_t^{(k)})} \]

where \(y_t^{(k)}\) denotes the \(k\)-lag history of \(Y\) and \(x_t^{(\ell)}\) the \(\ell\)-lag history of \(X\). Introduced by Schreiber (2000), transfer entropy is:

• Non-parametric: Does not assume linearity or Gaussianity
• Asymmetric: \(TE_{X \to Y} \neq TE_{Y \to X}\) in general, capturing directionality
• Conditional form: To control for confounders \(Z\), use conditional transfer entropy \(TE_{X \to Y \mid Z}\)

For Gaussian processes, transfer entropy relates to Granger causality: \(TE_{X \to Y}\) equals half the Granger \(F\)-statistic. Applications include neural information flow, gene regulatory interactions, and financial time series.

22.5.2 Directed Information

Directed information (Massey 1990) captures the total causal influence of \(X\) on \(Y\) over a time horizon:

\[ I(X^n \to Y^n) = \sum_{t=1}^n I(X^t; Y_t \mid Y^{t-1}) \]

It sums the incremental information that \(X^t\) provides about \(Y_t\) given \(Y\)’s past. Directed information connects to channel capacity and feedback systems, and is relevant for understanding communication in biological networks where information flows bidirectionally with delay.

22.5.3 Causal Entropy

Causal entropy quantifies the uncertainty in counterfactual distributions, the entropy of \(Y\) under interventions on \(X\):

\[ H_{\text{causal}}(Y \mid do(X)) = -\sum_x P(X=x) \sum_y P(Y=y \mid do(X=x)) \log P(Y=y \mid do(X=x)) \]

This measures the residual uncertainty in \(Y\) when we intervene on \(X\), contrasting with associational entropy \(H(Y \mid X)\) which conditions on observation. The difference captures how causal knowledge reduces uncertainty beyond correlation.

22.5.4 Interventional Embedding Entropy

Transfer entropy and related ConDC measures remain predictive: they ask how much the past of \(X\) reduces uncertainty in \(Y\) under the observed dynamics. Interventional Embedding Entropy (IEE) (Shi et al. 2026) targets IntDC instead: local geometry in delay-embedding space is used as a proxy for how perturbations of \(X\) would propagate into \(Y\), without requiring real interventions or an explicit ODE. Where causal entropy above assumes access to interventional distributions \(P(Y \mid do(X))\), IEE estimates an interventional ranking from observational series alone. Treat IEE as a discovery and ranking tool (see Causal Discovery, @chunk-iee-temporal-bridge); confirmed effects still need an identified estimand under a CDM.

22.5.5 Practical Considerations

Estimation from finite data requires care:

• Density estimation: Kernel density, \(k\)-nearest-neighbour, or binning for discrete approximations
• Bias correction: The Kraskov–Stögbauer–Grassberger (KSG) estimator reduces bias in mutual-information-based quantities
• Significance testing: Surrogate data methods (e.g. block permutation) to test against the null of no causal influence

The following conceptual code illustrates transfer entropy computation using a simple histogram-based estimator:

"""
Conceptual transfer entropy TE_{X→Y} computation.
Uses histogram-based probability estimates for illustration.
"""
function transfer_entropy_conceptual(x, y; k=2, ℓ=2)
    n = length(x)
    TE = 0.0
    for t in (max(k, ℓ)+1):(n-1)
        # Embeddings: y_t^(k), x_t^(ℓ)
        y_hist = y[t-k+1:t]
        x_hist = x[t-ℓ+1:t]
        y_next = y[t+1]
        # In practice: estimate p(y_{t+1}|y_t^k,x_t^ℓ) and p(y_{t+1}|y_t^k)
        # via histogram or k-NN; accumulate TE contribution
        # TE += p(y_next,y_hist,x_hist) * log(p(y_next|y_hist,x_hist) / p(y_next|y_hist))
    end
    return TE
end
# For production: use Associations.jl (transfer entropy, CCM, OCE) or TransferEntropy.jl

transfer_entropy_conceptual (generic function with 1 method)

22.6 Stratum context

Observation interactions produce records about an already realised process. The observation model relates \(Y_t\) to \(X_t\); it does not materialise \(X_t\).

22.7 Summary

The observation model is how a realised dynamical state is registered as data. Partial observability forces inference of \(X\) from \(Y\); missingness DAGs (MCAR–MNAR) extend the measurement model to whether a value is recorded. Measurement design and information-theoretic screens (including IEE) prepare the Measurement chapters that follow.

22.8 Further Reading

• State-Space Models: Inferring Structure from Observations: Inferring latent states from observations
• Observational Methods: Learning from Data: Methods for learning from observable data
• Experimental Design: Optimal Measurements: Designing optimal measurements

Gelman, Andrew, Aki Vehtari, Richard McElreath, et al. 2026. Bayesian Workflow. Chapman & Hall. https://avehtari.github.io/Bayesian-Workflow/.

Little, Roderick J. A., and Donald B. Rubin. 2019. Statistical Analysis with Missing Data. 3rd ed. Wiley.

Massey, James L. 1990. “Causality, Feedback and Directed Information.” Proceedings of the 1990 International Symposium on Information Theory and Its Applications (Hawaii, USA), 303–5.

Mohan, Karthika, and Judea Pearl. 2018. Graphical Models for Processing Missing Data. https://arxiv.org/abs/1801.03583v2.

Schreiber, Thomas. 2000. “Measuring Information Transfer.” Physical Review Letters 85 (2): 461–64. https://doi.org/10.1103/PhysRevLett.85.461.

Shi, Jifan, Yang Li, Juan Zhao, et al. 2026. “Deciphering Interventional Dynamical Causality from Non-Intervention Complex Systems.” The Innovation 7 (10): 101358. https://doi.org/10.1016/j.xinn.2026.101358.