23  From Dynamical to Observable: Measurement and Actualisation

Status: Draft

v0.5

23.1 Introduction

This chapter bridges Dynamical and Observable strata, showing how dynamic processes (inner strata) become actualised as observations (outer stratum) through measurement processes. This prepares for Part III where we work with observable data. This is the transition from Doing/Imagining in the Dynamical stratum to Seeing in the Observable stratum.

23.2 How Dynamic Processes Become Observable

23.2.1 The Observation Model

Dynamic processes exist in the Dynamical stratum (inner stratum), and become observable through the observation model:

\[ Y_t = h(X_t, C, U^y_t) \]

where:

• \(X_t\) is the latent state (exists in Dynamical stratum)
• \(Y_t\) is the observation (exists in Observable stratum)
• \(h\) is the observation function (encodes how inner strata become outer strata)
• \(U^y_t\) is measurement noise

23.2.2 The Actualisation Process

The observation model represents the process of actualisation:

• Dynamical stratum (inner): Dynamic processes, latent states \(X_t\)
• Observable stratum (outer): Actualised observations \(Y_t\)
• Observation function \(h\): How inner strata become outer strata

23.2.3 Implementation: Observation Models

We can demonstrate different types of observation models (Figure fig-chunk-observation-models-viz):

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(dirname(Base.active_project()), "scripts", "book_bootstrap.jl"))

@auto_using Random Distributions OrdinaryDiffEq CairoMakie

Random.seed!(42)

# Example: Latent state X_t evolving over time
function latent_dynamics!(du, u, p, t)
    """Latent state dynamics: exponential decay with rate 0.1."""
    X = u[1]
    du[1] = -0.1 * X  # Decay process
end

u0 = [1.0]
tspan = (0.0, 20.0)
prob = ODEProblem(latent_dynamics!, u0, tspan)
sol = solve(prob, Tsit5())

X_t = [u[1] for u in sol.u]  # Latent state

# Type 1: Direct measurement Y_t = X_t + noise
σ_direct = 0.1
Y_direct = X_t .+ rand(Normal(0, σ_direct), length(X_t))

# Type 2: Indirect measurement Y_t = h(X_t) + noise
# Example: h(X) = X² (nonlinear observation)
Y_indirect = X_t.^2 .+ rand(Normal(0, 0.05), length(X_t))

# Type 3: Aggregate measurement (if we had multiple states)
# Y_t = Σ X_i + noise (simplified here)
Y_aggregate = X_t .+ rand(Normal(0, 0.1), length(X_t))  # Simplified

println("Observation models:")
println("  Direct: Y_t = X_t + noise (σ = ", σ_direct, ")")
println("  Indirect: Y_t = X_t² + noise (nonlinear)")
println("  Inference: Must recover X_t from Y_t (addressed in state-space models)")

Observation models:
  Direct: Y_t = X_t + noise (σ = 0.1)
  Indirect: Y_t = X_t² + noise (nonlinear)
  Inference: Must recover X_t from Y_t (addressed in state-space models)

Figure 23.1: Different observation models: how latent states become observable

23.3 Partial Observability

23.3.1 The Problem

In most real systems, we cannot directly observe the latent state \(X_t\):

• Partial observability: We only observe \(Y_t\), not \(X_t\)
• Measurement noise: Observations are noisy
• Missing data: Some observations may be missing; the missingness mechanism is itself a causal story (next subsection)

23.3.2 Missingness mechanisms as DAGs

Classical labels MCAR / MAR / MNAR are claims about what causes a recording to fail (Little and Rubin 2019). A DAG makes those claims explicit: if \(C^*\) is the recorded value and \(F\) is a failure indicator (\(F=1\) means \(C^*\) is missing), arrows into \(F\) decide whether imputation from covariates is justified.

Rename the usual rainfall–richness teaching graph to a forest understorey survey: soil moisture \(W\), understorey cover \(C\) (latent when missing), survey failure \(F\), and (in one scenario) management intensity \(Z\).

Scenario	Causal claim	Practical consequence
(a) MCAR	\(F\) unrelated to \(W,C\)	Complete-case analysis unbiased for \(W\to C\)
(b) MAR	\(W \to F\)	Impute \(C\) using \(W\)
(c) Confounded	\(Z\to C\), \(Z\to F\)	Must include \(Z\) in the imputation model
(d) MNAR	\(C\to F\)	Missingness depends on the missing value; needs stronger assumptions

include(joinpath(dirname(Base.active_project()), "scripts", "book_bootstrap.jl"))
@auto_using DAGMakie CairoMakie Graphs

# Nodes: 1=W, 2=C, 3=C*, 4=F [, 5=Z]
g_mcar = SimpleDiGraph(4)
add_edge!(g_mcar, 1, 2); add_edge!(g_mcar, 2, 3); add_edge!(g_mcar, 4, 3)

g_mar = SimpleDiGraph(4)
add_edge!(g_mar, 1, 2); add_edge!(g_mar, 2, 3); add_edge!(g_mar, 4, 3); add_edge!(g_mar, 1, 4)

g_conf = SimpleDiGraph(5)
add_edge!(g_conf, 1, 2); add_edge!(g_conf, 2, 3); add_edge!(g_conf, 4, 3)
add_edge!(g_conf, 5, 2); add_edge!(g_conf, 5, 4)

g_mnar = SimpleDiGraph(4)
add_edge!(g_mnar, 1, 2); add_edge!(g_mnar, 2, 3); add_edge!(g_mnar, 4, 3); add_edge!(g_mnar, 2, 4)

layout4 = Point2f[Point2f(0.0, 1.0), Point2f(1.2, 1.0), Point2f(1.2, 0.0), Point2f(0.0, 0.0)]
layout5 = Point2f[Point2f(0.0, 1.0), Point2f(1.4, 1.0), Point2f(1.4, 0.0), Point2f(0.0, 0.0), Point2f(0.7, 0.5)]

5-element Vector{Point{2, Float32}}:
 [0.0, 1.0]
 [1.4, 1.0]
 [1.4, 0.0]
 [0.0, 0.0]
 [0.7, 0.5]

Four missingness stories for a forest cover survey. \(W\): soil moisture; \(C\): understorey cover (latent when missing); \(C^*\): recorded cover; \(F\): survey failure; \(Z\): management intensity. Dashed node outline marks the latent cover.

The Observable stratum inherits these graphs: \(h\) is not only “how cover becomes a number”, but also “when the number is written down”. Imputation models that ignore arrows into \(F\) are misspecified for (c) and (d). Under MAR (\(W \to F\)), a Bayesian regression of cover on moisture fitted on observed rows, then predicting missing rows, is a minimal sound imputer:

using Turing: Turing, @model, sample, NUTS
using Distributions
using Statistics
using Random

rng = Random.Xoshiro(11)
n = 200
W = randn(rng, n)
C_true = 0.2 .+ 0.8 .* W .+ 0.3 .* randn(rng, n)
miss = W .> 0.5                          # MAR: failure depends on moisture
W_obs, C_obs = W[.!miss], C_true[.!miss]

Turing.@model function cover_on_moisture(W, C)
    β0 ~ Normal(0, 2)
    βw ~ Normal(0, 2)
    σ ~ truncated(Normal(1, 1); lower = 0.05)
    for i in eachindex(C)
        C[i] ~ Normal(β0 + βw * W[i], σ)
    end
end

t_mar = @elapsed begin
    global ch_mar = sample(cover_on_moisture(W_obs, C_obs), NUTS(0.8), 400; progress = false)
end
β0̂, βŵ = mean(ch_mar[:β0]), mean(ch_mar[:βw])
C_imp = β0̂ .+ βŵ .* W
mae_miss = mean(abs.(C_imp[miss] .- C_true[miss]))
println("MAR imputer: βw≈", round(βŵ; digits = 3),
    "  MAE on missing = ", round(mae_miss; digits = 3),
    "  (", round(t_mar; digits = 3), " s)")

MAR imputer: βw≈0.801  MAE on missing = 0.212  (4.583 s)

MAR imputation for understorey cover: true \(C\) versus posterior-mean predictions. Orange points are units with survey failure (\(W>0.5\)); blue are observed.

MNAR (\(C\to F\)) needs a joint model for \((C,F)\); the MAR imputer above is then misspecified. Full Bayesian imputation belongs with estimation tooling; the structural point remains that missingness is part of the measurement DAG.

23.3.3 Inference Problem

Given observations \(Y_{1:T}\), we must infer the latent state \(X_{1:T}\):

\[ P(X_{1:T} \mid Y_{1:T}) \]

This is the inference problem addressed in State-Space Models: Inferring Structure from Observations.

23.3.4 Implementation: Partial Observability

We can demonstrate the inference problem when we only observe \(Y_t\), not \(X_t\) (Figure fig-chunk-partial-observability-viz):

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(project_root, "scripts", "ensure_packages.jl"))

@auto_using Random Distributions CairoMakie

Random.seed!(42)

# Example: We observe Y_t but not X_t
# Latent state: X_t (unknown)
# Observation: Y_t = X_t + noise

# Simulate true latent state
T = 50
X_true = zeros(T)
X_true[1] = 1.0
for t in 2:T
    X_true[t] = 0.9 * X_true[t-1] + rand(Normal(0, 0.1))  # AR(1) process
end

# Observations with noise
σ_obs = 0.2
Y_obs = X_true .+ rand(Normal(0, σ_obs), T)

# Inference problem: Given Y_obs, infer X_true
# Simple approach: Use observations directly (ignoring dynamics)
X_inferred_naive = Y_obs

# Better approach: Use dynamics + observations (Kalman filter - simplified)
# Here we show a simple smoothing approach
X_inferred_smooth = zeros(T)
α = 0.5  # Smoothing parameter
for t in 1:T
    if t == 1
        X_inferred_smooth[t] = Y_obs[t]
    else
        # Combine observation with prediction from dynamics
        X_pred = 0.9 * X_inferred_smooth[t-1]
        X_inferred_smooth[t] = α * Y_obs[t] + (1 - α) * X_pred
    end
end

println("Partial observability:")
println("  True latent state: X_t (unknown)")
println("  Observations: Y_t = X_t + noise")
println("  Inference problem: Recover X_t from Y_t")
println("  Naive approach: Use Y_t directly (ignores dynamics)")
println("  Better approach: Combine observations with dynamics (state-space inference)")

Partial observability:
  True latent state: X_t (unknown)
  Observations: Y_t = X_t + noise
  Inference problem: Recover X_t from Y_t
  Naive approach: Use Y_t directly (ignores dynamics)
  Better approach: Combine observations with dynamics (state-space inference)

Figure 23.2: Partial observability: inferring latent states from noisy observations

23.4 Measurement Processes

23.4.1 Types of Measurement

• Direct measurement: \(Y_t = X_t + \text{noise}\) (observe state directly with noise)
• Indirect measurement: \(Y_t = h(X_t) + \text{noise}\) (observe function of state)
• Aggregate measurement: \(Y_t = \sum_i X^i_t + \text{noise}\) (observe sum of states)
• Delayed measurement: \(Y_t = X_{t-k} + \text{noise}\) (observe past state)

23.4.2 Measurement Design

Question: What should we measure to maximise information about latent states?

Answer: Design measurements that maximise mutual information \(I(X_t; Y_t)\).

23.4.3 Implementation: Measurement Design

We can demonstrate how to choose measurements that maximise information (Figure fig-chunk-measurement-design-viz):

# Find project root and include ensure_packages.jl
project_root = let
    current = pwd()
    while !isfile(joinpath(current, "Project.toml")) && !isfile(joinpath(current, "_quarto.yml"))
        parent = dirname(current)
        parent == current && break
        current = parent
    end
    current
end
include(joinpath(project_root, "scripts", "ensure_packages.jl"))

@auto_using Random Distributions CairoMakie

Random.seed!(42)

# Example: Two possible measurement functions
# Option 1: Y₁ = X + noise (direct, high information)
# Option 2: Y₂ = sign(X) + noise (binary, lower information)

# Simulate latent state
n = 1000
X = rand(Normal(0, 1), n)

# Measurement option 1: Direct (high information)
σ1 = 0.1
Y1 = X .+ rand(Normal(0, σ1), n)

# Measurement option 2: Binary (lower information)
Y2 = sign.(X) .+ rand(Normal(0, 0.1), n)

# Approximate mutual information I(X; Y) ≈ H(X) - H(X|Y)
# Using correlation as proxy (higher correlation → higher mutual information)
cor1 = abs(cor(X, Y1))
cor2 = abs(cor(X, Y2))

println("Measurement design comparison:")
println("  Option 1 (direct): Correlation = ", round(cor1, digits=3), " (high information)")
println("  Option 2 (binary): Correlation = ", round(cor2, digits=3), " (lower information)")
println("  → Option 1 provides more information about X")
println("  → Design measurements to maximize I(X; Y)")

Measurement design comparison:
  Option 1 (direct): Correlation = 0.995 (high information)
  Option 2 (binary): Correlation = 0.793 (lower information)
  → Option 1 provides more information about X
  → Design measurements to maximize I(X; Y)

Figure 23.3: Measurement design: choosing observations to maximise information

23.5 Information-Theoretic Causal Measures

Beyond mutual information, information theory provides powerful tools for quantifying directed causal influence between time series. These measures capture how information flows from one process to another over time, complementing the observation and measurement frameworks discussed above.

23.5.1 Transfer Entropy

Transfer entropy measures directed information flow between time series, answering: “How much does the past of \(X\) help predict the future of \(Y\) beyond what \(Y\)’s own past already provides?”

The mathematical definition is:

\[ TE_{X \to Y} = \sum p(y_{t+1}, y_t^{(k)}, x_t^{(\ell)}) \log \frac{p(y_{t+1} \mid y_t^{(k)}, x_t^{(\ell)})}{p(y_{t+1} \mid y_t^{(k)})} \]

where \(y_t^{(k)}\) denotes the \(k\)-lag history of \(Y\) and \(x_t^{(\ell)}\) the \(\ell\)-lag history of \(X\). Introduced by Schreiber (2000), transfer entropy is:

• Non-parametric: Does not assume linearity or Gaussianity
• Asymmetric: \(TE_{X \to Y} \neq TE_{Y \to X}\) in general, capturing directionality
• Conditional form: To control for confounders \(Z\), use conditional transfer entropy \(TE_{X \to Y \mid Z}\)

For Gaussian processes, transfer entropy relates to Granger causality: \(TE_{X \to Y}\) equals half the Granger \(F\)-statistic. Applications include neural information flow, gene regulatory interactions, and financial time series.

23.5.2 Directed Information

Directed information (Massey 1990) captures the total causal influence of \(X\) on \(Y\) over a time horizon:

\[ I(X^n \to Y^n) = \sum_{t=1}^n I(X^t; Y_t \mid Y^{t-1}) \]

It sums the incremental information that \(X^t\) provides about \(Y_t\) given \(Y\)’s past. Directed information connects to channel capacity and feedback systems, and is relevant for understanding communication in biological networks where information flows bidirectionally with delay.

23.5.3 Causal Entropy

Causal entropy quantifies the uncertainty in counterfactual distributions, the entropy of \(Y\) under interventions on \(X\):

\[ H_{\text{causal}}(Y \mid do(X)) = -\sum_x P(X=x) \sum_y P(Y=y \mid do(X=x)) \log P(Y=y \mid do(X=x)) \]

This measures the residual uncertainty in \(Y\) when we intervene on \(X\), contrasting with associational entropy \(H(Y \mid X)\) which conditions on observation. The difference captures how causal knowledge reduces uncertainty beyond correlation.

23.5.4 Interventional Embedding Entropy

Transfer entropy and related ConDC measures remain predictive: they ask how much the past of \(X\) reduces uncertainty in \(Y\) under the observed dynamics. Interventional Embedding Entropy (IEE) (Shi et al. 2026) targets IntDC instead: local geometry in delay-embedding space is used as a proxy for how perturbations of \(X\) would propagate into \(Y\), without requiring real interventions or an explicit ODE. Where causal entropy above assumes access to interventional distributions \(P(Y \mid do(X))\), IEE estimates an interventional ranking from observational series alone. Treat IEE as a discovery and ranking tool (see Causal Discovery, @chunk-iee-temporal-bridge); confirmed effects still need an identified estimand under a CDM.

23.5.5 Practical Considerations

Estimation from finite data requires care:

• Density estimation: Kernel density, \(k\)-nearest-neighbour, or binning for discrete approximations
• Bias correction: The Kraskov–Stögbauer–Grassberger (KSG) estimator reduces bias in mutual-information-based quantities
• Significance testing: Surrogate data methods (e.g. block permutation) to test against the null of no causal influence

The following conceptual code illustrates transfer entropy computation using a simple histogram-based estimator:

"""
Conceptual transfer entropy TE_{X→Y} computation.
Uses histogram-based probability estimates for illustration.
"""
function transfer_entropy_conceptual(x, y; k=2, ℓ=2)
    n = length(x)
    TE = 0.0
    for t in (max(k, ℓ)+1):(n-1)
        # Embeddings: y_t^(k), x_t^(ℓ)
        y_hist = y[t-k+1:t]
        x_hist = x[t-ℓ+1:t]
        y_next = y[t+1]
        # In practice: estimate p(y_{t+1}|y_t^k,x_t^ℓ) and p(y_{t+1}|y_t^k)
        # via histogram or k-NN; accumulate TE contribution
        # TE += p(y_next,y_hist,x_hist) * log(p(y_next|y_hist,x_hist) / p(y_next|y_hist))
    end
    return TE
end
# For production: use Associations.jl (transfer entropy, CCM, OCE) or TransferEntropy.jl

Main.Notebook.transfer_entropy_conceptual

23.6 Stratum context

This chapter addresses the transition from Dynamical to Observable: how dynamic processes become actualised as observations. This prepares for Part III where we work with observable data to learn about causal mechanisms. The observation model \(Y_t = h(X_t, C, U^y_t)\) shows how inner strata (Dynamical) become outer strata (Observable).

23.7 Summary

The observation model \(Y_t = h(X_t, C, U^y_t)\) is how dynamical state becomes data. Partial observability forces inference of \(X\) from \(Y\); missingness DAGs (MCAR–MNAR) extend \(h\) to whether a value is recorded. Measurement design and information-theoretic screens (including IEE) prepare the Observable chapters that follow.

23.8 Further Reading

• State-Space Models: Inferring Structure from Observations: Inferring latent states from observations
• Observational Methods: Learning from Data: Methods for learning from observable data
• Experimental Design: Optimal Measurements: Designing optimal measurements

Little, Roderick J. A., and Donald B. Rubin. 2019. Statistical Analysis with Missing Data. 3rd ed. Wiley.

Massey, James L. 1990. “Causality, Feedback and Directed Information.” Proceedings of the 1990 International Symposium on Information Theory and Its Applications (Hawaii, USA), 303–5.

Schreiber, Thomas. 2000. “Measuring Information Transfer.” Physical Review Letters 85 (2): 461–64. https://doi.org/10.1103/PhysRevLett.85.461.

Shi, Jifan, Yang Li, Juan Zhao, et al. 2026. “Deciphering Interventional Dynamical Causality from Non-Intervention Complex Systems.” The Innovation 7 (10): 101358. https://doi.org/10.1016/j.xinn.2026.101358.