Empirical dynamic modeling

Simplex projection^{[20] [24] [25] [26]} is a nearest neighbor projection. It locates the ${\displaystyle k}${\displaystyle k} nearest neighbors to the location in the state-space from which a prediction is desired. To minimize the number of free parameters ${\displaystyle k}${\displaystyle k} is typically set to ${\displaystyle E+1}${\displaystyle E+1} defining an ${\displaystyle E+1}${\displaystyle E+1} dimensional simplex in the state-space. The prediction is computed as the average of the weighted phase-space simplex projected ${\displaystyle Tp}${\displaystyle Tp} points ahead. Each neighbor is weighted proportional to their distance to the projection origin vector in the state-space.

1. Find ${\displaystyle k}${\displaystyle k} nearest neighbor: ${\displaystyle N_{k}\gets {\text{NN}}(y,X,k)}${\displaystyle N_{k}\gets {\text{NN}}(y,X,k)}
2. Define the distance scale: ${\displaystyle d\gets \|X_{N_{1}}^{E}-y\|}${\displaystyle d\gets \|X_{N_{1}}^{E}-y\|}
3. Compute weights: For{${\displaystyle i=1,\dots ,k}${\displaystyle i=1,\dots ,k}} : ${\displaystyle w_{i}\gets \exp(-\|X_{N_{i}}^{E}-y\|/d)}${\displaystyle w_{i}\gets \exp(-\|X_{N_{i}}^{E}-y\|/d)}
4. Average of state-space simplex: ${\displaystyle {\hat {y}}\gets \sum _{i=1}^{k}\left(w_{i}X_{N_{i}+T_{p}}\right)/\sum _{i=1}^{k}w_{i}}${\displaystyle {\hat {y}}\gets \sum _{i=1}^{k}\left(w_{i}X_{N_{i}+T_{p}}\right)/\sum _{i=1}^{k}w_{i}}

S-Map^[21] extends the state-space prediction in Simplex from an average of the ${\displaystyle E+1}${\displaystyle E+1} nearest neighbors to a linear regression fit to all neighbors, but localised with an exponential decay kernel. The exponential localisation function is ${\displaystyle F(\theta )={\text{exp}}(-\theta d/D)}${\displaystyle F(\theta )={\text{exp}}(-\theta d/D)}, where ${\displaystyle d}${\displaystyle d} is the neighbor distance and ${\displaystyle D}${\displaystyle D} the mean distance. In this way, depending on the value of ${\displaystyle \theta }${\displaystyle \theta }, neighbors close to the prediction origin point have a higher weight than those further from it, such that a local linear approximation to the nonlinear system is reasonable. This localisation ability allows one to identify an optimal local scale, in-effect quantifying the degree of state dependence, and hence nonlinearity of the system.^[27]

Another feature of S-Map is that for a properly fit model, the regression coefficients between variables have been shown to approximate the gradient (directional derivative) of variables along the manifold.^[28] These Jacobians represent the time-varying interaction strengths between system variables.

1. Find ${\displaystyle k}${\displaystyle k} nearest neighbor: ${\displaystyle N\gets {\text{NN}}(y,X,k)}${\displaystyle N\gets {\text{NN}}(y,X,k)}
2. Sum of distances: ${\displaystyle D\gets {\frac {1}{k}}\sum _{i=1}^{k}\|X_{N_{i}}^{E}-y\|}${\displaystyle D\gets {\frac {1}{k}}\sum _{i=1}^{k}\|X_{N_{i}}^{E}-y\|}
3. Compute weights: For{${\displaystyle i=1,\dots ,k}${\displaystyle i=1,\dots ,k}} : ${\displaystyle w_{i}\gets \exp(-\theta \|X_{N_{i}}^{E}-y\|/D)}${\displaystyle w_{i}\gets \exp(-\theta \|X_{N_{i}}^{E}-y\|/D)}
4. Reweighting matrix: ${\displaystyle W\gets {\text{diag}}(w_{i})}${\displaystyle W\gets {\text{diag}}(w_{i})}
5. Design matrix: ${\displaystyle A\gets {\begin{bmatrix}1&X_{N_{1}}&X_{N_{1}-1}&\dots &X_{N_{1}-E+1}\1円&X_{N_{2}}&X_{N_{2}-1}&\dots &X_{N_{2}-E+1}\\\vdots &\vdots &\vdots &\ddots &\vdots \1円&X_{N_{k}}&X_{N_{k}-1}&\dots &X_{N_{k}-E+1}\end{bmatrix}}}${\displaystyle A\gets {\begin{bmatrix}1&X_{N_{1}}&X_{N_{1}-1}&\dots &X_{N_{1}-E+1}\1円&X_{N_{2}}&X_{N_{2}-1}&\dots &X_{N_{2}-E+1}\\\vdots &\vdots &\vdots &\ddots &\vdots \1円&X_{N_{k}}&X_{N_{k}-1}&\dots &X_{N_{k}-E+1}\end{bmatrix}}}
6. Weighted design matrix: ${\displaystyle A\gets WA}${\displaystyle A\gets WA}
7. Response vector at ${\displaystyle Tp}${\displaystyle Tp}: ${\displaystyle b\gets {\begin{bmatrix}X_{N_{1}+T_{p}}\\X_{N_{2}+T_{p}}\\\vdots \\X_{N_{k}+T_{p}}\end{bmatrix}}}${\displaystyle b\gets {\begin{bmatrix}X_{N_{1}+T_{p}}\\X_{N_{2}+T_{p}}\\\vdots \\X_{N_{k}+T_{p}}\end{bmatrix}}}
8. Weighted response vector: ${\displaystyle b\gets Wb}${\displaystyle b\gets Wb}
9. Least squares solution (SVD): ${\displaystyle {\hat {c}}\gets {\text{argmin}}_{c}\|Ac-b\|_{2}^{2}}${\displaystyle {\hat {c}}\gets {\text{argmin}}_{c}\|Ac-b\|_{2}^{2}}
10. Local linear model ${\displaystyle {\hat {c}}}${\displaystyle {\hat {c}}} is prediction: ${\displaystyle {\hat {y}}\gets {\hat {c}}_{0}+\sum _{i=1}^{E}{\hat {c}}_{i}y_{i}}${\displaystyle {\hat {y}}\gets {\hat {c}}_{0}+\sum _{i=1}^{E}{\hat {c}}_{i}y_{i}}

Multivariate Embedding^{[1] [12] [29]} recognizes that time-delay embeddings are not the only valid state-space construction. In Simplex and S-Map one can generate a state-space from observational vectors, or time-delay embeddings of a single observational time series, or both.

Convergent cross mapping (CCM)^[22] leverages a corollary to the Generalized Takens Theorem^[12] that it should be possible to cross predict or cross map between variables observed from the same system. Suppose that in some dynamical system involving variables ${\displaystyle X}${\displaystyle X} and ${\displaystyle Y}${\displaystyle Y}, ${\displaystyle X}${\displaystyle X} causes ${\displaystyle Y}${\displaystyle Y}. Since ${\displaystyle X}${\displaystyle X} and ${\displaystyle Y}${\displaystyle Y} belong to the same dynamical system, their reconstructions (via embeddings) ${\displaystyle M_{x}}${\displaystyle M_{x}}, and ${\displaystyle M_{y}}${\displaystyle M_{y}}, also map to the same system.

The causal variable ${\displaystyle X}${\displaystyle X} leaves a signature on the affected variable ${\displaystyle Y}${\displaystyle Y}, and consequently, the reconstructed states based on ${\displaystyle Y}${\displaystyle Y} can be used to cross predict values of ${\displaystyle X}${\displaystyle X}. CCM leverages this property to infer causality by predicting ${\displaystyle X}${\displaystyle X} using the ${\displaystyle M_{y}}${\displaystyle M_{y}} library of points (or vice versa for the other direction of causality), while assessing improvements in cross map predictability as larger and larger random samplings of ${\displaystyle M_{y}}${\displaystyle M_{y}} are used. If the prediction skill of ${\displaystyle X}${\displaystyle X} increases and saturates as the entire ${\displaystyle M_{y}}${\displaystyle M_{y}} is used, this provides evidence that ${\displaystyle X}${\displaystyle X} is casually influencing ${\displaystyle Y}${\displaystyle Y}.

Multiview Embedding^[23] is a Dimensionality reduction technique where a large number of state-space time series vectors are combitorially assessed towards maximal model predictability.