Transfer entropy

Transfer entropy is a non-parametric statistic measuring the amount of directed (time-asymmetric) transfer of information between two random processes.^{[1] [2] [3]} Transfer entropy from a process X to another process Y is the amount of uncertainty reduced in future values of Y by knowing the past values of X given past values of Y. More specifically, if ${\displaystyle X_{t}}${\displaystyle X_{t}} and ${\displaystyle Y_{t}}${\displaystyle Y_{t}} for ${\displaystyle t\in \mathbb {N} }${\displaystyle t\in \mathbb {N} } denote two random processes and the amount of information is measured using Shannon's entropy, the transfer entropy can be written as:

${\displaystyle T_{X\rightarrow Y}=H\left(Y_{t}\mid Y_{t-1:t-L}\right)-H\left(Y_{t}\mid Y_{t-1:t-L},X_{t-1:t-L}\right),}${\displaystyle T_{X\rightarrow Y}=H\left(Y_{t}\mid Y_{t-1:t-L}\right)-H\left(Y_{t}\mid Y_{t-1:t-L},X_{t-1:t-L}\right),}

where H(X) is Shannon's entropy of X. The above definition of transfer entropy has been extended by other types of entropy measures such as Rényi entropy.^{[3] [4]}

Transfer entropy is conditional mutual information,^{[5] [6]} with the history of the influenced variable ${\displaystyle Y_{t-1:t-L}}${\displaystyle Y_{t-1:t-L}} in the condition:

${\displaystyle T_{X\rightarrow Y}=I(Y_{t};X_{t-1:t-L}\mid Y_{t-1:t-L}).}${\displaystyle T_{X\rightarrow Y}=I(Y_{t};X_{t-1:t-L}\mid Y_{t-1:t-L}).}

Transfer entropy reduces to Granger causality for vector auto-regressive processes.^[7] Hence, it is advantageous when the model assumption of Granger causality doesn't hold, for example, analysis of non-linear signals.^{[8] [9]} However, it usually requires more samples for accurate estimation.^[10] The probabilities in the entropy formula can be estimated using different approaches (binning, nearest neighbors) or, in order to reduce complexity, using a non-uniform embedding.^[11] While it was originally defined for bivariate analysis, transfer entropy has been extended to multivariate forms, either conditioning on other potential source variables^[12] or considering transfer from a collection of sources,^[13] although these forms require more samples again.

Transfer entropy has been used for estimation of functional connectivity of neurons,^{[13] [14] [15]} social influence in social networks ^[8] and statistical causality between armed conflict events.^[16] Transfer entropy is a finite version of the directed information which was defined in 1990 by James Massey ^[17] as ${\displaystyle I(X^{n}\to Y^{n})=\sum _{i=1}^{n}I(X^{i};Y_{i}|Y^{i-1})}${\displaystyle I(X^{n}\to Y^{n})=\sum _{i=1}^{n}I(X^{i};Y_{i}|Y^{i-1})}, where ${\displaystyle X^{n}}${\displaystyle X^{n}} denotes the vector ${\displaystyle X_{1},X_{2},...,X_{n}}${\displaystyle X_{1},X_{2},...,X_{n}} and ${\displaystyle Y^{n}}${\displaystyle Y^{n}} denotes ${\displaystyle Y_{1},Y_{2},...,Y_{n}}${\displaystyle Y_{1},Y_{2},...,Y_{n}}. The directed information places an important role in characterizing the fundamental limits (channel capacity) of communication channels with or without feedback^{[18] [19]} and gambling with causal side information.^[20]

• Directed information
• Mutual information
• Conditional mutual information
• Causality
• Causality (physics)
• Structural equation modeling
• Rubin causal model

• "Transfer Entropy Toolbox". Google Code., a toolbox, developed in C++ and MATLAB, for computation of transfer entropy between spike trains.
• "Java Information Dynamics Toolkit (JIDT)". GitHub. 2019年01月16日., a toolbox, developed in Java and usable in MATLAB, GNU Octave and Python, for computation of transfer entropy and related information-theoretic measures in both discrete and continuous-valued data.
• "Multivariate Transfer Entropy (MuTE) toolbox". GitHub. 2019年01月09日., a toolbox, developed in MATLAB, for computation of transfer entropy with different estimators.

• Causality
• Nonlinear time series analysis
• Nonparametric statistics
• Entropy and information

• CS1 errors: missing periodical
• Articles with short description
• Short description is different from Wikidata