Uniform Embedding

NB: for the formalism needed to understand what follows please go to MuTE page and read SOME FORMALISM section

The large majority of approaches applied so far to estimate TE implicitly follow uniform conditioned embedding schemes where the components to be included in the embedding vectors are selected arbitrarily or separately for each time series. Note that the TE can be seen as a difference of two conditional entropies (CE), or equivalently as a sum of four Shannon entropies:

(1)   \begin{equation*} \begin{aligned}   TE_{X \rightarrow Y|\mathbf{Z}} &= H(Y_n | Y_n^-, \textbf{Z}_n^-) - H(Y_n | Y_n^-, X_n^-, \textbf{Z}_n^-) \\   &= H(Y_n, Y_n^-, \textbf{Z}_n^-)-H(Y_n^-, \textbf{Z}_n^-)  \\   &\qquad {} - H(Y_n,Y_n^-, X_n^-, \textbf{Z}_n^-)+H(Y_n^-, X_n^-, \textbf{Z}_n^-)  \end{aligned}  \end{equation*}

Taking into account, for instance, the vector Y_n^- approximated using the embedding vector V_n^Y=[Y_{n-m}\,Y_{n-2m}\ldots Y_{n-dm}], where d and m are the embedding dimension and embedding delay, the same for X_n^- and \mathbf{Z}_n^- approximated by V_n^X and V_n^{\mathbf{Z}}, it is possible to distinguish between a first phase during which the past states are collected and a second phase during which the estimate of the entropy, and consequently of the CE, is evaluated by means of the chosen estimator, according to the following pseudo code:

  1. build the vectors V = [V_n^Y, V_n^X, V_n^{\textbf{Z}}];
  2. use V and Y_n to evaluate the last two entropies of (1) and, consequently, the lowest CE term (CE2);
  3. use V \backslash \textbf{Z} to evaluate the first two entropies of (1) and, consequently, the highest CE term (CE1);
  4. compute TE as equal to the difference CE1 – CE2.

The obvious arbitrariness and redundancy introduced by this strategy are likely to cause problems such as overfitting and detection of false influences, Vlachos (2010). Moreover one should assess which TE values are significant. The significance tests associated to TE estimation based on UE are different for model-based and model free estimators.


  1. Vlachos, Ioannis, Kugiumtzis, Dimitris: Nonuniform state-space reconstruction and coupling detection. In: Phys Rev E, 82 (1), pp. 016207, 2010.

Looking for related topics? Search Google