On estimating the memory for finitarily Markovian processes

Finitarily Markovian processes are those processes {X_n}_{n = - ∞}^∞ for which there is a finite K (K = K ({X_n}_{n = - ∞}⁰)) such that the conditional distribution of X₁ given the entire past is equal to the conditional distribution of X₁ given only {X_n}_{n = 1 - K}⁰. The least such value of K is called the memory length. We give a rather complete analysis of the problems of universally estimating the least such value of K, both in the backward sense that we have just described and in the forward sense, where one observes successive values of {X_n} for n ≥ 0 and asks for the least value K such that the conditional distribution of X_{n + 1} given {X_i}_{i = n - K + 1}ⁿ is the same as the conditional distribution of X_{n + 1} given {X_i}_{i = - ∞}ⁿ. We allow for finite or countably infinite alphabet size.