PROBABILITY AND STATISTICS Vol. I Ergodic Properties of Statioary, Markov, ad Regeerative Processes Karl Grill ERGODIC PROPERTIES OF STATIONARY, MARKOV, AND REGENERATIVE PROCESSES Karl Grill Istitut für Statistik ud Wahrscheilichkeitstheorie, TU Wie, Austria Keywords: Ergodic theorems, statioary processes, Markov processes, regeerative processes, semi Markov processes Cotets. Itroductio. Ergodic theory for statioary processes.. The Mea Square Ergodic Theorem.. The Strog Ergodic Theorem 3. Ergodic properties of Markov processes 3.. Irreducible Markov Chais 4. Regeerative processes 4.. Defiitio 4.. Examples of Regeerative Processes. 4.3. Ergodic Theorems for Regeerative Processes 5. Applicatios of ergodic theorems 5.. Statistical Iferece for Markov Chais 5.. The Rage of a Radom Walk 5.3. Etropy Glossary Bibliography Biographical Sketch Summary Ergodic theorems give coditios for the covergece of time averages of stochastic processes. Probably the most atural processes to cosider i this respect are statioary processes, ad so versios of the ergodic theorem are established for both weak ad strict sese statioary processes. Aother class of stochastic processes for which ergodic theorems are easily obtaied is give by the Markov processes. I additio, regeerative processes, which exted the defiitio of Markov processes, are defied, ad the ergodic theorem is exteded to these processes, too. Fially some applicatios of the ergodic theorem are discussed.. Itroductio Ergodic theorems i geeral are about questios of the covergece of time averages of (ot ecessarily stochastic) processes. I the case of stochastic processes, there are two parts to a questio like this: first, there is the questio of existece: there is o eed to thik ay further if the time averages do ot coverge i the first place. The, there is the questio of the idetificatio of the limit; i the ideal case, the limit would be the Ecyclopedia of Life Support Systems (EOLSS)
PROBABILITY AND STATISTICS Vol. I Ergodic Properties of Statioary, Markov, ad Regeerative Processes Karl Grill mathematical expectatio of the associated radom variable (if oe ca give a clear iterpretatio to this, of course), but thigs do ot always tur out that simple. Ayway, the idea of time averages covergig to expectatios is oe of the corerstoes of statistical mechaics, ad, maybe i a somewhat lesser extet, of mathematical statistics. The archetypical example is the classical law of large umbers: for a sequece ξ ( ) of idepedet, idetically distributed radom variables with expectatio μ, oe simply has ξ ( k) μ () k = with probability oe as. Actually, eve the coverse is true i the followig sese: if the limit above exists, the the variables ξ ( ) do have a fiite expectatio, ad it is equal to μ. Give that oe hopes that the time averages will coverge to the expectatio of the oedimesioal margial distributio, it is obvious that the most importat case is the oe where this hope has ay chace to be fulfilled, that is, if the oedimesioal margials are all the same; so, the first case that will be studied is that of a statioary process. Actually, the statemet about the objective of ergodic theory that was made above is, to say the least, icomplete. Moder ergodic theory is far more geeral, ad i particular, it is ot restricted to questios from the theory of stochastic processes or from probability theory. Rather it ca be described as the theory of the iterates of a positive cotractio o some abstract fuctio space, that is, of a operator T that satisfies. If f 0, the Tf 0, ad. Tf f. I the case of a statioary process (i discrete time), the role of the operator T is, of course take by the shift operator θ (this operator maps ξ () t to ξ ( t + ) ad ca be exteded to the set of all radom variables that ca be defied i terms of ξ. For a more formal defiitio, see Statioary Processes. But statioary processes are ot the oly oes that come alog with a atural cotractio; the trasitio operators of a Markov process exhibit the same property. Thus, Markov processes (more precisely, Markov chais) are aother cadidate for studies related to ergodic theory. Fially, a slight extesio of the otio of a Markov process, the socalled regeerative processes, which have some importace i applied fields like queuig theory ad decisio theory, will be itroduced, ad their ergodic properties will be studied.. Ergodic Theory for Statioary Processes Ecyclopedia of Life Support Systems (EOLSS)
PROBABILITY AND STATISTICS Vol. I Ergodic Properties of Statioary, Markov, ad Regeerative Processes Karl Grill.. The Mea Square Ergodic Theorem As there are two ways i which statioarity ca be defied, amely weak statioarity, which oly states that the mea ad covariace fuctio remai uaffected by a shift time, ad strict statioarity, which demads shift ivariace for all fiitedimesioal margials, there aturally is more tha oe way to formulate ergodic theorems for statioary processes. I the case of a weak sese statioary process, the appropriate type of covergece to cosider is covergece i mea square. Actually, i Statioary Processes a related result has already bee proved, which shall be recalled here: Let ξ( ) be a cetered weakly statioary sequece, ad Z (.) is the associated spectral process (i.e., ad orthogoal icremet process such that ξ ( ) = π e it dz( t) ), the π lim ( k) Z(0) Z(0 ). m ξ = () m k= m If ξ ( t) is a cetered weakly statioary process i cotiuous time, oe ca likewise prove t lim ξ ( xdx ) = Z(0) Z(0 ). (3) t s s where Z is agai the associated spectral process. Thus, the limit (i square mea) of the time average exists ad equals the height of the jump of the radom fuctio Z i 0. So, a ecessary ad sufficiet coditio for the covergece of the mea to the expectatio of ξ ( t) (which is zero i the case studied here, but the geeral case is readily settled by cosiderig ξ ( t m) is the cotiuity of the spectral fuctio F at 0. This criterio is ice, but oe would rather have a criterio i terms of the correlatio fuctio itself; this goal is ot too hard to achieve, ad is give by the followig theorem: Theorem Let ξ ( ) be a weak sese statioary sequece with mea m ad covariace fuctio R (.) ; furthermore, let ξ( ) = ξ( k). (4) The lim Rk ( ) = 0, (5) if ad oly if Ecyclopedia of Life Support Systems (EOLSS)
PROBABILITY AND STATISTICS Vol. I Ergodic Properties of Statioary, Markov, ad Regeerative Processes Karl Grill lim (( ξ ( ) m) ) = 0 E. (6) Without loss of geerality, it may be assumed that m = 0 (otherwise, oe ca pass o to η( ) = ξ ( ) m). It is readily see that (5) is the limit of the covariace of ξ ( ) ad ξ (0), ad that (6) is the limit of the variace of ξ ( ). Thus, the theorem states that the variace of ξ ( ) teds to zero if ad oly if the covariace of ξ (0) ad ξ ( ) teds to zero. R( ) = R( k) (7) ad observe that by Cauchy s iequality, ( R ( )) = ( ( ξ(0) ξ( ))) ( ξ(0) ) ξ( ) ), so (6) implies (5). E E E (8) For the opposite directio, calculate the variace of ξ ( ) : E ( ξ( ) ) = ( ( k) ) ( ( k) ( l)) ξ + ξ ξ = E E k= 0 k< l kr( k) R(0). (9) k = By assumptio, R ( ) 0 as, ad so, by Kroecker s lemma, the first term above teds to zero as. The secod term obviously is egligible, so the theorem is completely proved. TO ACCESS ALL THE 3 PAGES OF THIS CHAPTER, Visit: http://www.eolss.et/eolsssampleallchapter.aspx Bibliography Howard, R.A. (97) Dyamic Probability Systems, 53pp. Vol. II. New York: Wiley [This discusses the semimarkov model] Foguel, S.R. (969) The Ergodic Theory of Markov Processes. 0pp. New York: Va Nostrad [This gives a itroductio to the moder view of ergodic theory for Markov processes] Ecyclopedia of Life Support Systems (EOLSS)
PROBABILITY AND STATISTICS Vol. I Ergodic Properties of Statioary, Markov, ad Regeerative Processes Karl Grill Kalashikov, V. (994) Topics o Regeerative Processes. 40pp. Boca Rato, FI, USA: CRC Press [This gives a detailed study of regeerative processes, icludig some rather ew material] Karli, S., ad Taylor, H.M. (974) A First Course i Stochastic Processes. 557pp. New York: Academic Press [This is a itroductio to the theory of stochastic processes ad also gives the ergodic theorem for statioary processes] Shields, P.C. (996) The Ergodic Theory of Discrete Sample Paths. 5pp. Providece, RI, USA: America Mathematical Society [This discusses the ergodic theory of statioary processes with a fiite state space] Biographical Sketch Karl Grill received the Ph.D. degree from TU Wie i 983. Sice 98 he is with TU Wie where he became a Associate Professor i 988. He was a visitig Professor i the Departmet of Statistics, Uiversity of Arizoa durig 999. From February to August 994, he held NSERC Foreig Researcher Award, Carleto Uiversity, Ottawa, Caada Ecyclopedia of Life Support Systems (EOLSS)