next up previous
Next: Expected Values Up: Computational Methods in Finance Previous: Introduction

Markov Chains

(This section assumes familiarity with basic probability theory using mathematicians' terminology. References on this include the probability books by G. C. Rota, W. Feller, Hoel and Stone, and B. V. Gnedenko.)

Many discrete time discrete state space stochastic models are Markov chains. Such a Markov chain is characterized by its state space, tex2html_wrap_inline213 , and its transition matrix, P. We use the following notations:

The transition probabilities have the properties:




The first is because the p(x,y) are probabilities, the second because the state x must go somewhere, possibly back to x. It is not true that


The Markov property is that knowledge of the state at time t is all the information about the present and past relevant to predicting the future. That is:


no matter what extra history information ( tex2html_wrap_inline261 , tex2html_wrap_inline221 ) we have. This may be thought of as a lack of long term memory. It may also be thought of as a completeness property of the model: the state space is rich enough to characterize the state of the system at time t completely.

The evolution equation for the probabilities u(x,t) is found using conditional probability:


To express this in matrix form, we suppose that the state space, tex2html_wrap_inline213 , is finite, and that the states have been numbered tex2html_wrap_inline271 , tex2html_wrap_inline221 , tex2html_wrap_inline275 . The transition matrix, P, is tex2html_wrap_inline279 and has (i,j) entry tex2html_wrap_inline283 . We sometimes conflate i with tex2html_wrap_inline287 and write tex2html_wrap_inline289 ; until you start programming the computer, there is no need to order the states. With this convention, (gif) can be interpreted as vector-matrix multiplication if we define a row vector tex2html_wrap_inline291 with components tex2html_wrap_inline293 , where we have written tex2html_wrap_inline295 for tex2html_wrap_inline297 . As long as ordering is unimportant, we could also write tex2html_wrap_inline299 . Now, (gif) can be rewritten


Since tex2html_wrap_inline301 is a row vector, the expression tex2html_wrap_inline303 does not make sense because the dimensions of the matrices are incompatible for matrix multiplication. The convention of using a row vector for the probabilities and therefore putting the vector in the left of the matrix is common in applied probability. The relation (gif) can be used repeatedly to yield


where tex2html_wrap_inline305 means P to the power t, not the transpose of P.

next up previous
Next: Expected Values Up: Computational Methods in Finance Previous: Introduction

Jonathan Goodman
Tue Sep 15 17:12:32 EDT 1998