Most valuation problems in quantitative finance represent the desired
value as the expected value of some random variable together with a
stochastic model of market movements.
Given the model, there are two ways
to compute the expected value using evolution equations. One method
starts with expected values known in the future and computes expected
values at successively earlier times until the present expected
values are found. The other method starts with given probabilities
for current market conditions and works forwards in time to find
probabilities for market conditions at a desired future time. These
two evolution equations are similar but not identical. One of the
differences is the natural direction of time change, backwards for
expected values and forwards for probabilities.
This lecture discusses these evolution equations in the simple case of discrete time and discrete ``state space''. The main ideas are contained here. The extension to more complex situations, continuous time or state space, are mainly technical. The relationship between them is called ``duality''. It is an extension of the relationship between a matrix and its transpose.
Many financial instruments allow the holder to make decisions along the way that effect the ultimate value of the instrument. American style options, loans that be repaid early, and convertible bonds are examples. To compute the value of such an instrument, we also seek the optimal decision strategy. Dynamic programming is a computational method that computes the value and decision strategy at the same time. It reduces the difficult ``multiperiod decision problem'' to a sequence of hopefully easier ``single period'' problems. It works backwards in time much as the expectation method does. The tree method commonly used to value American style stock options is an example of the general dynamic programming method.