Introduction
Deterministic dynamic optimization, finite horizon Deterministic dynamic optimization, infinite horizon Numerical methods for the cake-eating problem Further reading
Introduction
Copyright By PowCoder代写 加微信 powcoder
Introduction
• Many economics decisions involve time.
• The entire macroeconomics concerns issues of intertemporal decisions.
• Consumption and saving.
• Firm dynamics.
• Economic growth; Business cycle.
• A great many issues in industrial organization are dynamic. • Competition.
• Entry and exit. • Investment.
• Financial economics.
• Stock market participation. • Asset pricing.
• Capital structure.
Taxonomy of dynamic optimization:
• Discrete-time dynamic optimization. • Finite horizon and infinite horizon. • Deterministic and stochastic.
• Continuous-time dynamic optimization (optimal control). • Finite horizon and infinite horizon.
• Deterministic and stochastic.
We study the discrete-time dynamic optimization.
• Deterministic.
• Stochastic and finite-horizon.
Deterministic dynamic optimization, finite horizon
First example: cake-eating
An agent lives for a T periods, t = 0,1,2,3,··· ,T.
At the beginning of period zero the agent is given cake of size x0. In each period t, the agent eats some quantity ct of the cake.
Let xt be the cake size at the beginning of period t, then
xt+1 = xt − ct.
Assume that the cake does not depreciate (spoil) or grow. The consumer’s decision is to choose ct for all periods.
Consuming quantity ct of the cake gives the utility c1−γ
Utility function u(c) satisfies usual properties:
• Strictly increasing in consumption.
• Concave.
• limu′(c)=∞. c→0
The lifetime utility from eating the cake is discount factor.
u(ct)= t , with γ>0,γ≠ 1. 1−γ
β u(ct), where β ∈ (0, 1] is the • The contribution of ct to the lifetime utility is smaller when t is larger.
The agent needs to allocate the quantity of cake to eat ct over the life cycle. The agent’s optimization problem is formulated as:
T c1−γ max βtt
{c0,c1,··· ,cT} t=0 1 − γ
s.t. xt+1=xt−ct, t=0,1,2,···,T.
In addition, ct ≤ xt. In any period t,
• xt is the state variable which was determined in period t − 1. • ct is the control variable.
Some qualitative characterization:
At t = 0, the agent makes the consumption plan for the entire life. The trade-off of delaying consumption:
• Cost: delaying consumption is costly because of discounting, future consumption is less important than current consumption.
• Benefit: delaying consumption benefits because of the concavity of utility function.
Concavity: u(c)>αu(c1)+(1−α)u(c2)forα∈(0,1)andc=αc1 +(1−α)c2. Concavity of utility function implies consumption smoothing over time.
t Figure 1: CRRA utility function: u(c) = c1−γ .
Concavity implies that, if β = 1, then the agent should choose to eat the same amount in all periods, ct = x0/T.
Toseethat,considertwoperiodst=1,2,supposec1 =x0/T−εandc2 =x0/T+ε. Claim: u(c1) + u(c2) ≤ u(x0/T) + u(x0/T).
Let α ∈ 21 , by concavity: 21 u(c1) + 21 u(c2) ≤ u(x0/T) since x0/T = 21 c1 + 21 c2, thus claim is true.
For β < 1, we then should expect that
c0 >c1 >c2 >···>cT,
because future consumption contributes less to the life-time utility than the current consumption.
How does ct depend on γ? u′(c) = c−γ, implying that the larger γ, the faster the decline in marginal utility.
This suggests that the larger γ, the more consumption smoothing, consumption declines at a lower pace towards the later periods.
Summary: The larger the β and γ, the slower the consumption declines as the agent gets older.
Solving the cake-eating problem: the Lagrange method
There are two approaches to obtaining the optimal conditions for allocating the cake over time.
• The Lagrange method. • Dynamic programming.
The two approaches both lead to the same optimal dynamic allocation.
The Lagrange method: treat the dynamic optimization problem as a nonlinear constrained optimization problem.
Let λt be the Lagrange multiplier for the period-t equality constraint, the Lagrangian is
T c1−γ
L= βt t +λt(xt −ct −xt+1) −βTμTxT+1.
The Lagrangian means to maximize utility for each period, which must be done
simultaneously instead of period by period
Note: both ct and xt+1 are choice variables since the maximization is now unconstrained.
The last term is call the complementary slackness condition for the last period, it does not play an active role in finite-horizon problem, since xT+1 = 0.
The first-order necessary conditions are
∂L =βt(c−γ −λt)≡0, t=0,1,···,T−1. ∂ct t
∂L =βt(−λt)+βt+1λt+1 ≡0, t=0,···,T−1. ∂ xt+1
It is noted that, we do not need a first-order condition for the last period t = T, sinceweknowxT+1 =0andcT =xT.
The optimal choices are solved for from two sets of conditions. The Euler equations:
The first-order necessary condition for each period t are
c−γ = λt; λt = βλt+1. t
These two conditions give, for each period t, c−γ = βc−γ .
This equation is called the Euler equation in period t. There are T of them, from
t = 0 to t = T − 1.
The terminal boundary condition for period T:
cT = xT − xT+1, cT is maximized when the agent sets xT+1 = 0. Thus, cT = xT.
The Euler equation says, under the optimal allocation, the current marginal utility equals the discounted future marginal utility.
If β = 1, ct = ct+1 for each period t, as we said it earlier.
Between any two periods, the trade-off leads to equalizing the marginal utilities.
How to solve the problem with paper and pencil? Backward induction.
Start from the terminal boundary condition, go backward until the first period. Example, let T = 2, we then have the following optimal conditions
c−γ = βc−γ 01
c−γ = βc−γ 12
Step 1. Backward induction: we know c2 = x2. x2 is unknown, but is the state
variable in period 2, optimal consumption c2 is a function of state variable. Go backward to period 1, we have c1 = β− γ1 x2.
Go backward to period 0, we have c0 = β− γ1 2 x2.
Step 2. All optimal choices are a function of x2, we are ready to solve for x2. The ”lifetime budget constraint” is x0 = c0 + c1 + c2. Then,
x 0 = β − γ1 2 x 2 + β − γ1 x 2 + x 2 . Thisgives,x2=β−γ12+β−γ1 +1−1×0.
We now have solved the cake-eating problem, since we already obtained ct as functions of x2.
Using the value of x2, we obtain the optimal consumption for each period.
Conjecture: foranarbitraryT,xT =1+β−γ1 +β−γ2 +···+β−γT −1×0,and γ
ct =β−T−txT.
Verify this by yourself, by setting T = 4 and using the backward induction.
Solving the cake-eating problem: dynamic programming
Dynamic programming is the standard mathematical technique to solve large dynamic optimization problems.
Main idea: Convert the T-period problem into recursively a two-period problem. Why? The Euler equation suggests that, in each period, we only need to examine
the trade-off with the next period.
For example, in period t = 1, the Euler equation is u′(c1) = βu′(c2), which is the only necessary condition to characterize the optimal c1; The optimal c2 is characterized by the Euler equation in period t = 2.
Loosely speaking, one only needs to be concerned today and tomorrow, life after tomorrow is the concern when tomorrow arrives.
First, we introduce the value function.
The value function in period t is defined as the sum of discounted maximized
utilities in all subsequent periods including period t.
By the definition, denote the optimal consumption as c∗0 , c∗1 , · · · , c∗T .
The value function in t = 0 is V(x0) = u(c∗0) + βu(c∗1 ) + β2u(c∗2) + · · · + βTu(c∗T). The value function in t = 1 is V(x1) = u(c∗1 ) + βu(c∗2) + β2u(c∗3) + · · · + βT−1u(c∗T). The value function in t = 2 is V(x2) = u(c∗2) + βu(c∗3) + β2u(c∗4) + · · · + βT−2u(c∗T).
The value function in t = T is V(xT) = u(c∗T), but we know c∗T = xT, thus the value function at t = T is V(xT) = u(xT).
Observations
• Value function is the sum of discounted values of indirect utility, pretending we
already solved for optimal consumption.
• Value function in period t is a function of the state variable in period t, in our
• Value function in period t is the current-value, it is not discounted to period
• In the last period, of course, value function equals the value of indirect utility.
The value functions can be written in the following form:
2T V(x0)= max u(c0)+βu(c1)+β u(c2)+···+β u(cT) ,
{c0 ,c1 ,··· ,cT }
which can be written as V(x0) = max β u(ct).
Tt {c0,c1,···,cT} t=0
Clearly, the value function in period t = 0 is defined as the maximized objective function for the entire life.
T t−1 Similarly, V(x1) = max β
V(x2) = max β u(ct). {c2,···,cT} t=2
V(xT) = max u(cT). cT
{c1,···,cT} t=1 T t−2
Second, re-formulate the original problem using value functions. Originally, T t
s.t. xt+1=xt−ct, t=0,1,2,···,T From the definition of the value function, we observe
max β u(ct) {c0 ,c1 ,··· ,cT } t=0
V(x0) = max u(c0) + βu(c1) + β2u(c2) + · · · + βTu(cT)
{c0 ,c1 ,··· ,cT }
= max u(c0) + max
βu(c1) + β2u(c2) + · · · + βTu(cT)
u(c1) + βu(c2) + · · · + βT−1u(cT)
c0 {c1,c2,···,cT}
= max u(c0) + β max
c0 {c1 ,c2 ,··· ,cT }
= max [u(c0) + βV(x1)] . c0
In the big curly brackets, it says that when we choose c0 to maximize u(c0), we also take into account that a change c0 will change x1 since x1 = x0 − c0.
This means, when choosing c0, we need to take into account value function V(x1).
we cannot simply maximize u(c0) and ignore the rest. In period t = 1, the agent maximizes the sum of utilities from t = 1 on, but that entire value function in period t = 1 is dependent on the chosen value of c0.
By now, we have re-formulated the T-period optimization problem into a two-period problem.
In period t = 0, given x0, the agent chooses the optimal c0 to maximize V(x0):
subject to x1 = x0 − c0.
V(x0) = max [u(c0) + βV(x1)] . c0
Bellman equation
The objective function in the re-formulated problem is called the Bellman equation
(for period t = 0)
V(x0) = max [u(c0) + βV(x1)] . c0
Dynamic programming is the technique of solving dynamic optimization problems by studying the Bellman equations.
How to solve for the Bellman equation in period t = 0? There are different ways, we introduce them later.
We can obtain the Euler equation from the Bellman equation: Re-write the Bellman equation as V(x0) = maxx1 [u(x0 − x1) + βV(x1)] Take the derivatives w.r.t. x1 on terms inside the square brackets:
−u′(c0) + βV′(x1) = 0. But, V′(x1) = u′(c1), since we observe V′(x0) = u′(c0).
We then obtain the Euler equation for period t = 0: u′(c0) = βu′(c1).
Digression: why is V′(x1) = u′(c1)? From the Bellman equation, we have
V′(x0)=du(c0)·dc0 +βdV(x1)·dx1. dc0 dx0 dx1 dx0
Noting, dc0/dx0 = 1 − dx1/dx0 since c0 = x0 − x1. Then, V′(x0) = du(c0) + −du(c0) + βdV(x1) · dx1 .
dc0 dc0 dx1 dx0
Now, −du(c0) + βdV(x1) = 0 since it is the first-order optimal condition for c0.
• To see that, notice du(c0)/dc0 = du(c0)/dx1.
Therefore, V′(x0) = du(c0) , then V′(x1) = du(c1) . dc0 dc1
Loosely speaking, V′(x0) = du(c0) means that choices in the future will have been already optimized. dc0
The Bellman equation above is for period t = 0, we can obtain the Bellman equation for any period t.
In an arbitrary period t, the known is the state variable xt because having arrived period t, xt has already been determined in the past.
The Bellman equation in period t for the cake-eating problem is for cake-eating problem
V(xt) = max [u(ct) + βV(xt+1)] , ct
where xt+1 = xt − ct.
The Euler equation corresponding to this problem is u′(ct) = βu′(ct+1).
In summary, by re-formulating the T-period optimization problem into T − 1 Bellman equations, the optimal conditions remain the same as those derived from the Lagrange method.
Consider a consumer who lives for T periods, and leaves no bequest when consumer dies. Consumer is born with zero asset, that is, a0 = 0. In each period (also age), the consumer’s earnings is yt. Suppose that the consumer works until death. Utility in period t is given by u(ct). Discount factor is 0 < β < 1.
1. Formulate the Bellman equation for this consumer’s optimization problem. 2. Obtain the Euler equation at age t.
Deterministic dynamic optimization, infinite horizon
Deterministic dynamic optimization in infinite horizon
The infinite-horizon cake-eating problem is ∞ c1−γ
s.t. xt+1=xt−ct, t=0,1,2,···.
The first-order necessary conditions are the same as in the problem with finite
• The Euler equation for any two periods t and t + 1 still holds.
Two important differences from the finite-horizon problem:
• T = ∞, there is no longer a terminal condition.
• The discount factor must be less than 1, β ∈ (0, 1).
max βtt {c0,c1,··· ,} t=0 1 − γ
The terminal condition, as T → ∞, becomes a limiting condition, called the
transversality condition
lim βTu′(cT)xT+1 = 0. T→∞
It says, the total value of cake (stock), discounted to today, must equal zero as time goes to infinity.
How do we know if a plan is optimal? A plan satisfying the Euler equation and the transversality condition is optimal, if
1. the utility function is continuous, strictly concave, strictly increasing, and twice differentiable;
2. lim β u(ct) exists and is finite.
Dynamic programming in infinite horizon
The Bellman equation for the infinite-horizon problem is, for any period t, V(xt) = max [u(ct) + βV(xt+1)]
subject to xt+1 = xt − ct.
This is the same as in the finite-horizon problem, it is intuitive since the Bellman
equation concerns two consecutive periods only.
The Bellman equation is recursive, we can remove the time subscripts:
V(x) = max [u(c) + βV(x − c)] c
where we have substituted the constraint for xt+1
Solution method
In the infinite horizon, it is impossible to tabulate the optimal choice for each period since T = ∞.
It is sufficient to obtain the optimal consumption c as a function of state variable x. • This is true for all dynamic optimization problems.
Solving the infinite-horizon problems requires numerical methods.
Numerical methods for the cake-eating problem
Introduction
Rarely, the dynamic optimization problems have the analytical form of solutions. Typically, use numerical methods to solve the Bellman equation:
• finding optimal choices given the state variable.
For problems with terminal conditions (T is finite), backward induction can be used:
• Starting from period T, successively going back to period 0. We have done this earlier.
For problems in infinite horizon (T = ∞), use iterations on the Bellman equation. A main numerical method is the value function iteration (VFI):
• Step 1: guess an initial value of V, that is, set V = V0, for all selected values of x. • Step 2: Update the value function value V to V1 for each value of x, by solving
V1(x)= max{u(c)+βV0(x−c)}. 0≤c≤x
• Repeat Step 2, as follows (in the kth iteration)
Vk+1(x)= max{u(c)+βVk(x−c)}.
• Stop when Vk+1 is approximately equal to Vk.
Implementing the VFI needs to take care of the following:
• State variable x is continuous, we must have a grid of possible values of x. • Computer does not handle continuity.
• For each x value on the grid, in finding x′ (or c), we must interpolate Vk, • because the optimal x′ may not be on the grid.
For example, suppose we pre-set a grid for x, xg = [0.1, 0.5, 1.0, 1.5, 2]. For grid point x = 1.0, the optimal value x′ may be x′ = 0.9, but 0.9 is not a grid point. We only know the value V(x) for x values on grid.
Another method is the policy function iteration (PFI).
Key idea: guess and iterate on the values of c (or x′) instead of the values of V.
See the jupyter notebook for implementing the cake-eating problem with infinite horizon.
Further reading
Further readings
• quantecon.org: Cake-eating problem I
• Dynamic Economics, Chapter 1, Adda and Cooper.
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com