x ( t ) ,λ( t )) ( x ( t ) − ˆ x ( t )) dt. Moreover, we have M x ( t, ˆ x ( t ) ,λ( t )) = H x ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) (7.18) = − ˙ λ ( t ) , where the first line follows by an Envelope Theorem type reasoning (since H y =0 from equation (7.13)), while the second lin e follows from (7.15). Next, exploiting the de finition of the maximized Hamiltonian, we have Z t 1 0 M (t, x ( t ) ,λ( t )) dt = W ( x ( t ) ,y( t )) + Z t 1 0 λ ( t ) g (t, x ( t ) ,y( t )) dt, and Z t 1 0 M (t, ˆ x ( t ) ,λ( t )) dt = W (ˆ x ( t ) , ˆ y ( t )) + Z t 1 0 λ ( t ) g (t, ˆ x ( t ) , ˆ y ( t )) dt Equation (7.17) together with (7.18) then implies W ( x ( t ) ,y( t )) ≤ W (ˆ x ( t ) , ˆ y ( t )) (7.19) + Z t 1 0 λ ( t )[ g (t, ˆ x ( t ) , ˆ y ( t )) − g (t, x ( t ) ,y( t ))] dt − Z t 1 0 ˙ λ ( t )( x ( t ) − ˆ x ( t )) dt. Integrating the last term by parts and using the fact that by feasibility x (0) = ˆ x (0) = x 0 and by the transversality condition λ ( t 1 ) = 0, we obtain Z t 1 0 ˙ λ ( t )( x ( t ) − ˆ x ( t )) dt = − Z t 1 0 λ ( t ) µ ˙ x ( t ) − · ˆ x ( t ) ¶ dt. Substituting this into (7.19), we obtain W ( x ( t ) ,y( t )) ≤ W (ˆ x ( t ) , ˆ y ( t )) (7.20) + Z t 1 0 λ ( t )[ g (t, ˆ x ( t ) , ˆ y ( t )) − g (t, x ( t ) ,y( t ))] dt + Z t 1 0 λ ( t ) ∙ ˙ x ( t ) − · ˆ x ( t ) ¸ dt. 327 Intr oduction to Modern Economic Growth Since by de finition of the admissible pairs ( x ( t ) ,y( t )) and (ˆ x ( t ) , ˆ y ( t )), we ha ve · ˆ x ( t )= g (t, ˆ x ( t ) , ˆ y ( t )) and ˙ x ( t )= g (t, x ( t ) ,y( t )), (7.20) implies that W ( x ( t ) ,y( t )) ≤ W (ˆ x ( t ) , ˆ y ( t )) for any admissible pair ( x ( t ) ,y( t )), establishing the fi rstpartofthe theorem. If M is strictly concave in x, then the inequality in (7.17) is strict, and therefore thesameargumentestablishes W ( x ( t ) ,y( t)) <W(ˆ x ( t ) , ˆ y ( t )), and no other ˆ x ( t ) could achieve the same value, establishing the second part. ¤ Theorems 7.4 and 7.5 pla y an important ro le in the applications of optimal con- trol. They ensure that a pair (ˆ x ( t ) , ˆ y ( t )) that satisfi es the necessary conditions speci fied in Theorem 7.3 and the su fficiency conditions in either Theorem 7.4 or Theorem 7.5 is indeed an optimal solution. This is importan t, since without Theo- rem 7.4 and Theorem 7.5, Theorem 7.3 does not tell us that there exists an interior continuous solution, thus an admissible pair that satis fies the conditions of Theorem 7.3 may not constitute an optimal solution. Unfortunately, however, both Theorem 7 .4 and Theorem 7.5 are not straightfor- ward to check since neither concavity nor convexity of the g ( ·) function would guar- antee the concavity of the Hamiltonian u nless we know something about the sign of the costate variable λ ( t). Nevertheless, in many economically interesting situations, we can ascertain that the costate variable λ ( t) is ev erywhere positive. For example, a su ffi cient (but not necessary) condition for this would be f x ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) > 0 (see Exercise 7.9). Below we will see that λ ( t) is related to the value of relaxing the constraint on the maximization problems, whic h also gives us another way of ascertaining that it is positive (or negative depending on the problem). Once we know that λ ( t) is positive, c h ecking Theorem 7.4 is straigh tforward, especially when f and g are concave functions. 7.2.2. Generalizations. The abo ve theorems can be immediately generalized to the case in which the state variable and the controls are vectors rather than scalars, and also to the case in which there are other constraints. The constrained case requires constraint quali fication conditions as in the standard finite-dimensional optimization case (see, e.g., Simon and Blume, 1994). These are slightly more messy 328 Intr oduction to Modern Economic Growth to express, and since we will make no use of the constrained maximization problems in this book, we will not state these theorems. The vector-valued theorems are direct generalizations of the ones presented above and are useful in gro w th models with multiple capital goods. In particular, let (7.21) max x ( t ) , y ( t ) W ( x ( t ) , y ( t )) ≡ Z t 1 0 f (t, x ( t ) , y ( t )) dt subject to (7.22) ˙ x ( t)=g ( t, x ( t ) , y ( t)) , and (7.23) y ( t ) ∈ Y ( t)forall t , x (0) = x 0 and x ( t 1 )= x 1 . Here x ( t ) ∈ R K for some K ≥ 1 is the state variable and again y ( t ) ∈ Y ( t ) ⊂ R N for some N ≥ 1 is the control variable. In addition, we again assume that f and g are con tinuously di fferentiable functi ons. We then have: Theorem 7.6. (Maximum Principle for Multivariate Problems) Con- sider the problem of maximizing (7.21) subject to (7.22) and (7.23), with f and g continuously di fferentiable, has an interior continuous solution ˆy ( t ) ∈ IntY ( t ) with corresponding path of state variable ˆx ( t ) .Let H (t, x , y , λ ) be given by (7.24) H (t, x , y , λ ) ≡ f (t, x ( t ) , y ( t)) + λ ( t ) g ( t, x ( t ) , y ( t)) , where λ ( t ) ∈ R K . Then the optimal control ˆy ( t ) and the corresponding path of the state variable x ( t ) satisfy the following necessary conditions: (7.25) ∇ y H (t, ˆx ( t ) , ˆy ( t ) , λ ( t)) = 0 for all t ∈ [0 ,t 1 ] . (7.26) ˙ λ ( t)= −∇ x H (t, ˆx ( t ) , ˆy ( t ) , λ ( t)) for all t ∈ [0 ,t 1 ] . (7.27) ˙ x ( t)=∇ λ H (t, ˆx ( t ) , ˆy ( t ) , λ ( t)) for all t ∈ [0 ,t 1 ] , x (0) = x 0 and x (1) = x 1 . Proof. SeeExercise7.10. ¤ Moreover, we have straightforward generalizations of the su fficiency conditions. The proofs of these theorems are v ery similar to those of Theorems 7.4 and 7.5 and are thus omitted. 329 Intr oduction to Modern Economic Growth Theorem 7.7. (Mangasarian Suffi cient Conditions) Consider the prob- lem of maximizing (7.21) subject to (7.22) and (7.23), with f and g continuously di fferentiable. De fine H (t, x , y , λ ) as in (7.24), and suppose that an interior contin- uous solution ˆy ( t ) ∈ IntY ( t ) and the corresponding path of state variable ˆx ( t ) satisfy (7.25)-(7.27). Suppose also that for the resulting costate variable λ ( t ) , H ( t, x , y , λ ) is jointly concave in ( x , y ) for all t ∈ [0 ,t 1 ] , then ˆy ( t ) and the corresponding ˆx ( t ) achieves a global maximu m of (7.21). Moreover, if H (t, x , y , λ ) is strictly jointly concave, then the pair ( ˆx ( t ) , ˆy ( t)) achieves the unique global maximum of (7.21). Theorem 7.8. (Arrow Su fficient Conditions) Consider the problem of maximizing (7.21) subject to (7.22) and (7.23), with f and g c ontinuously di ffer- entiable. De fine H (t, x , y , λ ) as in (7.24), and suppose that an interior continu- ous solution ˆy ( t ) ∈ IntY ( t ) and the corresponding path of state variable ˆx ( t ) sat- isfy (7.25)-(7.27). Suppose also that for the resulting costate variable λ ( t ) ,de fi ne M (t, x , λ ) ≡ H (t, x , ˆy , λ ) .If M (t, x , λ ) is concave in x for all t ∈ [0 ,t 1 ] ,then ˆy ( t ) and the corresponding ˆx ( t ) achieve a global maximum of (7.21). Moreover, if M (t, x , λ ) is strictly concave in x, then the pair ( ˆx ( t ) , ˆy ( t)) achieves the unique global maximum of (7.21). The proofs of both of these Theorems are similar to that of Theorem 7.5 and are left to the reader. 7.2.3. Limitations. The limitations of what we have done so far are obvious. First, we have assumed that a contin uous and interior solution to the optimal control problem exists. Second, and equally important for our purposes, we have so far looked at the finite horizon case, whereas analysis of growth models requires us to solve in fi nite horizon problems. To deal with both of these issues, we need to look at the more modern theory of optimal con trol. This is done in the next section. 7.3. In finite-Horizon Optimal Control The results presented so far are most useful in developing an int uition for how dynamic optimization in continuous time works. While a number of problems in eco- nomics require finite-horizon optimal control, most economic problems–including 330 Intr oduction to Modern Economic Growth almost all growth models–are more naturally formulated as in finite-horizon prob- lems. This is obvious in the context of economic growth, but is also the case in repeated games, political economy or indust rial organization, where even if individ- uals ma y have finite expected lifes, the end date of the game or of their lives may be uncertain. For this reason, the canoni cal model of optimization and economic problems is the in finite-horizon one. 7.3.1. The Basic Problem: Necessary and Su fficient Conditions. Let us focus on in finite-horizon control with a single control and a single state variable. Using the same notation as above, the problem is (7.28) max x ( t ),y( t ) W ( x ( t ) ,y( t )) ≡ Z ∞ 0 f (t, x ( t ) ,y( t )) dt subject to (7.29) ˙ x ( t )= g (t, x ( t ) ,y( t )) , and (7.30) y ( t ) ∈ R for all t , x (0) = x 0 and lim t →∞ x ( t ) ≥ x 1 . The main di fference is that now time runs to in finity. Note also that this problem allows for an implicit choice over the endpoin t x 1 , since there is no terminal date. The last part of (7.30) imposes a lower bound on this endpoint. In addition, w e ha ve further simpli fied the problem by removing the feasibility requirement that the control y ( t) should always belong to the set Y, instead simply requiring this function to be real-valued. For this problem, we call a pair ( x ( t ) ,y( t)) admissible if y ( t) is a piecewise continuous function of time, meaning that it has at most a finite number of discon- tinuities. 4 Since x ( t) is given by a continuous di fferen tial equation, the piecewise continuity of y ( t)ensuresthat x ( t) is piecewise smooth. Allowing for piecewise continuous con trols is a signi ficant generalization of the above approach. There are a number of technical di fficulties when dealing with the in finite-horizon case, which are similar to those in the discrete time analysis. Primary among those 4 More generally, y ( t) could be allowed to have a countable number of discontinuities, but this added generality is not necessary for any economic application. 331 Intr oduction to Modern Economic Growth is the fact that the value of the functional in (7.28) may not be finite. We will deal with some of these issues below. The main theorem for the in finite-horizon optimal con trol problem is the follow- ing more general version of the Maximum Principle. Before stating this theorem, let us recall that the Hamiltonian is de fined by (7.12), with the only di fference that the horizon is now in finite. In addition, let us de fine the value function,whichisthe analogue of the value function in discre te time dynamic programming introduced in the previous chapter: V ( t 0 ,x 0 ) ≡ max x ( t ) ∈ R ,y( t ) ∈ R Z ∞ t 0 f (t, x ( t ) ,y( t )) dt (7.31) subject to ˙ x ( t )= g (t, x ( t ) ,y( t )) , x ( t 0 )= x 0 and lim t →∞ x ( t ) ≥ x 1 . In words, V ( t 0 ,x 0 ) gives the optimal value of the dynamic maximization problem starting at time t 0 with state variable x 0 .Clearly,wehavethat (7.32) V ( t 0 ,x 0 ) ≥ Z ∞ t 0 f (t, x ( t ) ,y( t )) dt for any admissible pair ( x ( t ) ,y( t )) . Note that as in the previous chapter, there are issues related to whether the “max” is reached. When it is not reached, we should be using “sup” instead. However, recall that we have assumed that all admissible pairs give finite value, so that V ( t 0 ,x 0 ) < ∞ , and our focus throughout will be on admissible pairs (ˆ x ( t ) , ˆ y ( t )) that are optimal solutions to (7.28) subject to (7.29) and (7.30), and thus reach the value V ( t 0 ,x 0 ). Our first result is a w eaker version of the Principle of Optimality, which we encountered in the context of discrete time dynamic programming in the previous chapter: Lemma 7.1. (Principle of Optimality) Suppose that the pair (ˆ x ( t ) , ˆ y ( t )) is an optimal solution to (7.28) subject to (7.29) and (7.30), i.e., it reaches the maximum value V ( t 0 ,x 0 ) .Then, (7.33) V ( t 0 ,x 0 )= Z t 1 t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt + V ( t 1 , ˆ x ( t 1 )) for all t 1 ≥ t 0 . 332 Intr oduction to Modern Economic Growth Proof. We have V ( t 0 ,x 0 ) ≡ Z ∞ t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt = Z t 1 t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt + Z ∞ t 1 f (t, ˆ x ( t ) , ˆ y ( t )) dt. The proof is completed if V ( t 1 , ˆ x ( t 1 )) = R ∞ t 1 f (t, ˆ x ( t ) , ˆ y ( t )) dt .Byde fi nition V ( t 1 , ˆ x ( t 1 )) ≥ R ∞ t 1 f (t, x ( t ) ,y( t )) dt for all admissible ( x ( t ) ,y( t)). Thus this equality can only fail if V ( t 1 , ˆ x ( t 1 )) > R ∞ t 1 f (t, ˆ x ( t ) , ˆ y ( t )) dt .Toobtainacon- tradiction, suppose that this is the case. Then there must exist an admissible pair from t 1 onwards, (˜ x ( t ) , ˜ y ( t )) with ˜ x ( t 1 )=ˆ x ( t 1 )suchthat R ∞ t 1 f (t, ˜ x ( t ) , ˜ y ( t )) dt > R ∞ t 1 f (t, ˆ x ( t ) , ˆ y ( t )) dt . Then construct the pair ( x ( t ) , y ( t )) such that ( x ( t ) , y ( t )) = (ˆ x ( t ) , ˆ y ( t )) for all t ∈ [ t 0 ,t 1 ]and( x ( t ) , y ( t )) = (˜ x ( t ) , ˜ y ( t )) for all t ≥ t 1 .Since (˜ x ( t ) , ˜ y ( t )) is admissible from t 1 onwards with ˜ x ( t 1 )=ˆ x ( t 1 ), ( x ( t ) , y ( t )) is admis- sible, and moreover, Z ∞ t 0 f (t, x ( t ) , y ( t )) dt = Z t 1 t 0 f (t, x ( t ) , y ( t )) dt + Z ∞ t 1 f (t, x ( t ) , y ( t )) dt = Z t 1 t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt + Z ∞ t 1 f (t, ˜ x ( t ) , ˜ y ( t )) dt > Z t 1 t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt + Z ∞ t 1 f (t, ˆ x ( t ) , ˆ y ( t )) dt = Z ∞ t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt = V ( t 0 ,x 0 ) , which con tradicts (7.32) establishing that V ( t 1 , ˆ x ( t 1 )) = R ∞ t 1 f (t, ˆ x ( t ) , ˆ y ( t )) dt and thus (7.33). ¤ Two features in this version of the Principle of Optimalit y are noteworth y. First, in contrast to the similar equation in the previous chapter, it may appear that there is no discounting in (7.33). This is not the case, since the discounting is embedded in the instantaneous payo ff function f , and is th us implicit in V ( t 1 , ˆ x ( t 1 )). Second, this lemma may appear to contradict our discussion of “time consistency” in the previous chapter, since the lemma is state d without additional assumptions that ensure time consistency. The important poin t here is that in the time consistency 333 Intr oduction to Modern Economic Growth discussion, the decision-maker considered updating his or her plan, with the payo ff function being potentially di fferent after date t 1 (at least because bygones we re bygones). In contrast, here the payo ff function remains constant. The issue of time consistency is discussed further in Exercise 7.19. We now state one of the main results of this chapter. Theorem 7.9. (Infi nite-Horizon Maximum Principle) Suppose that prob- lem of maximizing (7.28) subject to (7.29) and (7.30), with f and g continuously di fferentiable, has a piecewise continuous solution ˆ y ( t ) with corresponding path of state variable ˆ x ( t ) .Let H (t, x, y, λ) be given by (7.12). Then the optimal control ˆ y ( t ) and the corresponding path of the state variable ˆ x ( t ) are such that the Hamil- tonian H (t, x, y, λ) satis fies the Maximum Principle, that H (t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) ≥ H (t, ˆ x ( t ) ,y,λ ( t)) for all y ( t ) , for all t ∈ R . Moreover, whenever ˆ y ( t ) is continuous, the following necessary con- ditions are satis fied: (7.34) H y ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) = 0 , (7.35) ˙ λ ( t )=− H x ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) , (7.36) ˙ x ( t )= H λ ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) ,with x (0) = x 0 and lim t →∞ x ( t ) ≥ x 1 , for all t ∈ R + . The proof of this theorem is relatively long and will be provided later in this section. 5 Notice that the optimal solution always satis fies the Maximum Principle. In addition, whenever the optimal control, ˆ y ( t ), is a continuous function of time, the conditions (7.34)-(7.36) are also satis fied. This quali fication is necessary, since we now allow ˆ y ( t ) to be a piecewise continuous function of time. The fact that ˆ y ( t ) 5 The reader may also wonder when an optimal p iecewise continuous solution will exist as h y pothesized in the theorem. Unfortunately, the conditions to ensure that a solution exists are rather involved. The most straightforward approach is to look for Lebesgue integrable controls, and impose enough structure to ensure that the cons traint set is compact and the objective function is continuous. In most economic problems there w ill be enough structure to ensure the existence of an interior solution and this structure will also often guarantee that the solution is continuous. 334 Intr oduction to Modern Economic Growth is a piecewise continuous function implies that the optimal control may include dis- continuities, but these will be relatively “rare”–in particular, it will be continuous “most of the time”. More important, the added generality of allowing discontinuities is somewhat super fluous in most economic applicatio ns, because economic problems often have enough structure to ensure that ˆ y ( t ) is indeed a contin u ous function of time. Consequently, in most economic problems (and in all of the models studied in this book) it will be su ffi cient to focus on the necessary conditions (7.34)-(7.36). It is also useful to hav e a di fferen t version of the necessary conditions in Theorem 7.9, which are directly comparable to the necessary conditions generated by dynamic programming in the discrete time dynami c optimization problems studied in the previous chapter. In particular, the necessary conditions can also be expressed in the form of the so-called Hamilton-Jacobi-Bellman (HJB) equation. Theorem 7.10. (Hamilton-Jacobi-Bellman Equations) Let V (t, x) be as de fined in (7.31) and suppose that the hypotheses in Theorem 7.9 hold. Then when- ever V (t, x) is di fferentiable in ( t, x ), the optimal pair (ˆ x ( t ) , ˆ y ( t )) satis fi es the HJB equation: (7.37) f (t, ˆ x ( t ) , ˆ y ( t )) + ∂V (t, ˆ x ( t )) ∂t + ∂V (t, ˆ x ( t )) ∂x g (t, ˆ x ( t ) , ˆ y ( t )) = 0 for all t ∈ R . Proof. FromLemma7.1,wehavethatfortheoptimalpair(ˆ x ( t ) , ˆ y ( t )), V ( t 0 ,x 0 )= Z t t 0 f (s, ˆ x ( s ) , ˆ y ( s )) ds + V (t, ˆ x ( t )) for all t . Di fferentiating this with respect to t and using the di fferentiability of V and Leibniz’s rule, we obtain f (t, ˆ x ( t ) , ˆ y ( t )) + ∂V (t, ˆ x ( t )) ∂t + ∂V (t, ˆ x ( t )) ∂x ˙ x ( t ) = 0 for all t. Setting ˙ x ( t )= g (t, ˆ x ( t ) , ˆ y ( t )) gives (7.37). ¤ The HJB equation will be useful in providing an intuition for the Maximum Principle, in the proof of Theorem 7.9 and also in many of the endogenous technology models studied below. For now it su ffices to note a few important features. First, given that the continuous di fferentiability of f and g, the assumption that V (t, x) is di fferentiable is not very restrictive, since the optimal con trol ˆ y ( t ) is piecewise 335 Intr oduction to Modern Economic Growth continuous. From the de finition (7.31), at all t where ˆ y ( t ) is continuous, V (t, x )will also be di fferentiable in t. Moreo ver, an envelope theorem type argument also implies that when ˆ y ( t ) is continuous, V (t, x ) should also be di fferentiable in x (though the exact conditions to ensure di fferentiability in x are somewhat involved). Second, (7.37) is a partial di fferential equation, since it features the derivative of V with respect to both time and the state variable x. Third, this partial di fferential equation also has a similarity to the Euler equation derived in the context of discrete time dynamic programming. In particular, the simplest Euler equation (6.22) required the current gain from increasing the control variable to be equal to the discounted loss of value. The current equation has a similar interpretation, with the first term corresponding to the current gain and the la st term to the potential discounted loss of value. The second term results from the fact that the maximized value can also change over time. Since in Theorem 7.9 there is no boundary condition similar to x ( t 1 )= x 1 ,we may expect that there should be a transversality condition similar to the condition that λ ( t 1 ) = 0 in Theorem 7.1. One might be tempted to impose a transversality condition of the form (7.38) lim t →∞ λ ( t )=0 , which would be generalizing the condition that λ ( t 1 ) = 0 in Theorem 7.1. But this is not in general the case. We will see an example where this does not apply soon. A milder transversality condition of the form (7.39) lim t →∞ H (t, x, y, λ )=0 always applies, but is not easy to ch eck. S tronger transversality conditions apply when we put more structure on the problem. We will discuss these issues in Section 7.4 below. Before presenting these result s, there are immediat e generalizations of the su fficiency theorems to this case. Theorem 7.11. (Mangasarian Suffi cient Conditions for Infi nite Hori- zon) Consider the problem of maximizing (7.2 8) subject to (7.29) and (7.30), with f and g continuously di fferentiable. De fine H (t, x, y, λ) as in (7.12), and suppose that a piecewise continuous solution ˆ y ( t ) and the corresponding path of state variable 336 Intr oduction to Modern Economic Growth ˆ x ( t ) satisfy (7.34)-(7.36). Suppose also t hat for the resulting costate variable λ ( t ) , H (t, x, y, λ) is jointly concave in ( x, y ) for all t ∈ R + and that lim t →∞ λ ( t )(ˆ x ( t ) − ˜ x ( t )) ≤ 0 for all ˜ x ( t ) implied by an admissible control path ˜ y ( t ) ,then ˆ y ( t ) and the corre- sponding ˆ x ( t ) achieve the unique global maximum of (7.28). Theorem 7.12. (A rrow Su fficient Conditions for In finite Horizon) Consider the problem of maximizing (7.28) subject to (7.29) and (7.30), with f and g continuously di fferentiable. De fine H (t, x, y, λ) as in (7.12), and suppose that a piecewise continuous solution ˆ y ( t ) and the corresponding path of state variable ˆ x ( t ) satisfy (7.34)-(7.36). Given the resulting costate variable λ ( t ) ,de fi ne M (t, x, λ) ≡ H (t, x, ˆ y ( t ) ,λ ) .If M ( t, x, λ ) is concave in x and lim t →∞ λ ( t )(ˆ x ( t ) − ˜ x ( t )) ≤ 0 for all ˜ x ( t ) implied by an admissible control path ˜ y ( t ) ,thenthepair (ˆ x ( t ) , ˆ y ( t )) achieves the unique global maximum of (7.28). The proofs of both of these theorems are similar to that of Theorem 7.5 and are left for the reader (See Exercise 7.11). Notice that both of these su fficiency theorems involve the di fficult to check condi- tion that lim t →∞ λ ( t )( x ( t ) − ˜ x ( t )) ≤ 0forall˜ x ( t )impliedbyanadmissiblecontrol path ˜ y ( t ). This condition will disappear when we can impose a proper transversality condition. 7.3.2. Economic Intuition. The Maximum Principle is not only a po werful mathematical tool, but from an economic point of view, it is the right tool ,because it captures the essential economic intuition of dynamic economic problems. In this subsection, we provide t wo di fferent and complementary eco nomic intuitions for the Maximum Principle. One of them is based o n the original form as stated in Theorem 7.3 or Theorem 7.9, while the other is based on the dynamic programming (HJB) versionprovidedinTheorem7.10. To obtain the first intuition consider the problem of maximizing (7.40) Z t 1 0 H (t, ˆ x ( t ) ,y( t ) ,λ( t )) dt = Z t 1 0 [ f ( t, ˆ x ( t ) ,y( t )) + λ ( t ) g (t, ˆ x ( t ) ,y( t ))] dt with respect to the en tire function y ( t) for given λ ( t)andˆ x ( t ), where t 1 can be finite or equal to + ∞. The condition H y ( t, ˆ x ( t ) ,y( t ) ,λ( t )) = 0 would then 337 Intr oduction to Modern Economic Growth be a necessary condition for this alternative maximization problem. Therefore, the Maximum Principle is implicitly maximizing the sum the original maximand R t 1 0 f (t, ˆ x ( t ) ,y( t )) dt plus an additional term R t 1 0 λ ( t ) g (t, ˆ x ( t ) ,y( t )) dt . Under- standing why this is true provides much of the intuition for the Maxim um Principle. First recall that V (t, ˆ x ( t )) is defi nedinequation(7.33)asthevalueofstarting at time t with state variable ˆ x ( t ) and pursuing the optimal policy from then on. We will see in the next subsection, in particular in equation (7.43), that λ ( t )= ∂V (t, ˆ x ( t )) ∂x . Therefore, similar to the Lagrange mult ipliers in the theory of constraint opti- mization, λ ( t) measures the impact of a small increase in x on the optimal value of the program. Consequently, λ ( t) is the (shadow) value of relaxing the con- straint (7.29) by increasing the value of x ( t)attime t . 6 Moreo ver, recall that ˙ x ( t )= g (t, ˆ x ( t ) ,y( t )), so that the second term in the Hamiltonian is equivalent to R t 1 0 λ ( t ) ˙ x ( t ) dt . This is clearly the shadow value of x ( t)attime t and the increase in the stock of x ( t) at this point. Moreover, recall that x ( t) is the state variable, thus we can think of it as a “stock” variable in contrast to the control y ( t), which corresponds to a “ flow” variable. Therefore, maximizing (7.40) is equivale nt to maximizing instantaneous returns as given by the function f (t, ˆ x ( t ) ,y( t )), plus the value of stock of x ( t), as given by λ ( t), times the increase in the stock, ˙ x ( t ). This implies that the essence of the Maximum Principle is to maximize the flow return plus the value of the current stock of the state variable. This stock- flo w type maximization has a clear economic logic Let us next turn to the interpreting the costate equation, ˙ λ ( t )=− H x ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) = − f x ( t, ˆ x ( t ) , ˆ y ( t )) − λ ( t ) g x ( t, ˆ x ( t ) , ˆ y ( t )) . This equation is also intuitive. Since λ ( t) is the value of the stock of the state variable, x ( t), ˙ λ ( t ) is the appreciation in this stock variable. A small increase in x 6 Here I am using the language of “relaxing the constraint” implicitly presuming that a high value of x ( t) contributes to increasing the value of the objectiv e function. This simpli fies termi- nology, but is not necessary for any of the arguments, since λ ( t) can be negative. 338 Intr oduction to Modern Economic Growth will change the current flow return plus the value of the stock by the amount H x , butitwillalsoa ff ect the value of the stock by the amount ˙ λ ( t ). The Maximum Principle states that this gain should be equal to the depreciation in the value of the stock, − ˙ λ ( t ), since, otherwise, it would be possible to c hange the x ( t ) and increase the value of H (t, x ( t ) ,y( t)). The second and complementary intuition for the Maximum Principle comes from the HJB equation (7.37) in Theorem 7.10. In particular, let us consider an expo- nentially discounted problem like those di scussed in greater detail in Section 7.5 below, so that f (t, x ( t ) ,y( t)) = exp ( − ρt) f ( x ( t ) ,y ( t)). In this case, clearly V (t, x ( t )) = exp (−ρt) V ( x ( t )), and moreover, by de finition, ∂V (t, x ( t)) ∂t =exp( − ρt) h ˙ V ( x ( t )) − ρV ( x ( t )) i . Using these observations and the fact that V x ( t, ( x ( t))) = λ ( t), the Hamilton- Jacobi-Bellman equation takes the “stationary” form ρV (ˆ x ( t )) = f (ˆ x ( t ) , ˆ y ( t )) + λ ( t ) g (t, ˆ x ( t ) , ˆ y ( t )) + ˙ V (ˆ x ( t )) . Thisisaverycommonequationindynamic economic analysis and can be interpreted as a “no-arbitrage asset value equation”. We can think of V ( x)asthevalueofan asset traded in the stock market and ρ as the required rate of return for (a large num ber of) investors. When will investors be happ y to hold this asset? Loosely speaking, they will do so when the asset pays out at least the required rate of return. In contrast, if the asset pays out more than the required rate of return, there would be excess demand for it from t he investors until its value adjusts so that its rate of return becomes equal to the required rate of return. Therefore, we can think of the return on this asset in “equilibrium” being equal to the required rate of return, ρ. The return on the assets come from two sources: first, “dividends,” that is current returns paid out to investors. In the current context, we can think of this as f (ˆ x ( t ) , ˆ y ( t )) + λ ( t ) g (t, ˆ x ( t ) , ˆ y ( t )) (with an argument similar to the above discussion). If this dividend were constant and equal to d, and there were no other returns, then we would naturally have that V ( x)= d/ρ or ρV ( x)= d. 339 Intr oduction to Modern Economic Growth However, in general the returns to the holding an asset come not only from dividends but also from capital gains or losses (appr eciation or depreciation of the asset). In the current context, this is equal to ˙ V ( x ). Therefore, instead of ρV ( x )= d ,we have ρV ( x)=d + ˙ V ( x ) . Thus, at an in tuitive lev el, the Maximum Principle amounts to requiring that the maximized value of dynamic maximization program, V ( x), and its rate of change, ˙ V ( x ), should be consistent with th is no-arbitrage condition. 7.3.3. Proof of Theorem 7.9*. In this subsection, we provide a sketch of the proof of Theorems 7.9. A fully rigorous proof of Theorem 7.9 is quite long and involved. It can be found in a number of sources mentioned in the references below. The version provided here con tains all the basic ideas, but is stated under the assumption that V (t, x )istwicedi ff erentiable in t and x . As discussed above, the assumption that V (t, x )isdi ff eren tiable in t and x is not particularly restrictive, though the additional assumption that it is twice di fferentiable is quite stringent. The main idea of the proof is due to Pontryagin and co-authors. Instead of smooth variations from the optimal pair (ˆ x ( t ) , ˆ y ( t )), the method of proof considers “needle-like” variations, that is, piecewise continuous paths for the control variable that can deviate from the optimal contro l path by an arbitrary amount for a small interval of time. Sketch Proof of Theorem 7.9: Suppose that the admissible pair (ˆ x ( t ) , ˆ y ( t )) is a solution and attains the maximal value V (0 ,x 0 ). Take an arbitrary t 0 ∈ R + . Construct the following perturbation: y δ ( t)=ˆ y ( t ) for all t ∈ [0 ,t 0 )andforsome su fficiently small ∆ t and δ ∈ R , y δ ( t)=δ for t ∈ [ t 0 ,t 0 + ∆ t ] for all t ∈ [ t 0 ,t 0 + ∆ t ]. Moreover, let y δ ( t)for t ≥ t 0 + ∆ t be the optimal control for V ( t 0 + ∆ t, x δ ( t 0 + ∆ t )), where x δ ( t) is the value of the state variable resulting from the perturbed con- trol y δ ,with x δ ( t 0 + ∆ t ) being the value at time t 0 + ∆ t . Note by construction x δ ( t 0 )=ˆ x ( t 0 )(since y δ ( t)=ˆ y ( t )forall t ∈ [0 ,t 0 ]). 340 Intr oduction to Modern Economic Growth Since the pair (ˆ x ( t ) , ˆ y ( t )) is optimal, w e have that V ( t 0 , ˆ x ( t 0 )) = Z ∞ t 0 f (t, ˆ x ( t ) , ˆ y ( t )) dt ≥ Z ∞ t 0 f (t, x δ ( t ) ,y δ ( t)) dt = Z t 0 + ∆ t t 0 f (t, x δ ( t ) ,y δ ( t)) dt + V ( t 0 + ∆ t, x δ ( t 0 + ∆ t )) , where the last equality uses the fact that the admissible pair ( x δ ( t ) ,y δ ( t)) is optimal starting with state variable x δ ( t 0 + ∆ t )attime t 0 + ∆ t . Rearranging terms and dividing by ∆ t yields V ( t 0 + ∆ t, x δ ( t 0 + ∆ t )) − V ( t 0 , ˆ x ( t 0 )) ∆ t ≤− R t 0 + ∆ t t 0 f (t, x δ ( t ) ,y δ ( t)) dt ∆ t for all ∆ t ≥ 0. Now tak e limits as ∆ t → 0 and note that x δ ( t 0 )=ˆ x ( t 0 )andthat lim ∆ t → 0 R t 0 + ∆ t t 0 f (t, x δ ( t ) ,y δ ( t)) dt ∆ t = f ( t, x δ ( t ) ,y δ ( t)) . Moreover, let T ⊂ R + be the set of points where the optimal control ˆ y ( t )isa continuous function of time. Note that T is a dense subset of R + since ˆ y ( t )isa piecewise continuous function. Let us now tak e V to be a di fferentiable function of time at all t ∈ T ,sothat lim ∆ t → 0 V ( t 0 + ∆ t, x δ ( t 0 + ∆ t )) − V ( t 0 , ˆ x ( t 0 )) ∆ t = ∂V (t, x δ ( t)) ∂t + ∂V (t, x δ ( t)) ∂x ˙ x δ ( t ) , = ∂V (t, x δ ( t)) ∂t + ∂V (t, x δ ( t)) ∂x g (t, x δ ( t ) ,y δ ( t)) , where ˙ x δ ( t)=g ( t, x δ ( t ) ,y δ ( t)) is the law of motion of the state variable given b y (7.29) together with the control y δ . Putting all these together, we obtain that f ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) + ∂V ( t 0 ,x δ ( t 0 )) ∂t + ∂V ( t 0 ,x δ ( t 0 )) ∂x g ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) ≤ 0 for all t 0 ∈ T (which correspond to point s of continuity of ˆ y ( t )) and for all admissible perturbation pairs ( x δ ( t ) ,y δ ( t)). Moreov er, from Theorem 7.10, which applies at all t 0 ∈ T , (7.41) f ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) + ∂V ( t 0 , ˆ x ( t 0 )) ∂t + ∂V ( t 0 , ˆ x ( t 0 )) ∂x g ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) = 0 . 341 Intr oduction to Modern Economic Growth Once more using the fact that x δ ( t 0 )=ˆ x ( t 0 ), this implies that f ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) + ∂V ( t 0 , ˆ x ( t 0 )) ∂x g ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) ≥ (7.42) f ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) + ∂V ( t 0 , ˆ x ( t 0 )) ∂x g ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) for all t 0 ∈ T and for all admissible perturbation pairs (x δ ( t ) ,y δ ( t)). Now de fining (7.43) λ ( t 0 ) ≡ ∂V ( t 0 , ˆ x ( t 0 )) ∂x , Inequality (7.42) can be written as f ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) + λ ( t 0 ) g ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) ≥ f ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) + λ ( t 0 ) g ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) H ( t 0 , ˆ x ( t 0 ) , ˆ y ( t 0 )) ≥ H ( t 0 ,x δ ( t 0 ) ,y δ ( t 0 )) for all admissible ( x δ ( t 0 ) ,y δ ( t 0 )) . Therefore, H (t, ˆ x ( t ) , ˆ y ( t )) ≥ max y H (t, ˆ x ( t ) ,y ) . This establishes the Maximum Principle. The necessary condition (7.34) directly follows from the Maximum Principle together with the fact that H is di fferentiable in x and y (a consequence of the fact that f and g are di fferentiable in x and y). Condition (7.36) holds by de finition. Finally, (7.35) follows from di fferentiating (7.41) with respect to x at all points of continuity of ˆ y ( t ), which giv es ∂f (t, ˆ x ( t ) , ˆ y ( t )) ∂x + ∂ 2 V (t, ˆ x ( t )) ∂t∂x + ∂ 2 V (t, ˆ x ( t )) ∂x 2 g (t, ˆ x ( t ) , ˆ y ( t )) + ∂V (t, ˆ x ( t )) ∂x ∂g (t, ˆ x ( t ) , ˆ y ( t )) ∂x =0 , for all for all t ∈ T .Usingthede fi nition of the Hamiltonian, this gives (7.35). ¤ 7.4. More on Transversality Conditions We next turn to a study of the boundary conditions at in finity in in finite-horizon maximization problems. As in the discrete time optimization problems, these limit- ing boundary conditions are referred to as “transversality conditions”. As mentioned 342 Intr oduction to Modern Economic Growth above, a natural conjecture might be that, as in the finite-horizon case, the transver- sality condition should be similar to that in Theorem 7.1, with t 1 replaced with the limit of t →∞ , that is, lim t →∞ λ ( t ) = 0. The following example, whic h is very close to the original Ramsey model, illustrates that this is not the case; without fur- ther assumptions, the valid transversality condition is given by the weaker condition (7.39). Example 7.2. Consider the following problem: max Z ∞ 0 [log ( c ( t)) − log c ∗ ] dt subject to ˙ k ( t )=[ k ( t )] α − c ( t ) − δk ( t ) k (0) = 1 and lim t →∞ k ( t ) ≥ 0 where c ∗ ≡ [ k ∗ ] α − δk ∗ and k ∗ ≡ ( α/δ ) 1 /(1− α ) . In other words, c ∗ is the maximum level of consumption that can be achieved in the steady state of this model and k ∗ is the corresponding steady-state level of capital. This way of writing the objective function makes sure that the integral converges and takes a finite value (since c ( t ) cannot exceed c ∗ forever). The Hamiltonian is straigh tforward to construct; it does not explicitly depend on time and takes the form H (k, c,λ ) = [log c ( t ) − log c ∗ ]+ λ [ k ( t ) α − c ( t ) − δk ( t)] , and implies the following necessary conditions (dropping time dependence to sim- plify the notation): H c ( k,c,λ )= 1 c ( t ) − λ ( t )=0 H k ( k,c,λ )= λ ( t ) ¡ αk ( t ) α − 1 − δ ¢ = − ˙ λ ( t ) . It can be veri fied that any optimal path must feature c ( t ) → c ∗ as t →∞ .This, ho wever, implies that lim t →∞ λ ( t )= 1 c ∗ > 0and lim t →∞ k ( t )= k ∗ . 343 Intr oduction to Modern Economic Growth Therefore, the equivalent of the standard finite-horizon transversality conditions do not hold. It can be veri fied, however, that along the optimal path we have lim t →∞ H ( k ( t ) ,c( t ) ,λ( t )) = 0 . We will next see that this is indeed the relevant transversality condition. Theorem 7.13. Suppose that problem of maximizing (7.28) subject to (7.29) and (7.30), with f and g continuously di fferentiable, has an interior piecewise continuous solution ˆ y ( t ) with corresponding path of state variable ˆ x ( t ) . Suppose moreover that lim t →∞ V (t, ˆ x ( t )) exists (where V (t, x ( t )) is de fined in (7.33)). Let H (t, x, y, λ) be given by (7.12). Then the optimal control ˆ y ( t ) and the corresponding path of the state variable ˆ x ( t ) satisfy the necessary conditions (7.34)-(7.36) and the transversality condition (7.44) lim t →∞ H (t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) = 0 . Proof. Let us focus on points where V (t, x )isdi ff erentiable in t and x so that the Hamilton-Jacobi-Bellman equation, (7.37) holds. Noting that ∂V ( t, ˆ x ( t )) /∂x = λ ( t ), this equation can be written as ∂V (t, ˆ x ( t )) ∂t + f ( t, ˆ x ( t ) , ˆ y ( t )) + λ ( t ) g (t, ˆ x ( t ) , ˆ y ( t )) = 0 for all t ∂V (t, ˆ x ( t )) ∂t + H ( t, ˆ x ( t ) , ˆ y ( t ) ,λ( t )) = 0 for all t. (7.45) Now take the limit as t →∞ . Since lim t →∞ V (t, ˆ x ( t )) exists, we have that ei- ther lim t →∞ ∂V (t, ˆ x ( t )) /∂t > 0 everyw here, so that lim t →∞ V (t, ˆ x ( t )) = +∞ , or lim t →∞ ∂V (t, ˆ x ( t )) /∂t < 0 everywhere, so that lim t →∞ V (t, ˆ x ( t )) = −∞ or lim t →∞ ∂V (t, ˆ x ( t )) /∂t =0. The first two possibilities are ruled out by the hypoth- esis that an optimal solution that reac he s the maximum exists. Thus we must have lim t →∞ ∂V (t, ˆ x ( t )) /∂t = 0. (7.45) then implies (7.44). ¤ The transversality condition (7.44) is no t particularly convenient to w ork with. In the next section, we will see that as we consider discounted in finite-horizon prob- lems stronger and more useful versions of thi s transversality condition can be devel- oped. 344 Intr oduction to Modern Economic Growth 7.5. Discounted In finite-Horizon Optimal Control Part of the di fficulty, especially regarding the ab sence of a transversality condi- tion, comes from the fact that we did not impose enough structure on the functions f and g . As discussed above, our interest is with the grow th models where the utility is discounted exponentially . Consequently , economically interesting problems often take the following more speci ficform: (7.46) max x ( t ),y( t ) W ( x ( t ) ,y( t )) ≡ Z ∞ 0 exp ( − ρt) f ( x ( t ) ,y ( t)) dt with ρ> 0 , subject to (7.47) ˙ x ( t )= g ( x ( t ) ,y( t )) , and (7.48) y ( t ) ∈ R for all t , x (0) = x 0 and lim t →∞ x ( t ) ≥ x 1 . Notice that throughout we assume ρ>0, so that there is indeed discounting . Thespecialfeatureofthisproblemisthattheobjectivefunction, f,dependson time only through exponential discou n ting, while the constraint equation, g,isnot a function of time directly. The Hamiltonian in this case would be: H (t, x ( t ) ,y( t ) ,λ( t )) = exp (−ρt) f ( x ( t ) ,y( t )) + λ ( t ) g ( x ( t ) ,y( t )) =exp( − ρt )[ f ( x ( t ) ,y( t )) + µ ( t ) g ( x ( t ) ,y( t ))] , where the second line de fines (7.49) µ ( t ) ≡ exp ( ρt) λ ( t ) . This equation makes it clear that the Ham iltonian depends on time explicitly only through the exp ( − ρt )term. In fact, in this case, rather than working with the standard Hamiltonian, we can work with the current-value Hamiltonian ,de fi ned as (7.50) ˆ H ( x ( t ) ,y( t ) ,µ( t )) ≡ f ( x ( t ) ,y( t )) + µ ( t ) g ( x ( t ) ,y( t )) which is “autonomous” in the sense that it does not directly depend on time. The following result establishes the ne cessity of a stronger transversality con- dition under some additional assumption s, which are typically met in economic applications. In preparation for thi s result, let us refer to the functions f (x, y )and 345 Intr oduction to Modern Economic Growth g (x, y ) as weakly monotone, if each one is monotone in eac h of its arguments (for example, nondecreasing in x and nonincreasing in y). Furtherm ore, let us simplify the statement of this theorem by a ssuming that the optimal control ˆ y ( t )isevery- where a continuous function of time (though this is not necessary for any of the results). Theorem 7.14. (Maximum Principle for Discounted In finite-Horizon Problems) Suppose that problem of maximizing (7.46) subject to (7.47) and (7.48), with f and g continuously di fferentiable, has a solution ˆ y ( t ) with corresponding path of state variable ˆ x ( t ) . Suppose moreover that lim t →∞ V (t, ˆ x ( t )) exists (where V (t, x ( t )) is de fi ned in (7.33)). Let ˆ H (ˆ x, ˆ y, µ ) be the current-value Hamiltonian given by (7.50). Then the optimal control ˆ y ( t ) and the corresponding path of the state variable ˆ x ( t ) satisfy the following necessary conditions: (7.51) ˆ H y (ˆ x ( t ) , ˆ y ( t ) ,µ( t )) = 0 for all t ∈ R + , (7.52) ρµ ( t ) − ˙ µ ( t )= ˆ H x (ˆ x ( t ) , ˆ y ( t ) ,µ( t )) for all t ∈ R + , (7.53) ˙ x ( t )= ˆ H µ (ˆ x ( t ) , ˆ y ( t ) ,µ( t )) for all t ∈ R + , x (0) = x 0 and lim t →∞ x ( t ) ≥ x 1 , and the transversality condition (7.54) lim t →∞ exp ( − ρt) ˆ H (ˆ x ( t ) , ˆ y ( t ) ,µ( t )) = 0 . Moreover, if f and g are weakly monotone, the transversality condition can be strengthened to: (7.55) lim t →∞ [exp ( − ρt) µ ( t)ˆ x ( t )] = 0 . Proof. The derivation of the necessary conditions (7.51)-(7.53) and the transver- sality condition (7.54) follows by using the de finition of the current-value Hamilton- ian and from Theorem 7.13. They are left for as an exercise (see Exercise 7.13). We therefore only give the proof for the stronger transversalit y condition (7.55). The weaker transversality condition (7.54) can be written as lim t →∞ exp ( − ρt) f (ˆ x ( t ) , ˆ y ( t )) + lim t →∞ exp ( − ρt) µ ( t ) g (ˆ x ( t ) , ˆ y ( t )) = 0 . 346 Intr oduction to Modern Economic Growth The first term m ust be equal to zero, since otherwise lim t →∞ V (t, ˆ x ( t )) = ∞ or −∞ ,andthepair(ˆ x ( t ) , ˆ y ( t )) cannot be reaching the optimal solution. Therefore (7.56) lim t →∞ exp ( − ρt) µ ( t ) g (ˆ x ( t ) , ˆ y ( t )) = lim t →∞ exp ( − ρt) µ ( t ) ˙ x ( t )=0 . Since lim t →∞ V (t, ˆ x ( t )) exists and f and g are w eakly monotone, lim t →∞ ˆ y ( t )and lim t →∞ ˆ x ( t ) must exist, though they may be in finite (otherwise the limit of V (t, ˆ x ( t )) would fail to exist). The latter fact also implies that lim t →∞ ˙ x ( t ) exists (though it may also be in finite). Moreover, lim t →∞ ˙ x ( t ) is nonnegative, since otherwise the condition lim t →∞ x ( t ) ≥ x 1 would be violated. From (7.5 2), (7.54) implies that as t →∞ , λ ( t ) ≡ exp(− ρt ) µ ( t ) → κ for some κ ∈ R + . Suppose firstthatlim t →∞ ˙ x ( t ) = 0. Then lim t →∞ ˆ x ( t )=ˆ x ∗ ∈ R (i.e., a fi nite value). This also implies that f (ˆ x ( t ) , ˆ y ( t )), g (ˆ x ( t ) , ˆ y ( t )) and therefore f y (ˆ x ( t ) , ˆ y ( t )) and g y (ˆ x ( t ) , ˆ y ( t )) limit to constant values. Then from (7.51), we ha ve that as t →∞ , µ ( t ) → µ ∗ ∈ R (i.e., a fi nite value). This implies that κ =0 and (7.57) lim t →∞ exp ( − ρt) µ ( t)=0 , and moreover since lim t →∞ ˆ x ( t )=ˆ x ∗ ∈ R , (7.55) also follows. Suppose now that lim t →∞ ˙ x ( t )= g ˆ x ( t ), where g ∈ R + ,sothatˆ x ( t )grows at an exponential rate . Then substituting this into (7.56) we obtain (7.55). Next, suppose that 0 < lim t →∞ ˙ x ( t ) <g ˆ x ( t ), for any g> 0, so that ˆ x ( t )grows at less than an exponential rate . In this case, since ˙ x ( t )isincreasingovertime, (7.56) implies that (7.57) must hold and thus again we must have that as t →∞ , λ ( t ) ≡ exp(− ρt) µ ( t ) → 0, i.e., κ =0(otherwiselim t →∞ exp ( − ρt) µ ( t ) ˙ x ( t )= lim t →∞ ˙ x ( t ) > 0, violating (7.56)) and thus lim t →∞ ˙ µ ( t ) /µ ( t ) <ρ .Sinceˆ x ( t ) grows at less than an exponential rate, we also have lim t →∞ exp ( − gt )ˆ x ( t )=0 for any g> 0, and in particular for g = ρ − lim t →∞ ˙ µ ( t ) /µ ( t ). Consequently, asymptotically µ ( t)ˆ x ( t ) grows at a rate low er than ρ and we again obtain (7.55). Finally, suppose that lim t →∞ ˙ x ( t ) >g ˆ x ( t )forany g< ∞ , i.e., ˆ x ( t )growsat more than an exponential rate .Inthiscase,forany g> 0, we have that lim t →∞ exp( − ρt) µ ( t ) ˙ x ( t ) ≥ g lim t →∞ exp( − ρt) µ ( t)ˆ x ( t ) ≥ gκ lim t →∞ ˆ x ( t ) ≥ 0 , 347 Intr oduction to Modern Economic Growth where the first inequalit y exploits the fact that lim t →∞ ˙ x ( t ) >g ˆ x ( t ) and the second, the fact that λ ( t ) ≡ exp(− ρt) µ ( t ) → λ and that ˆ x ( t ) is increasing. But from (7.56), lim t →∞ exp( − ρt) µ ( t ) ˙ x ( t ) = 0, so that all the inequalities in this expression mu st hold as equality, and thu s (7.55) must be satis fied, completing the proof of the theorem. ¤ The proof of Theorem 7.14 also clari fies the importance of discounting. Without discounting the key equation, (7.56), is not necessarily true, and the rest of the proof does not go through. Theorem 7.14 is the most important re sult of this c h apter and will be used in almost all continuous time optimizat ions problems in this book. Throughout, when we refer to a discounted in finite-horizon optimal control problem, we mean a problem that satis fies all the assumptions in Theorem 7.14, including the weak monotonicity assumptions on f and g . Consequently, for our canonical in finite- horizon optimal control problems the stro nger transversality condition (7.55) will be necessary. Notice that compared to the transversality condition in the finite- horizon case (e.g., Theorem 7.1), there is the additional term exp ( − ρt ). This is be- cause the transversality condition appl ies to the original costate variable λ ( t), i.e., lim t →∞ [ x ( t ) λ ( t)] = 0, and as sho wn above, the current-value costate variable µ ( t ) is given by µ ( t)=exp( ρt ) λ ( t). Note also that the stronger transversality condition takes the form lim t →∞ [exp ( − ρt) µ ( t)ˆ x ( t )] = 0, not simply lim t →∞ [exp ( − ρt) µ ( t)] = 0. Exercise 7.17 illustrates why this is. The su fficiency theorems can also be strengthened now by incorporating the transversality condition (7.55) and expressing the conditions in terms of the current- value Hamiltonian: Theorem 7.15. (Mangasarian Suffi cient Conditions for Discounted In finite-Horizon Problems) Consider the problem of maximizing (7.46) subject to (7.47) and (7.48), with f and g continuously di fferentiable and weakly monotone. De fine ˆ H (x, y, µ) as the current-value Hamiltonian as in (7.50), and suppose that a solution ˆ y ( t ) and the corresponding path of state variable x ( t ) satisfy (7.51)-(7.53) and (7.55). Suppose also that lim t →∞ V (t, ˆ x ( t )) exists and that for the resulting current-value costate variable µ ( t ) , ˆ H (x, y, µ) is jointly concave in ( x, y ) for all 348 Intr oduction to Modern Economic Growth t ∈ R + , then ˆ y ( t ) and the corresponding ˆ x ( t ) achieve the unique global maximum of (7.46). Theorem 7.16. (Arrow Su fficient Conditions for Discounted In finite- Horizon Problems) Consider the problem of maximizing (7.46) subject to (7.47) and (7.48), with f and g continuously di fferentiable and weakly monotone. De fine ˆ H (x, y, µ) as the current-value Hamiltonian as in (7.50), and suppose that a solu- tion ˆ y ( t ) and the corresponding path of state variable x ( t ) satisfy (7.51)-(7.53) and which leads to (7.55). Given the resul ting current-value costate variable µ ( t ) ,de fi ne M (t, x, µ) ≡ ˆ H (x, ˆ y, µ ) .Supposethat lim t →∞ V (t, ˆ x ( t )) exists and that M (t, x, µ) is concave in x. Then ˆ y ( t ) and the corresponding ˆ x ( t ) achieve the unique global maximum of (7.46). The proofs of these two theorems are again omitted and left as exercises (see Exercise 7.12). We next pro vide a simple example of discounted in finite-horizon optimal control. Example 7.3. One of the most common examples of this type of dynamic opti- mization problem is that of the optimal t ime path of consuming a non-renewable resource. In particular, imagine the problem of an in finitely-lived individual that has access to a non-renewable or exhaustibl e resource of size 1. The instantaneous utility of consuming a flow of resources y is u ( y), where u :[0 , 1] → R is a strictly increasing, con tinuously di fferentiable and strictly concave function. The individual discounts the future exponentially with discount rate ρ>0, so that his objective function at time t = 0 is to maximize Z ∞ 0 exp ( − ρt) u ( y ( t)) dt. The constraint is that the remaining size of the resource at time t , x ( t)evolves according to ˙ x ( t )=− y ( t ) , which captures the fact that the resource is not renewable and becomes depleted as more of it is consumed. Naturally, we also need that x ( t ) ≥ 0. The current-value Hamiltonian takes the form ˆ H ( x ( t ) ,y( t ) ,µ( t )) = u ( y ( t )) − µ ( t ) y ( t ) . 349 Intr oduction to Modern Economic Growth Theorem 7.14 implies the following necessary condition for an interior continuously di fferentiable solution (ˆ x ( t ) , ˆ y ( t )) to this problem. There should exist a continu- ously di fferentiable function µ ( t) such that u 0 (ˆ y ( t )) = µ ( t ) , and ˙ µ ( t )=ρµ ( t ) . The second condition follows since neither the constraint nor the objectiv e function depend on x ( t). This is the famous Hotelling rule for the exploitation of exhaustible resources. It charts a path for the shado w value of the exhaustible resource. In particular, integrating both sides of this eq uation and using the boundary condition, we obtain that µ ( t )= µ (0) exp (ρt) . Now combining this with the first-order condition for y ( t), we obtain ˆ y ( t )= u 0 − 1 [ µ (0) exp ( ρt)] , where u 0 − 1 [ ·] is the inverse function of u 0 , which exists and is strictly decreasing by virtue of the fact that u is strictly concave. This eq uation immediately implies that the amount of the resource consumed is monotonically decreasing over time. This is economically intuitive: because of discounting, there is preference for early consumption, whereas delayed consumption has no return (there is no production or interest payments on the stock). Nevertheless, the entire resource is not consumed immediately, since there is also a prefere nce for smooth consumption arising from the fact that u ( ·) is strictly concave. Combining the previous equation wi th the resource constraint gives ˙ x ( t )=− u 0 − 1 [ µ (0) exp ( ρt)] . Integrating this equation and using the boundary condition that x (0) = 1, we obtain ˆ x ( t )=1 − Z t 0 u 0 − 1 [ µ (0) exp ( ρs)] ds. Since along any optimal path we must have lim t →∞ ˆ x ( t ) = 0, we have that Z ∞ 0 u 0 − 1 [ µ (0) exp ( ρs)] ds =1. 350 Intr oduction to Modern Economic Growth Therefore, the initial value of the costate variable µ (0) must be chosen so as to satisfy this equation. Notice also that in this proble m both the objective function, u ( y ( t)), and the constraint function, − y ( t ), are weakly monotone in the state and the control vari- ables, so the stronger form of the transvers ality condition, (7.55), holds. You are asked to verify that this condition is satis fied in Exercise 7.20. 7.6. A First Look at Optimal Growth in Continu ous Time In this section, we brie fly show that the main theorems developed so far apply to the problem of optimal growth, which was introduced in Chapter 5 and then an- alyzed in discrete time in the previous chapte r. We will not provide a full treatment of this model here, since this is the topic of the next chapter. Consider the neoclassical economy without any population growth and without any tec hnological progress. In this case, t he optimal gro wth problem in continuous time can be written as: max [ k ( t ) ,c ( t)] ∞ t =0 Z ∞ 0 exp ( − ρt) u ( c ( t)) dt, subject to ˙ k ( t )= f ( k ( t )) − δk ( t ) − c ( t ) and k (0) > 0. Recall that u : R + → R is strictly increasing, continu ously di ff eren- tiable and strictly concave, while f ( ·)satis fi es our basic assumptions, Assumptions 1 and 2. Clearly, the objective function u ( c) is weakly monotone. The constraint function, f ( k ) − δk − c, is decreasing in c,butmaybenonmonotonein k .How- ever, without loss of any generality we can restrict attention to k ( t ) ∈ £ 0 , ¯ k ¤ ,where ¯ k is defi nedsuchthat f 0 ¡ ¯ k ¢ = δ. Increasing the capital stock above this level would reduce output and thus consumption both today and in the future. When k ( t ) ∈ £ 0 , ¯ k ¤ , the constraint function is also weakly monotone in k and we can apply Theorem 7.14. Let us first set up the current-value Hamiltonian, which, in this case, takes the form (7.58) ˆ H (k, c,µ )= u ( c ( t)) + µ ( t)[ f ( k ( t )) − δk ( t ) − c ( t )] , 351 Intr oduction to Modern Economic Growth with state variable k, control variable c and current-value costate variable µ . From Theorem 7.14, the follo wing are the necessary conditions: ˆ H c ( k, c, µ )=u 0 ( c ( t)) − µ ( t)=0 ˆ H k ( k, c, µ )=µ ( t )( f 0 ( k ( t)) − δ )= ρµ ( t ) − ˙ µ ( t ) lim t →∞ [exp ( − ρt) µ ( t ) k ( t)] = 0. Moreover, the first necessary condition i mmediately implies that µ ( t ) > 0(since u 0 > 0 everywhere). Consequently, the current-value Hamiltonian given in (7.58) consists of the sum of two strictly concave functions and is itself strictly concave and thus satis fies the conditions of Theorem 7.15. Therefore, a solution that satis fies these necessary conditions in fact give s a global maximum. Characterizing the solution of these necessary conditions also establishes the existence of a solution in this case. Since an analysis of optimal growth in the neoclassical model is more relevant inthecontextofthenextchapter,wedonotprovidefurtherdetailshere. 7.7. The q-Theory of Inv estment As another application of the methods developed in this chapter, we consider the canonical model of investment under adjustment costs, also known as the q-theory of investment. This problem is not only useful as an application of optimal control techniques, but it is one of the basic mod els of standard macroeconomic theory. The economic problem is that of a price-taking firm trying to maximize the present discounted value of its pro fits. The only twist relative to the problems we ha ve studied so far is that this firm is subject to “adjustment” costs when it chang es its capital stoc k. In particular, let the capital stock of the firm be k ( t)andsuppose that the firm has access to a production function f ( k ( t)) that satis fies Assumptions 1 and 2. For simplicity, let us normalize the price of the output of the firm to 1 in terms of the finalgoodatalldates. The firm is subject to adjustment costs captured by the function φ ( i), which is strictly increasing, continuously di fferentiable and strictly con vex, and satis fies φ (0) = φ 0 (0) = 0. This implies that in addition to the cost of purchasing inv estment goods (whic h given the normalization of price is equal to i foranamountofinvestment i), the firm incurs a cost of adjusting its production 352 Intr oduction to Modern Economic Growth structure given by the convex function φ ( i). In some models, the adjustment cost is tak en to be a function of investment relative to capital, i.e., φ (i/k ) instead of φ ( i ), but this makes no diff erence for our main focus. We also assume that installed capital depreciates at an exponential rate δ and that the firm maximizes its net present discounted earnings with a discount rate equal to the interest rate r,which is assumed to be constant. The firm’s problem can be written as max k ( t ),i( t ) Z ∞ 0 exp ( − rt )[ f ( k ( t )) − i ( t ) − φ ( i ( t ))] dt subject to (7.59) ˙ k ( t )= i ( t ) − δk ( t ) and k ( t ) ≥ 0, with k (0) > 0 given. Clearly, both the objective function and the constraint function are weakly monotone, th us we can apply Theorem 7.14. Notice that φ ( i) does not contribute to capital accumulation; it is simply a cost. Moreover, since φ is strictly convex, it implies that it is not optimal for the firm to make “large” adjustments. Therefore it will act as a force towards a smoother time path of investment. To char acterize the optimal investmen t plan of the fi rm, let us write the current- value Hamiltonian: ˆ H (k, i,q) ≡ [ f ( k ( t )) − i ( t ) − φ ( i ( t ))] + q ( t )[ i ( t ) − δk ( t)] , whereweused q ( t) instead of the familiar µ ( t) for the costate variable, for reasons that will be apparent soon. The necessary conditions for this problem are standa rd (suppressing the “ˆ” to denote the optimal values in order to reduce notation): ˆ H i ( k, i, q )= − 1 − φ 0 ( i ( t)) + q ( t)=0 ˆ H k ( k, i, q )=f 0 ( k ( t)) − δq ( t)= rq ( t ) − ˙ q ( t ) lim t →∞ exp ( − rt ) q ( t ) k ( t)=0 . The first necessary condition implies that (7.60) q ( t)=1+ φ 0 ( i ( t)) for all t . 353 Intr oduction to Modern Economic Growth Di fferentiating this equation with respect to time, w e obtain (7.61) ˙ q ( t )= φ 00 ( i ( t)) ˙ i ( t ) . Substituting this into the second necessa ry condition, we obtain the following la w of motion for investment: (7.62) ˙ i ( t )= 1 φ 00 ( i ( t)) [( r + δ)(1+ φ 0 ( i ( t))) − f 0 ( k ( t))] . A number of in teresting economic features emerge from this equation. First, as φ 00 ( i ) tends to zero, it can be veri fied that ˙ i ( t ) diverges, meaning that investment jumps to a particular value. In other words, it can be shown that this value is such that the capital stock immediately reaches its state-state value (see Exercise 7.22). This is intuitive. As φ 00 ( i) tends to zero, φ 0 ( i) becomes linear. In this case, adjustment costs simply increase the cost of investment linearly and do not create any need for smoothing. In contrast, when φ 00 ( i ( t)) > 0,therewillbeamotiveforsmoothing, ˙ i ( t ) will take a finite value, and investment will adjust slowly. Therefore, as claimed above, adjustment costs lead to a smoother path of investment. We can now analyze the behavior of investme nt and capital stoc k using the di fferential equations (7.59) and (7.62). First, it can be veri fied easily that there exists a unique steady-state solution with k> 0. This solution involves a level of capital stock k ∗ for the firm and investment just enough to replenish the depreciated capital, i ∗ = δk ∗ . This steady-state lev el of capital satis fies the first-order condition (corresponding to the righ t hand side of (7.62) being equal to zero): f 0 ( k ∗ )=( r + δ )(1+ φ 0 ( δk ∗ )) . This first-order condition di ff ers from the standard “modi fied golden rule” condition, which requires the marginal product of capital to be equal to the in terest rate plus the depreciation rate, because an additio nal cost of having a higher capital stock is that there will have to be more investment to replenish depreciated capital. This is captured by the term φ 0 ( δk ∗ ). Since φ is strictly convex and f is strictly concave and satis fies the Inada conditions (f rom Assumption 2), there exists a unique value of k ∗ that satis fies this condition. 354 Intr oduction to Modern Economic Growth The analysis of dynamics in this case requires somewhat di fferent ideas than those used in the basic Solow growth model (cf., Theorems 2.4 and 2.5). In par- ticular, instead of global stability in the k - i space, the correct concept is one of saddle-path stability . The reason for this is that instead of an initial value con- straint, i (0) is pinned down by a boundary condition at “in finity,” that is, to satisfy the transversality condition, lim t →∞ exp ( − rt ) q ( t ) k ( t)=0 . This implies that in the context of the curre nt theory, with one state and one control variable, we should have a one-dimensiona l manifold (a curve) along which capital- inve stment pairs tend towards the steady s tate. This manifold is also referred to as the “stable arm”. The initial value of investment, i (0), will then be determined so that the econom y starts along this manif old. In fact, if any capital-investment pair (rather than only pairs along this one dimensional manifold) were to lead to the steady state, we would not know how to determine i (0); in other words, there would be an “indeterminacy” of equilibria. Mathematically, rather than requiring all eigenvalues of the linearized system to be negative, what we require now is saddle- path stability, which involves the number of negative eigen values to be the same as the number of state variables. This notion of saddle path stability will be central in most of growth models we will study. Let this now make these notions more precise by considering the following generalizatio ns of Theorems 2.4 and 2.5: Theorem 7.17. Consider the following linear di fferential equation system (7.63) ˙ x ( t)= Ax ( t)+ b with initial value x (0) ,where x ( t ) ∈ R n for all t and A is an n × n matrix. Let x ∗ be the steady state of the system given by Ax ∗ + b =0 . Suppose that m ≤ n of the eigenvalues of A have negative real parts. Then there exists an m-dimensional manifold M of R n such that starting from any x (0) ∈ M
