Undergraduate Texts in Mathematics
Sheldon Axler
Linear Algebra
Done Right
Third Edition
Undergraduate Texts in Mathematics
Undergraduate Texts in Mathematics
Series Editors:
Sheldon Axler
San Francisco State University, San Francisco, CA, USA
Kenneth Ribet
University of California, Berkeley, CA, USA
Advisory Board:
Colin Adams, Williams College, Williamstown, MA, USA
Alejandro Adem, University of British Columbia, Vancouver, BC, Canada Ruth Charney, Brandeis University, Waltham, MA, USA
Irene M. Gamba, The University of Texas at Austin, Austin, TX, USA
Roger E. Howe, Yale University, New Haven, CT, USA
David Jerison, Massachusetts Institute of Technology, Cambridge, MA, USA Jeffrey C. Lagarias, University of Michigan, Ann Arbor, MI, USA
Jill Pipher, Brown University, Providence, RI, USA
Fadil Santosa, University of Minnesota, Minneapolis, MN, USA
Amie Wilkinson, University of Chicago, Chicago, IL, USA
Undergraduate Texts in Mathematics are generally aimed at third- and fourth- year undergraduate mathematics students at North American universities. These texts strive to provide students and teachers with new perspectives and novel approaches. The books include motivation that guides the reader to an appreciation of interrelations among different aspects of the subject. They feature examples that illustrate key concepts as well as exercises that strengthen understanding.
For further volumes:
http://www.springer.com/series/666
Sheldon Axler
Linear Algebra Done Right Third edition
123
Sheldon Axler
Department of Mathematics San Francisco State University San Francisco, CA, USA
ISSN 0172-6056
ISBN 978-3-319-11079-0
DOI 10.1007/978-3-319-11080-6
Springer Cham Heidelberg New York Dordrecht London
ISSN 2197-5604 (electronic) ISBN 978-3-319-11080-6 (eBook)
Library of Congress Control Number: 2014954079
Mathematics Subject Classification (2010): 15-01, 15A03, 15A04, 15A15, 15A18, 15A21
⃝c Springer International Publishing 2015
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
Typeset by the author in LaTeX
Cover figure: For a statement of Apollonius’s Identity and its connection to linear algebra, see the last exercise in Section 6.A.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
Contents
Preface for the Instructor xi Preface for the Student xv Acknowledgments xvii
1 Vector Spaces 1
1.A Rn and Cn 2 Complex Numbers
2
Lists 5
Fn 6
Digression on Fields Exercises 1.A 11
10
1.B Definition of Vector Space 12
Exercises 1.B 17
1.C Subspaces 18
Sums of Subspaces 20
Direct Sums 21 Exercises 1.C 24
2 Finite-Dimensional Vector Spaces 27
2.A Span and Linear Independence 28 Linear Combinations and Span 28
Linear Independence 32 Exercises 2.A 37
v
vi Contents
2.B Bases 39
Exercises 2.B 43 2.C Dimension 44
Exercises 2.C 48 3 Linear Maps 51
3.A The Vector Space of Linear Maps 52 Definition and Examples of Linear Maps 52
Algebraic Operations on L.V; W / 55 Exercises 3.A 57
3.B Null Spaces and Ranges 59 Null Space and Injectivity 59
Range and Surjectivity 61
Fundamental Theorem of Linear Maps 63 Exercises 3.B 67
3.C Matrices 70
Representing a Linear Map by a Matrix 70
Addition and Scalar Multiplication of Matrices 72 Matrix Multiplication 74
Exercises 3.C 78
3.D Invertibility and Isomorphic Vector Spaces 80 Invertible Linear Maps 80
Isomorphic Vector Spaces 82
Linear Maps Thought of as Matrix Multiplication 84 Operators 86
Exercises 3.D 88
3.E Products and Quotients of Vector Spaces 91 Products of Vector Spaces 91
Products and Direct Sums 93 Quotients of Vector Spaces 94 Exercises 3.E 98
3.F Duality 101
The Dual Space and the Dual Map 101
The Null Space and Range of the Dual of a Linear Map The Matrix of the Dual of a Linear Map 109
The Rank of a Matrix 111
Exercises 3.F 113
4 Polynomials 117
Complex Conjugate and Absolute Value 118
Uniqueness of Coefficients for Polynomials 120 The Division Algorithm for Polynomials 121 Zeros of Polynomials 122
Factorization of Polynomials over C 123 Factorization of Polynomials over R 126 Exercises 4 129
5 Eigenvalues, Eigenvectors, and Invariant Subspaces 131
5.A Invariant Subspaces 132 Eigenvalues and Eigenvectors 133
Restriction and Quotient Operators 137 Exercises 5.A 138
5.B Eigenvectors and Upper-Triangular Matrices 143 Polynomials Applied to Operators 143
Existence of Eigenvalues 145 Upper-Triangular Matrices 146 Exercises 5.B 153
5.C Eigenspaces and Diagonal Matrices 155 Exercises 5.C 160
6 Inner Product Spaces 163
6.A Inner Products and Norms 164 Inner Products 164
Norms 168 Exercises 6.A 175
104
Contents
vii
viii Contents
6.B Orthonormal Bases 180
Linear Functionals on Inner Product Spaces 187 Exercises 6.B 189
6.C Orthogonal Complements and Minimization Problems 193 Orthogonal Complements 193
Minimization Problems 198 Exercises 6.C 201
7 Operators on Inner Product Spaces 203
7.A Self-Adjoint and Normal Operators 204 Adjoints 204
Self-Adjoint Operators 209 Normal Operators 212 Exercises 7.A 214
7.B The Spectral Theorem 217
The Complex Spectral Theorem 217
The Real Spectral Theorem 219 Exercises 7.B 223
7.C Positive Operators and Isometries 225 Positive Operators 225
Isometries 228 Exercises 7.C 231
7.D Polar Decomposition and Singular Value Decomposition 233 Polar Decomposition 233
Singular Value Decomposition 236 Exercises 7.D 238
8 Operators on Complex Vector Spaces 241
8.A Generalized Eigenvectors and Nilpotent Operators 242 Null Spaces of Powers of an Operator 242
Generalized Eigenvectors 244 Nilpotent Operators 248 Exercises 8.A 249
8.B Decomposition of an Operator 252
Description of Operators on Complex Vector Spaces 252
Multiplicity of an Eigenvalue 254 Block Diagonal Matrices 255 Square Roots 258
Exercises 8.B 259
8.C Characteristic and Minimal Polynomials 261 The Cayley–Hamilton Theorem 261
The Minimal Polynomial 262 Exercises 8.C 267
8.D Jordan Form 270 Exercises 8.D 274
9 Operators on Real Vector Spaces 275
9.A Complexification 276
Complexification of a Vector Space 276
Complexification of an Operator 277
The Minimal Polynomial of the Complexification 279 Eigenvalues of the Complexification 280 Characteristic Polynomial of the Complexification 283 Exercises 9.A 285
9.B Operators on Real Inner Product Spaces 287 Normal Operators on Real Inner Product Spaces 287
Isometries on Real Inner Product Spaces 292 Exercises 9.B 294
10 Trace and Determinant 295
10.A Trace 296
Change of Basis 296
Trace: A Connection Between Operators and Matrices 299 Exercises 10.A 304
Contents ix
x
Contents
10.B Determinant 307
Determinant of an Operator 307
Determinant of a Matrix 309 The Sign of the Determinant 320 Volume 323
Exercises 10.B 330
Photo Credits 333 Symbol Index 335 Index 337
Preface for the Instructor
You are about to teach a course that will probably give students their second exposure to linear algebra. During their first brush with the subject, your students probably worked with Euclidean spaces and matrices. In contrast, this course will emphasize abstract vector spaces and linear maps.
The audacious title of this book deserves an explanation. Almost all linear algebra books use determinants to prove that every linear operator on a finite-dimensional complex vector space has an eigenvalue. Determinants are difficult, nonintuitive, and often defined without motivation. To prove the theorem about existence of eigenvalues on complex vector spaces, most books must define determinants, prove that a linear map is not invertible if and only if its determinant equals 0, and then define the characteristic polynomial. This tortuous (torturous?) path gives students little feeling for why eigenvalues exist.
In contrast, the simple determinant-free proofs presented here (for example, see 5.21) offer more insight. Once determinants have been banished to the end of the book, a new route opens to the main goal of linear algebra— understanding the structure of linear operators.
This book starts at the beginning of the subject, with no prerequisites other than the usual demand for suitable mathematical maturity. Even if your students have already seen some of the material in the first few chapters, they may be unaccustomed to working exercises of the type presented here, most of which require an understanding of proofs.
Here is a chapter-by-chapter summary of the highlights of the book:
Chapter 1: Vector spaces are defined in this chapter, and their basic proper- ties are developed.
Chapter 2: Linear independence, span, basis, and dimension are defined in this chapter, which presents the basic theory of finite-dimensional vector spaces.
xi
xii Preface for the Instructor
Chapter 3: Linear maps are introduced in this chapter. The key result here is the Fundamental Theorem of Linear Maps (3.22): if T is a linear map on V, then dim V D dim null T C dim range T. Quotient spaces and duality are topics in this chapter at a higher level of abstraction than other parts of the book; these topics can be skipped without running into problems elsewhere in the book.
Chapter 4: The part of the theory of polynomials that will be needed to understand linear operators is presented in this chapter. This chapter contains no linear algebra. It can be covered quickly, especially if your students are already familiar with these results.
Chapter 5: The idea of studying a linear operator by restricting it to small subspaces leads to eigenvectors in the early part of this chapter. The highlight of this chapter is a simple proof that on complex vector spaces, eigenvalues always exist. This result is then used to show that each linear operator on a complex vector space has an upper-triangular matrix with respect to some basis. All this is done without defining determinants or characteristic polynomials!
Chapter 6: Inner product spaces are defined in this chapter, and their basic properties are developed along with standard tools such as orthonormal bases and the Gram–Schmidt Procedure. This chapter also shows how orthogonal projections can be used to solve certain minimization problems.
Chapter 7: The Spectral Theorem, which characterizes the linear operators for which there exists an orthonormal basis consisting of eigenvectors, is the highlight of this chapter. The work in earlier chapters pays off here with especially simple proofs. This chapter also deals with positive operators, isometries, the Polar Decomposition, and the Singular Value Decomposition.
Chapter 8: Minimal polynomials, characteristic polynomials, and gener- alized eigenvectors are introduced in this chapter. The main achievement of this chapter is the description of a linear operator on a complex vector space in terms of its generalized eigenvectors. This description enables one to prove many of the results usually proved using Jordan Form. For example, these tools are used to prove that every invertible linear operator on a complex vector space has a square root. The chapter concludes with a proof that every linear operator on a complex vector space can be put into Jordan Form.
Chapter 9: Linear operators on real vector spaces occupy center stage in this chapter. Here the main technique is complexification, which is a natural extension of an operator on a real vector space to an operator on a complex vector space. Complexification allows our results about complex vector spaces to be transferred easily to real vector spaces. For example, this technique is used to show that every linear operator on a real vector space has an invariant subspace of dimension 1 or 2. As another example, we show that that every linear operator on an odd-dimensional real vector space has an eigenvalue.
Chapter 10: The trace and determinant (on complex vector spaces) are defined in this chapter as the sum of the eigenvalues and the product of the eigenvalues, both counting multiplicity. These easy-to-remember defini- tions would not be possible with the traditional approach to eigenvalues, because the traditional method uses determinants to prove that sufficient eigenvalues exist. The standard theorems about determinants now become much clearer. The Polar Decomposition and the Real Spectral Theorem are used to derive the change of variables formula for multivariable integrals in a fashion that makes the appearance of the determinant there seem natural.
This book usually develops linear algebra simultaneously for real and complex vector spaces by letting F denote either the real or the complex numbers. If you and your students prefer to think of F as an arbitrary field, then see the comments at the end of Section 1.A. I prefer avoiding arbitrary fields at this level because they introduce extra abstraction without leading to any new linear algebra. Also, students are more comfortable thinking of polynomials as functions instead of the more formal objects needed for polynomials with coefficients in finite fields. Finally, even if the beginning part of the theory were developed with arbitrary fields, inner product spaces would push consideration back to just real and complex vector spaces.
You probably cannot cover everything in this book in one semester. Going through the first eight chapters is a good goal for a one-semester course. If you must reach Chapter 10, then consider covering Chapters 4 and 9 in fifteen minutes each, as well as skipping the material on quotient spaces and duality in Chapter 3.
A goal more important than teaching any particular theorem is to develop in students the ability to understand and manipulate the objects of linear algebra. Mathematics can be learned only by doing. Fortunately, linear algebra has many good homework exercises. When teaching this course, during each class I usually assign as homework several of the exercises, due the next class. Going over the homework might take up a third or even half of a typical class.
Preface for the Instructor xiii
xiv Preface for the Instructor
Major changes from the previous edition:
This edition contains 561 exercises, including 337 new exercises that were not in the previous edition. Exercises now appear at the end of each section, rather than at the end of each chapter.
Many new examples have been added to illustrate the key ideas of linear algebra.
Beautiful new formatting, including the use of color, creates pages with an unusually pleasant appearance in both print and electronic versions. As a visual aid, definitions are in beige boxes and theorems are in blue boxes (in color versions of the book).
Each theorem now has a descriptive name.
New topics covered in the book include product spaces, quotient spaces,
and duality.
Chapter 9 (Operators on Real Vector Spaces) has been completely rewritten to take advantage of simplifications via complexification. This approach allows for more streamlined presentations in Chapters 5 and 7 because those chapters now focus mostly on complex vector spaces.
Hundreds of improvements have been made throughout the book. For example, the proof of Jordan Form (Section 8.D) has been simplified.
Please check the website below for additional information about the book. I may occasionally write new sections on additional topics. These new sections will be posted on the website. Your suggestions, comments, and corrections are most welcome.
Best wishes for teaching a successful linear algebra class!
Sheldon Axler
Mathematics Department
San Francisco State University San Francisco, CA 94132, USA
website: linear.axler.net e-mail: linear@axler.net Twitter: @AxlerLinear
Preface for the Student
You are probably about to begin your second exposure to linear algebra. Unlike your first brush with the subject, which probably emphasized Euclidean spaces and matrices, this encounter will focus on abstract vector spaces and linear maps. These terms will be defined later, so don’t worry if you do not know what they mean. This book starts from the beginning of the subject, assuming no knowledge of linear algebra. The key point is that you are about to immerse yourself in serious mathematics, with an emphasis on attaining a deep understanding of the definitions, theorems, and proofs.
You cannot read mathematics the way you read a novel. If you zip through a page in less than an hour, you are probably going too fast. When you encounter the phrase “as you should verify”, you should indeed do the verification, which will usually require some writing on your part. When steps are left out, you need to supply the missing pieces. You should ponder and internalize each definition. For each theorem, you should seek examples to show why each hypothesis is necessary. Discussions with other students should help.
As a visual aid, definitions are in beige boxes and theorems are in blue boxes (in color versions of the book). Each theorem has a descriptive name.
Please check the website below for additional information about the book. I may occasionally write new sections on additional topics. These new sections will be posted on the website. Your suggestions, comments, and corrections are most welcome.
Best wishes for success and enjoyment in learning linear algebra!
Sheldon Axler
Mathematics Department
San Francisco State University San Francisco, CA 94132, USA
website: linear.axler.net e-mail: linear@axler.net Twitter: @AxlerLinear
xv
Acknowledgments
I owe a huge intellectual debt to the many mathematicians who created linear algebra over the past two centuries. The results in this book belong to the common heritage of mathematics. A special case of a theorem may first have been proved in the nineteenth century, then slowly sharpened and improved by many mathematicians. Bestowing proper credit on all the contributors would be a difficult task that I have not undertaken. In no case should the reader assume that any theorem presented here represents my original contribution. However, in writing this book I tried to think about the best way to present lin- ear algebra and to prove its theorems, without regard to the standard methods and proofs used in most textbooks.
Many people helped make this a better book. The two previous editions of this book were used as a textbook at about 300 universities and colleges. I received thousands of suggestions and comments from faculty and students who used the second edition. I looked carefully at all those suggestions as I was working on this edition. At first, I tried keeping track of whose suggestions I used so that those people could be thanked here. But as changes were made and then replaced with better suggestions, and as the list grew longer, keeping track of the sources of each suggestion became too complicated. And lists are boring to read anyway. Thus in lieu of a long list of people who contributed good ideas, I will just say how truly grateful I am to everyone who sent me suggestions and comments. Many many thanks!
Special thanks to Ken Ribet and his giant (220 students) linear algebra class at Berkeley that class-tested a preliminary version of this third edition and that sent me more suggestions and corrections than any other group.
Finally, I thank Springer for providing me with help when I needed it and for allowing me the freedom to make the final decisions about the content and appearance of this book. Special thanks to Elizabeth Loew for her wonderful work as editor and David Kramer for unusually skillful copyediting.
Sheldon Axler
xvii
CHAPTER
1
Vector Spaces
Linear algebra is the study of linear maps on finite-dimensional vector spaces. Eventually we will learn what all these terms mean. In this chapter we will define vector spaces and discuss their elementary properties.
In linear algebra, better theorems and more insight emerge if complex numbers are investigated along with real numbers. Thus we will begin by introducing the complex numbers and their basic properties.
We will generalize the examples of a plane and ordinary space to Rn and Cn, which we then will generalize to the notion of a vector space. The elementary properties of a vector space will already seem familiar to you.
Then our next topic will be subspaces, which play a role for vector spaces analogous to the role played by subsets for sets. Finally, we will look at sums of subspaces (analogous to unions of subsets) and direct sums of subspaces (analogous to unions of disjoint sets).
René Descartes explaining his work to Queen Christina of Sweden. Vector spaces are a generalization of the description of a plane using two coordinates, as published by Descartes in 1637.
LEARNING OBJECTIVES FOR THIS CHAPTER basic properties of the complex numbers
Rn and Cn
vector spaces
subspaces
sums and direct sums of subspaces
© Springer International Publishing 2015 1 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__1
2 CHAPTER 1 Vector Spaces 1.A Rn and Cn
Complex Numbers
You should already be familiar with basic properties of the set R of real numbers. Complex numbers were invented so that we can take square roots of negative numbers. The idea is to assume we have a square root of 1, denoted i, that obeys the usual rules of arithmetic. Here are the formal definitions:
1.1
Definition complex numbers
A complex number is an ordered pair .a; b/, where a; b 2 R, but
we will write this as a C bi.
The set of all complex numbers is denoted by C:
C D fa C bi W a; b 2 Rg:
Addition and multiplication on C are defined by
.a C bi/ C .c C di/ D .a C c/ C .b C d/i; .aCbi/.cCdi/D.acbd/C.ad Cbc/iI
here a;b;c;d 2 R.
If a 2 R, we identify a C 0i with the real number a. Thus we can think of R as a subset of C. We also usually write 0 C bi as just bi, and we usually write 0 C 1i as just i.
The symbol i was first used to de- p
note 1 by Swiss mathematician Leonhard Euler in 1777.
1.2 Example
Using multiplication as defined above, you should verify that i2 D 1. Do not memorize the formula for the product of two complex numbers; you can always rederive it by recalling that i2 D 1 and then using the usual rules of arithmetic (as given by 1.3).
Evaluate .2 C 3i/.4 C 5i/.
Solution .2 C 3i/.4 C 5i/ D 2 4 C 2 .5i/ C .3i/ 4 C .3i/.5i/ D 8 C 10i C 12i 15
D 7 C 22i
SECTION 1.A Rn and Cn 3
1.3 Properties of complex arithmetic
commutativity
̨ C ˇ D ˇ C ̨ and ̨ˇ D ˇ ̨ for all ̨; ˇ 2 C;
associativity
. ̨Cˇ/C D ̨C.ˇC/ and . ̨ˇ/ D ̨.ˇ/ for all ̨;ˇ; 2 C;
identities
C 0 D and 1 D for all 2 C;
additive inverse
for every ̨ 2 C, there exists a unique ˇ 2 C such that ̨ C ˇ D 0;
multiplicative inverse
for every ̨ 2 C with ̨ ¤ 0, there exists a unique ˇ 2 C such that ̨ˇ D 1;
distributive property
. ̨ C ˇ/ D ̨ C ˇ for all ; ̨; ˇ 2 C.
The properties above are proved using the familiar properties of real numbers and the definitions of complex addition and multiplication. The next example shows how commutativity of complex multiplication is proved. Proofs of the other properties above are left as exercises.
1.4 Example Show that ̨ˇ D ˇ ̨ for all ̨;ˇ; 2 C.
Solution Suppose ̨ D a C bi and ˇ D c C d i , where a; b; c; d 2 R. Then
the definition of multiplication of complex numbers shows that
and
̨ˇ D .a C bi/.c C di/
D .ac bd/ C .ad C bc/i
ˇ ̨ D .c C di/.a C bi/
D .ca db/ C .cb C da/i:
The equations above and the commutativity of multiplication and addition of real numbers show that ̨ˇ D ˇ ̨.
4 CHAPTER 1 Vector Spaces
1.5 Definition ̨, subtraction, 1= ̨, division Let ̨; ˇ 2 C.
Let ̨ denote the additive inverse of ̨. Thus ̨ is the unique complex number such that
̨ C . ̨/ D 0: Subtraction on C is defined by
ˇ ̨ D ˇ C . ̨/:
For ̨ ¤ 0, let 1= ̨ denote the multiplicative inverse of ̨. Thus 1= ̨
is the unique complex number such that
̨.1= ̨/ D 1:
Division on C is defined by
ˇ= ̨ D ˇ.1= ̨/:
So that we can conveniently make definitions and prove theorems that apply to both real and complex numbers, we adopt the following notation:
Thus if we prove a theorem involving F, we will know that it holds when F is replaced with R and when F is replaced with C.
Elements of F are called scalars. The word “scalar”, a fancy word for “number”, is often used when we want to emphasize that an object is a number,
as opposed to a vector (vectors will be defined soon).
For ̨ 2 F and m a positive integer, we define ̨m to denote the product of
1.6 Notation F
Throughout this book, F stands for either R or C.
The letter F is used because R and C are examples of what are called fields.
̨ with itself m times:
̨m D ̨ ̨: „ƒ‚…
m times
Clearly . ̨m/n D ̨mn and . ̨ˇ/m D ̨mˇm for all ̨; ˇ 2 F and all positive integers m; n.
Lists
Before defining Rn and Cn, we look at two important examples.
concept of lists.
SECTION 1.A Rn and Cn 5
1.7
Example R2 and R3
The set R2, which you can think of as a plane, is the set of all ordered
pairs of real numbers:
R2 D f.x;y/ W x;y 2 Rg:
The set R3, which you can think of as ordinary space, is the set of all
ordered triples of real numbers:
R3 D f.x;y;z/ W x;y;z 2 Rg:
To generalize R2 and R3 to higher dimensions, we first need to discuss the
1.8 Definition list, length
Suppose n is a nonnegative integer. A list of length n is an ordered collection of n elements (which might be numbers, other lists, or more abstract entities) separated by commas and surrounded by parentheses. A list of length n looks like this:
.x1;:::;xn/:
Two lists are equal if and only if they have the same length and the same elements in the same order.
Thus a list of length 2 is an ordered pair, and a list of length 3 is an ordered triple.
Sometimes we will use the word list without specifying its length. Re- member, however, that by definition each list has a finite length that is a nonnegative integer. Thus an object that looks like
.x1;x2;:::/;
which might be said to have infinite length, is not a list.
A list of length 0 looks like this: . /. We consider such an object to be a
list so that some of our theorems will not have trivial exceptions.
Lists differ from sets in two ways: in lists, order matters and repetitions
have meaning; in sets, order and repetitions are irrelevant.
Many mathematicians call a list of length n an n-tuple.
6
CHAPTER 1 Vector Spaces
1.9
Example lists versus sets
Thelists.3;5/and.5;3/arenotequal,butthesetsf3;5gandf5;3gare
equal.
The lists .4; 4/ and .4; 4; 4/ are not equal (they do not have the same
length), although the sets f4; 4g and f4; 4; 4g both equal the set f4g. Fn
To define the higher-dimensional analogues of R2 and R3, we will simply replace R with F (which equals R or C) and replace theFana 2 or 3 with an arbitrary positive integer. Specifically, fix a positive integer n for the rest of this section.
1.10 Definition Fn
Fn is the set of all lists of length n of elements of F:
Fn D f.x1;:::;xn/ W xj 2 F for j D 1;:::;ng:
For .x1;:::;xn/ 2 Fn and j 2 f1;:::;ng, we say that xj is the jth coordinate of .x1; : : : ; xn/.
If F D R and n equals 2 or 3, then this definition of Fn agrees with our previous notions of R2 and R3.
1.11 Example C4 is the set of all lists of four complex numbers: C4 D f.z1;z2;z3;z4/ W z1;z2;z3;z4 2 Cg:
If n 4, we cannot visualize Rn as a physical object. Similarly, C1 can be thought of as a plane, but for n 2, the human brain cannot provide a full image of Cn. However, even if n is large, we can perform algebraic manip- ulations in Fn as easily as in R2 or R3. For example, addition in Fn is defined as follows:
For an amusing account of how R3 would be perceived by crea- tures living in R2, read Flatland: A Romance of Many Dimensions, by Edwin A. Abbott. This novel, published in 1884, may help you imagine a physical space of four or more dimensions.
Often the mathematics of Fn becomes cleaner if we use a single letter to denote a list of n numbers, without explicitly writing the coordinates. For example, the result below is stated with x and y in Fn even though the proof requires the more cumbersome notation of .x1; : : : ; xn/ and .y1; : : : ; yn/.
where the second and fourth equalities above hold because of the definition of addition in Fn and the third equality holds because of the usual commutativity of addition in F.
If a single letter is used to denote
an element of Fn, then the same letter
with appropriate subscripts is often used
when coordinates must be displayed. For example, if x 2 Fn, then letting x equal .x1; : : : ; xn/ is good notation, as shown in the proof above. Even better, work with just x and avoid explicit coordinates when possible.
SECTION 1.A Rn and Cn 7
1.12 Definition addition in Fn
Addition in Fn is defined by adding corresponding coordinates:
.x1;:::;xn/C.y1;:::;yn/D.x1 Cy1;:::;xn Cyn/:
1.13 Commutativity of addition in Fn If x; y 2 Fn, then x C y D y C x.
Proof
Suppose x D .x1;:::;xn/ and y D .y1;:::;yn/. Then
x C y D .x1; : : : ; xn/ C .y1; : : : ; yn/ D .x1 C y1; : : : ; xn C yn/
D .y1 C x1; : : : ; yn C xn/
D .y1;:::;yn/ C .x1;:::;xn/ D y C x;
The symbol means “end of the proof ”.
1.14 Definition 0
Let 0 denote the list of length n whose coordinates are all 0:
0 D .0;:::;0/:
8 CHAPTER 1 Vector Spaces
Here we are using the symbol 0 in two different ways—on the left side of the equation in 1.14, the symbol 0 denotes a list of length n, whereas on the right side, each 0 denotes a number. This potentially confusing practice actually causes no problems because the context always makes clear what is intended.
1.15 Example Consider the statement that 0 is an additive identity for Fn: xC0Dx forallx2Fn:
Is the 0 above the number 0 or the list 0?
Solution Here 0 is a list, because we have not defined the sum of an element
of Fn (namely, x) and the number 0.
Elements of R2 can be thought of as points or as vectors.
A picture can aid our intuition. We will draw pictures in R2 because we can sketch this space on 2-dimensional surfaces such as paper and blackboards. A typical element of R2 is a point x D .x1; x2/. Sometimes we think of x not as a point but as an arrow starting at the origin and ending at .x1; x2/, as shown here. When we think of x as an arrow, we refer to it as a vector.
When we think of vectors in R2 as arrows, we can move an arrow parallel to itself (not changing its length or di- rection) and still think of it as the same vector. With that viewpoint, you will often gain better understanding by dis- pensing with the coordinate axes and the explicit coordinates and just think- ing of the vector, as shown here.
Whenever we use pictures in R2 or use the somewhat vague language of points and vectors, remember that these are just aids to our understand- ing, not substitutes for the actual math- ematics that we will develop. Although we cannot draw good pictures in high- dimensional spaces, the elements of these spaces are as rigorously defined as elements of R2.
x1, x2 x
x
x
A vector.
Mathematical models of the econ- omy can have thousands of vari- ables, say x1;:::;x5000, which means that we must operate in R5000 . Such a space cannot be dealt with geometrically. However, the algebraic approach works well. Thus our subject is called linear algebra.
SECTION 1.A Rn and Cn 9
For example, .2; 3; 17; ; p2/ is an element of R5, and we may casually refer to it as a point in R5 or a vector in R5 without worrying about whether the geometry of R5 has any physical meaning.
Recall that we defined the sum of two elements of Fn to be the element of Fn obtained by adding corresponding coordinates; see 1.12. As we will now see, addition has a simple geometric interpretation in the special case of R2.
Suppose we have two vectors x and y in R2 that we want to add. Move the vector y parallel to itself so that its initial point coincides with the end point of the vector x, as shown here. The sum x C y then equals the vector whose initial point equals the initial point of x and whose end point equals the end point of the vector y, as shown here.
The sum of two vectors.
xy
x
y
In the next definition, the 0 on the right side of the displayed equation below is the list 0 2 Fn.
1.16 Definition additive inverse in Fn
For x 2 Fn, the additive inverse of x, denoted x, is the vector x 2 Fn
such that
In other words, if x D .x1;:::;xn/, then x D .x1;:::;xn/.
x C .x/ D 0:
For a vector x 2 R2, the additive in-
verse x is the vector parallel to x and
with the same length as x but pointing in
the opposite direction. The figure here
illustrates this way of thinking about the
additive inverse in R2.
Having dealt with addition in Fn, we A vector and its additive inverse.
now turn to multiplication. We could
define a multiplication in Fn in a similar fashion, starting with two elements of Fn and getting another element of Fn by multiplying corresponding coor- dinates. Experience shows that this definition is not useful for our purposes. Another type of multiplication, called scalar multiplication, will be central to our subject. Specifically, we need to define what it means to multiply an element of Fn by an element of F.
x
x
10
CHAPTER 1
Vector Spaces
1.17 Definition scalar multiplication in Fn
The product of a number and a vector in Fn is computed by multiplying
each coordinate of the vector by :
.x1;:::;xn/ D .x1;:::;xn/I
here 2 F and .x1;:::;xn/ 2 Fn.
Scalar multiplication has a nice ge- ometric interpretation in R2. If is a positive number and x is a vector in R2, then x is the vector that points in the same direction as x and whose length is times the length of x. In other words, to get x, we shrink or stretch x by a factor of , depending on whether < 1 or > 1.
If is a negative number and x is a vector in R2, then x is the vector that points in the direction opposite to that of x and whose length is jj times the length of x, as shown here.
In scalar multiplication, we multi- ply together a scalar and a vector, getting a vector. You may be famil- iar with the dot product in R2 or R3, in which we multiply together two vectors and get a scalar. Gen- eralizations of the dot product will become important when we study inner products in Chapter 6.
x
32 x 12 x
Scalar multiplication.
Digression on Fields
A field is a set containing at least two distinct elements called 0 and 1, along with operations of addition and multiplication satisfying all the properties listed in 1.3. Thus R and C are fields, as is the set of rational numbers along with the usual operations of addition and multiplication. Another example of a field is the set f0; 1g with the usual operations of addition and multiplication except that 1 C 1 is defined to equal 0.
In this book we will not need to deal with fields other than R and C.
However, many of the definitions, theorems, and proofs in linear algebra that
work for both R and C also work without change for arbitrary fields. If you
prefer to do so, throughout Chapters 1, 2, and 3 you can think of F as denoting
an arbitrary field instead of R or C, except that some of the examples and
exercises require that for each positive integer n we have 1 C 1 C C 1 ¤ 0. „ ƒ‚ …
n times
EXERCISES 1.A
2 Show that
1=.a C bi/ D c C di: p
1C 3i 2
SECTION 1.A Rn and Cn 11
1 Suppose a and b are real numbers, not both 0. Find real numbers c and d such that
is a cube root of 1 (meaning that its cube equals 1).
3 Find two distinct square roots of i.
4 Showthat ̨CˇDˇC ̨forall ̨;ˇ2C.
5 Showthat. ̨Cˇ/CD ̨C.ˇC/forall ̨;ˇ;2C.
6 Showthat. ̨ˇ/D ̨.ˇ/forall ̨;ˇ;2C.
7 Show that for every ̨ 2 C, there exists a unique ˇ 2 C such that ̨ C ˇ D 0.
8 Showthatforevery ̨2Cwith ̨¤0,thereexistsauniqueˇ2Csuch that ̨ˇ D 1.
9 Showthat. ̨Cˇ/D ̨Cˇforall; ̨;ˇ2C.
10 Find x 2 R4 such that
.4; 3; 1; 7/ C 2x D .5; 9; 6; 8/:
11 Explain why there does not exist 2 C such that
.2 3i; 5 C 4i; 6 C 7i/ D .12 5i; 7 C 22i; 32 9i/:
12 Showthat.xCy/CzDxC.yCz/forallx;y;z2Fn.
13 Showthat.ab/xDa.bx/forallx2Fn andalla;b2F.
14 Showthat1xDxforallx2Fn.
15 Showthat.xCy/DxCyforall2Fandallx;y2Fn.
16 Showthat.aCb/xDaxCbxforalla;b2Fandallx2Fn.
12 CHAPTER 1 Vector Spaces
1.B Definition of Vector Space
The motivation for the definition of a vector space comes from properties of addition and scalar multiplication in Fn: Addition is commutative, associative, and has an identity. Every element has an additive inverse. Scalar multiplica- tion is associative. Scalar multiplication by 1 acts as expected. Addition and scalar multiplication are connected by distributive properties.
We will define a vector space to be a set V with an addition and a scalar multiplication on V that satisfy the properties in the paragraph above.
1.18 Definition addition, scalar multiplication
An addition on a set V is a function that assigns an element uCv 2 V
to each pair of elements u; v 2 V.
A scalar multiplication on a set V is a function that assigns an ele- mentv2V toeach2Fandeachv2V.
Now we are ready to give the formal definition of a vector space.
1.19 Definition vector space
A vector space is a set V along with an addition on V and a scalar multi-
plication on V such that the following properties hold: commutativity
u C v D v C u for all u; v 2 V ;
associativity
.u C v/ C w D u C .v C w/ and .ab/v D a.bv/ for all u; v; w 2 V and all a; b 2 F;
additive identity
thereexistsanelement02V suchthatvC0Dvforallv2V; additive inverse
foreveryv2V,thereexistsw2V suchthatvCwD0; multiplicative identity
1v D v for all v 2 V ;
distributive properties
a.u C v/ D au C av and .a C b/v D av C bv for all a; b 2 F and all u; v 2 V.
SECTION 1.B Definition of Vector Space 13 The following geometric language sometimes aids our intuition.
The scalar multiplication in a vector space depends on F. Thus when we need to be precise, we will say that V is a vector space over F instead of saying simply that V is a vector space. For example, Rn is a vector space over R, and Cn is a vector space over C.
1.20 Definition vector, point
Elements of a vector space are called vectors or points.
1.21 Definition real vector space, complex vector space A vector space over R is called a real vector space.
A vector space over C is called a complex vector space.
Usually the choice of F is either obvious from the context or irrelevant. Thus we often assume that F is lurking in the background without specifically mentioning it.
With the usual operations of addition and scalar multiplication, Fn is a vector space over F, as you should verify. The example of Fn motivated our definition of vector space.
1.22 Example F1 is defined to be the set of all sequences of elements of F:
F1 D f.x1;x2;:::/ W xj 2 F for j D 1;2;:::g:
Addition and scalar multiplication on F1 are defined as expected:
.x1;x2;:::/C.y1;y2;:::/D.x1 Cy1;x2 Cy2;:::/; .x1;x2;:::/ D .x1;x2;:::/:
With these definitions, F1 becomes a vector space over F, as you should verify. The additive identity in this vector space is the sequence of all 0’s.
Our next example of a vector space involves a set of functions.
The simplest vector space contains only one point. In other words, f0g is a vector space.
14 CHAPTER 1 Vector Spaces
1.23 Notation FS
If S is a set, then FS denotes the set of functions from S to F.
Forf;g2FS,thesumf Cg2FS isthefunctiondefinedby .f Cg/.x/Df.x/Cg.x/
for all x 2 S.
For2Fandf 2FS,theproductf 2FS isthefunction
defined by for all x 2 S.
.f /.x/ D f .x/
As an example of the notation above, if S is the interval Œ0; 1 and F D R, then RŒ0;1 is the set of real-valued functions on the interval Œ0; 1.
You should verify all three bullet points in the next example.
1.24 Example FS is a vector space
If S is a nonempty set, then FS (with the operations of addition and
scalar multiplication as defined above) is a vector space over F. The additive identity of FS is the function 0 W S ! F defined by
0.x/ D 0
Forf 2FS,theadditiveinverseoff isthefunctionf WS !F
for all x 2 S. defined by
for all x 2 S.
.f /.x/ D f .x/
Our previous examples of vector spaces, Fn and F1, are special cases of the vector space FS because a list of length n of numbers in F can be thought of as a function from f1;2;:::;ng to F and a sequence of numbers in F can be thought of as a function from the set of
The elements of the vector space RŒ0;1 are real-valued functions on Œ0; 1, not lists. In general, a vector space is an abstract entity whose elements might be lists, functions, or weird objects.
positive integers to F. In other words, we can think of Fn as Ff1;2;:::;ng and we can think of F1 as Ff1;2;::: g.
SECTION 1.B Definition of Vector Space 15
Soon we will see further examples of vector spaces, but first we need to develop some of the elementary properties of vector spaces.
The definition of a vector space requires that it have an additive identity. The result below states that this identity is unique.
Proof Suppose 0 and 00 are both additive identities for some vector space V. Then
00 D00 C0D0C00 D0;
where the first equality holds because 0 is an additive identity, the second equality comes from commutativity, and the third equality holds because 00 is an additive identity. Thus 00 D 0, proving that V has only one additive identity.
Each element v in a vector space has an additive inverse, an element w in the vector space such that v C w D 0. The next result shows that each element in a vector space has only one additive inverse.
Proof Suppose V is a vector space. Let v 2 V. Suppose w and w0 are additive inverses of v. Then
wDwC0DwC.vCw0/D.wCv/Cw0 D0Cw0 Dw0: Thus w D w0, as desired.
Because additive inverses are unique, the following notation now makes sense.
1.25 Unique additive identity
A vector space has a unique additive identity.
1.26 Unique additive inverse
Every element in a vector space has a unique additive inverse.
1.27 Notation v, w v Let v;w 2 V. Then
v denotes the additive inverse of v; wvisdefinedtobewC.v/.
16 CHAPTER 1 Vector Spaces
Almost all the results in this book involve some vector space. To avoid having to restate frequently that V is a vector space, we now make the necessary declaration once and for all:
In the next result, 0 denotes a scalar (the number 0 2 F) on the left side of the equation and a vector (the additive identity of V ) on the right side of the equation.
Proof For v 2 V, we have
0v D .0 C 0/v D 0v C 0v:
Adding the additive inverse of 0v to both sides of the equation above gives 0 D 0v, as desired.
In the next result, 0 denotes the addi- tive identity of V. Although their proofs are similar, 1.29 and 1.30 are not identical. More precisely, 1.29 states that the product of the scalar 0 and any vector equals the vector 0, whereas 1.30
states that the product of any scalar and the vector 0 equals the vector 0.
Proof For a 2 F, we have
a0 D a.0 C 0/ D a0 C a0:
Adding the additive inverse of a0 to both sides of the equation above gives 0 D a0, as desired.
Now we show that if an element of V is multiplied by the scalar 1, then the result is the additive inverse of the element of V.
1.28 Notation V
For the rest of the book, V denotes a vector space over F.
1.29 The number 0 times a vector 0v D 0 for every v 2 V.
Note that 1.29 asserts something about scalar multiplication and the additive identity of V. The only part of the definition of a vector space that connects scalar multi- plication and vector addition is the distributive property. Thus the dis- tributive property must be used in the proof of 1.29.
1.30 A number times the vector 0 a0 D 0 for every a 2 F.
SECTION 1.B Definition of Vector Space 17
1.31 The number 1 times a vector .1/v D v for every v 2 V.
Proof For v 2 V, we have
v C .1/v D 1v C .1/v D 1 C .1/v D 0v D 0:
This equation says that .1/v, when added to v, gives 0. Thus .1/v is the additive inverse of v, as desired.
EXERCISES 1.B
1 Provethat.v/Dvforeveryv2V.
2 Supposea2F,v2V,andavD0.ProvethataD0orvD0.
3 Suppose v; w 2 V. Explain why there exists a unique x 2 V such that v C 3x D w.
4 The empty set is not a vector space. The empty set fails to satisfy only one of the requirements listed in 1.19. Which one?
5 Show that in the definition of a vector space (1.19), the additive inverse condition can be replaced with the condition that
0v D 0 for all v 2 V:
Here the 0 on the left side is the number 0, and the 0 on the right side is the additive identity of V. (The phrase “a condition can be replaced” in a definition means that the collection of objects satisfying the definition is unchanged if the original condition is replaced with the new condition.)
6 Let 1 and 1 denote two distinct objects, neither of which is in R. Define an addition and scalar multiplication on R [ f1g [ f1g as you could guess from the notation. Specifically, the sum and product of two
real numbers is as usual, and for t 2 R define 88
ˆ<1 if t < 0; t1Dˆ:0 if tD0; 1 if t > 0;
ˆ<1 t.1/Dˆ:0 if tD0;
1 if t > 0;
t C .1/ D .1/ C t D 1;
t C 1 D 1 C t D 1;
1 C 1 D 1; .1/ C .1/ D 1; 1 C .1/ D 0:
Is R [ f1g [ f1g a vector space over R? Explain.
if t < 0;
18 CHAPTER 1 Vector Spaces 1.C Subspaces
By considering subspaces, we can greatly expand our examples of vector spaces.
1.32 Definition subspace
A subset U of V is called a subspace of V if U is also a vector space (using the same addition and scalar multiplication as on V ).
1.33 Example
f.x1; x2; 0/ W x1; x2 2 Fg is a subspace of F3.
The next result gives the easiest way to check whether a subset of a vector space is a subspace.
Some mathematicians use the term linear subspace, which means the same as subspace.
1.34 Conditions for a subspace
A subset U of V is a subspace of V if and only if U satisfies the following three conditions:
additive identity
02U
closed under addition
u;w2U impliesuCw2U;
closed under scalar multiplication a2Fandu2U impliesau2U.
Proof If U is a subspace of V, then U satisfies the three conditions above by the definition of vector space.
Conversely, suppose U satisfies the three conditions above. The first con- dition above ensures that the additive identity of V is in U.
The second condition above ensures that addition makes sense on U. The third condition ensures that scalar mul- tiplication makes sense on U.
The additive identity condition above could be replaced with the condition that U is nonempty (then taking u 2 U, multiplying it by 0, and using the condition that U is closed under scalar multiplication would imply that 0 2 U ). However, if U is indeed a subspace of V, then the easiest way to show that U is nonempty is to show that 0 2 U.
If u 2 U, then u [which equals .1/u by 1.31] is also in U by the third condition above. Hence every element of U has an additive inverse in U.
The other parts of the definition of a vector space, such as associativity and commutativity, are automatically satisfied for U because they hold on the larger space V. Thus U is a vector space and hence is a subspace of V.
The three conditions in the result above usually enable us to determine quickly whether a given subset of V is a subspace of V. You should verify all the assertions in the next example.
1.35 Example subspaces
(a) Ifb2F,then
f.x1;x2;x3;x4/2F4 Wx3 D5x4 Cbg isasubspaceofF4 ifandonlyifbD0.
(b) The set of continuous real-valued functions on the interval Œ0; 1 is a subspace of RŒ0;1.
(c) The set of differentiable real-valued functions on R is a subspace of RR .
(d) The set of differentiable real-valued functions f on the interval .0; 3/
suchthatf0.2/DbisasubspaceofR.0;3/ ifandonlyifbD0.
(e) The set of all sequences of complex numbers with limit 0 is a subspace
of C1.
Verifying some of the items above
shows the linear structure underlying parts of calculus. For example, the sec- ond item above requires the result that the sum of two continuous functions is continuous. As another example, the fourth item above requires the result that for a constant c, the derivative of cf equals c times the derivative of f .
The subspaces of R2 are precisely f0g, R2, and all lines in R2 through the origin. The subspaces of R3 are precisely f0g, R3, all lines in R3 through the origin, and all planes in R3 through the origin. To prove that all these objects are indeed subspaces is easy—the hard part is to show that they are the only subspaces of R2 and R3. That task will be easier after we introduce some additional tools in the next chapter.
SECTION 1.C Subspaces 19
Clearly f0g is the smallest sub- space of V and V itself is the largest subspace of V. The empty set is not a subspace of V because a subspace must be a vector space and hence must contain at least one element, namely, an additive identity.
20 CHAPTER 1 Vector Spaces Sums of Subspaces
When dealing with vector spaces, we are usually interested only in subspaces, as opposed to arbitrary subsets. The notion of the sum of subspaces will be useful.
The union of subspaces is rarely a subspace (see Exercise 12), which is why we usually work with sums rather than unions.
1.36 Definition sum of subsets
Suppose U1;:::;Um are subsets of V. The sum of U1;:::;Um, denoted U1 CCUm,isthesetofallpossiblesumsofelementsofU1;:::;Um. More precisely,
U1 CCUm Dfu1 CCum Wu1 2U1;:::;um 2Umg:
Let’s look at some examples of sums of subspaces.
1.37 Example Suppose U is the set of all elements of F3 whose second and third coordinates equal 0, and W is the set of all elements of F3 whose first and third coordinates equal 0:
U Df.x;0;0/2F3 Wx2Fg and W Df.0;y;0/2F3 Wy2Fg: Then
U C W D f.x; y; 0/ W x; y 2 Fg;
as you should verify.
1.38 Example Suppose that U D f.x;x;y;y/ 2 F4 W x;y 2 Fg and
W D f.x;x;x;y/ 2 F4 W x;y 2 Fg. Then
U C W D f.x; x; y; z/ 2 F4 W x; y; z 2 Fg;
as you should verify.
The next result states that the sum of subspaces is a subspace, and is in fact the smallest subspace containing all the summands.
1.39 Sum of subspaces is the smallest containing subspace
Suppose U1;:::;Um are subspaces of V. Then U1 C C Um is the smallest subspace of V containing U1; : : : ; Um.
Proof It is easy to see that 0 2 U1 CCUm and that U1 CCUm is closed under addition and scalar multiplication. Thus 1.34 implies that U1 CCUm isasubspaceofV.
Clearly U1;:::;Um are all con- tained in U1 C C Um (to see this, consider sums u1 C C um where all except one of the u’s are 0). Con- versely, every subspace of V contain- ingU1;:::;Um containsU1CCUm (because subspaces must contain all fi- nite sums of their elements). Thus U1 C C Um is the smallest subspace of V containing U1;:::;Um.
Direct Sums
Suppose U1;:::;Um are subspaces of V. Every element of U1 C C Um can be written in the form
u1 CCum;
where each uj is in Uj . We will be especially interested in cases where each vector in U1 C C Um can be represented in the form above in only one way. This situation is so important that we give it a special name: direct sum.
SECTION 1.C Subspaces 21
Sums of subspaces in the theory of vector spaces are analogous to unions of subsets in set theory. Given two subspaces of a vector space, the smallest subspace con- taining them is their sum. Analo- gously, given two subsets of a set, the smallest subset containing them is their union.
1.40 Definition direct sum
Suppose U1; : : : ; Um are subspaces of V.
The sum U1 C C Um is called a direct sum if each element of U1 CCUm can be written in only one way as a sum u1 CCum,whereeachuj isinUj.
If U1 CCUm is a direct sum, then U1 ̊ ̊Um denotes U1 C C Um, with the ̊ notation serving as an indication that this is a direct sum.
1.41 Example Suppose U is the subspace of F3 of those vectors whose last coordinate equals 0, and W is the subspace of F3 of those vectors whose first two coordinates equal 0:
U Df.x;y;0/2F3 Wx;y2Fg and W Df.0;0;z/2F3 Wz2Fg: Then F3 D U ̊ W, as you should verify.
7 CHAPTER 1 Vector Spaces
1.42 Example Suppose Uj is the subspace of Fn of those vectors whose coordinates are all 0, except possibly in the jth slot (thus, for example, U2 D f.0;x;0;:::;0/ 2 Fn W x 2 Fg). Then
Fn D U1 ̊ ̊ Un;
Sometimes nonexamples add to our understanding as much as examples.
1.43 Example Let
U1 Df.x;y;0/2F3 Wx;y 2Fg;
U2 Df.0;0;z/2F3 Wz2Fg;
U3 Df.0;y;y/2F3 Wy 2Fg: ShowthatU1 CU2 CU3 isnotadirectsum.
Solution Clearly F3 D U1 C U2 C U3, because every vector .x; y; z/ 2 F3 can be written as
.x; y; z/ D .x; y; 0/ C .0; 0; z/ C .0; 0; 0/;
where the first vector on the right side is in U1, the second vector is in U2, and the third vector is in U3.
However, F3 does not equal the direct sum of U1; U2; U3, because the vector .0; 0; 0/ can be written in two different ways as a sum u1 C u2 C u3, with each uj in Uj . Specifically, we have
as you should verify.
.0; 0; 0/ D .0; 1; 0/ C .0; 0; 1/ C .0; 1; 1/ .0; 0; 0/ D .0; 0; 0/ C .0; 0; 0/ C .0; 0; 0/;
and, of course,
where the first vector on the right side of each equation above is in U1, the
second vector is in U2, and the third vector is in U3.
The definition of direct sum requires that every vector in the sum have a unique representation as an appropriate sum. The next result shows that when deciding whether a sum of subspaces is a direct sum, we need only consider whether 0 can be uniquely written as an appropriate sum.
The symbol ̊, which is a plus sign inside a circle, serves as a re- minder that we are dealing with a special type of sum of subspaces— each element in the direct sum can be represented only one way as a sum of elements from the specified subspaces.
SECTION 1.C Subspaces 23
1.44 Condition for a direct sum
Suppose U1; : : : ; Um are subspaces of V. Then U1 C C Um is a direct sumifandonlyiftheonlywaytowrite0asasumu1 CCum,where eachuj isinUj,isbytakingeachuj equalto0.
Proof First suppose U1 C C Um is a direct sum. Then the definition of direct sum implies that the only way to write 0 as a sum u1 CCum, where eachuj isinUj,isbytakingeachuj equalto0.
Nowsupposethattheonlywaytowrite0asasumu1 CCum,where each uj is in Uj , is by taking each uj equal to 0. To show that U1 CCUm is a direct sum, let v 2 U1 C C Um. We can write
v D u1 C C um
for some u1 2 U1; : : : ; um 2 Um. To show that this representation is unique,
suppose we also have
v D v1 C C vm;
where v1 2 U1; : : : ; vm 2 Um. Subtracting these two equations, we have
0D.u1 v1/CC.um vm/:
Becauseu1 v1 2U1;:::;um vm 2Um,theequationaboveimpliesthat
each uj vj equals 0. Thus u1 D v1;:::;um D vm, as desired.
The next result gives a simple condition for testing which pairs of sub-
spaces give a direct sum.
Proof First suppose that U CW is a direct sum. If v 2 U \W, then 0 D v C .v/, where v 2 U and v 2 W. By the unique representation of0asthesumofavectorinU andavectorinW,wehavevD0. Thus U \ W D f0g, completing the proof in one direction.
To prove the other direction, now suppose U \ W D f0g. To prove that U CW isadirectsum,supposeu2U,w2W,and
0 D u C w:
To complete the proof, we need only show that u D w D 0 (by 1.44). The equationaboveimpliesthatuDw2W. Thusu2U \W. HenceuD0, which by the equation above implies that w D 0, completing the proof.
1.45 Direct sum of two subspaces
Suppose U and W are subspaces of V. Then U C W is a direct sum if andonlyifU \W Df0g.
24 CHAPTER 1
Vector Spaces
The result above deals only with the case of two subspaces. When ask- ing about a possible direct sum with more than two subspaces, it is not enough to test that each pair of the subspaces intersect only at 0. To see this, consider Example 1.43. In that nonexample of a direct sum, we have U1\U2 DU1\U3 DU2\U3 Df0g.
Sums of subspaces are analogous to unions of subsets. Similarly, di- rect sums of subspaces are analo- gous to disjoint unions of subsets. No two subspaces of a vector space can be disjoint, because both con- tain 0. So disjointness is replaced, at least in the case of two sub- spaces, with the requirement that the intersection equals f0g.
EXERCISES 1.C
1 For each of the following subsets of F3, determine whether it is a sub- space of F3:
(a) f.x1;x2;x3/2F3 Wx1 C2x2 C3x3 D0g;
(b) f.x1;x2;x3/2F3 Wx1 C2x2 C3x3 D4g;
(c) f.x1; x2; x3/ 2 F3 W x1x2x3 D 0g;
(d) f.x1;x2;x3/ 2 F3 W x1 D 5x3g.
2 Verify all the assertions in Example 1.35.
3 Show that the set of differentiable real-valued functions f on the interval
.4; 4/ such that f 0.1/ D 3f .2/ is a subspace of R.4;4/.
4 Suppose b 2 R. Show that the set of continuous real-valued functions f
on the interval Œ0; 1 such that R 1 f D b is a subspace of RŒ0;1 if and
only if b D 0.
0
5 Is R2 a subspace of the complex vector space C2?
6 (a) Is f.a;b;c/ 2 R3 W a3 D b3g a subspace of R3? (b) Is f.a;b;c/ 2 C3 W a3 D b3g a subspace of C3?
7 Give an example of a nonempty subset U of R2 such that U is closed under addition and under taking additive inverses (meaning u 2 U wheneveru2U),butU isnotasubspaceofR2.
8 Give an example of a nonempty subset U of R2 such that U is closed under scalar multiplication, but U is not a subspace of R2.
9 A function f W R ! R is called periodic if there exists a positive number psuchthatf.x/Df.xCp/forallx2R. Isthesetofperiodic functions from R to R a subspace of RR ? Explain.
10 Suppose U1 and U2 are subspaces of V. Prove that the intersection U1 \ U2 is a subspace of V.
11 Prove that the intersection of every collection of subspaces of V is a subspace of V.
12 Prove that the union of two subspaces of V is a subspace of V if and only if one of the subspaces is contained in the other.
13 Prove that the union of three subspaces of V is a subspace of V if and only if one of the subspaces contains the other two.
[This exercise is surprisingly harder than the previous exercise, possibly because this exercise is not true if we replace F with a field containing only two elements.]
14 Verify the assertion in Example 1.38.
15 SupposeUisasubspaceofV.WhatisUCU?
16 Is the operation of addition on the subspaces of V commutative? In other words,ifU andW aresubspacesofV,isU CW DW CU?
17 Is the operation of addition on the subspaces of V associative? In other words, if U1; U2; U3 are subspaces of V, is
.U1 CU2/CU3 DU1 C.U2 CU3/‹
18 Does the operation of addition on the subspaces of V have an additive
identity? Which subspaces have additive inverses?
19 Prove or give a counterexample: if U1; U2; W are subspaces of V such that
U1 C W D U2 C W;
then U1 D U2.
20 Suppose
U Df.x;x;y;y/2F4 Wx;y 2Fg: FindasubspaceW ofF4 suchthatF4 DU ̊W.
SECTION 1.C Subspaces 25
26 CHAPTER 1 Vector Spaces
21 Suppose
U Df.x;y;xCy;xy;2x/2F5 Wx;y2Fg: FindasubspaceW ofF5 suchthatF5 DU ̊W.
22 Suppose
U Df.x;y;xCy;xy;2x/2F5 Wx;y2Fg:
Find three subspaces W1; W2; W3 of F5, none of which equals f0g, such
thatF5 DU ̊W1 ̊W2 ̊W3.
23 Prove or give a counterexample: if U1; U2; W are subspaces of V such
that
VDU1 ̊W and VDU2 ̊W;
then U1 D U2.
24 AfunctionfWR!Riscalledevenif
f.x/ D f.x/
for all x 2 R. A function f W R ! R is called odd if
f.x/ D f.x/
for all x 2 R. Let Ue denote the set of real-valued even functions on R and let Uo denote the set of real-valued odd functions on R. Show that RR D Ue ̊ Uo.
CHAPTER
2
Finite-Dimensional Vector Spaces
Let’s review our standing assumptions:
In the last chapter we learned about vector spaces. Linear algebra focuses not on arbitrary vector spaces, but on finite-dimensional vector spaces, which we introduce in this chapter.
American mathematician Paul Halmos (1916–2006), who in 1942 published the first modern linear algebra book. The title of Halmos’s book was the same as the title of this chapter.
2.1 Notation F, V
F denotes R or C.
V denotes a vector space over F.
LEARNING OBJECTIVES FOR THIS CHAPTER span
linear independence bases
dimension
© Springer International Publishing 2015 27 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__2
28 CHAPTER 2 Finite-Dimensional Vector Spaces
2.A Span and Linear Independence
We have been writing lists of numbers surrounded by parentheses, and we will continue to do so for elements of Fn; for example, .2; 7; 8/ 2 F3. However, now we need to consider lists of vectors (which may be elements of Fn or of other vector spaces). To avoid confusion, we will usually write lists of vectors without surrounding parentheses. For example, .4; 1; 6/; .9; 5; 7/ is a list of length 2 of vectors in R3.
Linear Combinations and Span
Adding up scalar multiples of vectors in a list gives what is called a linear combination of the list. Here is the formal definition:
2.2 Notation list of vectors
We will usually write lists of vectors without surrounding parentheses.
2.3 Definition linear combination
A linear combination of a list v1;:::;vm of vectors in V is a vector of
the form
where a1;:::;am 2 F.
a1v1 CCamvm;
2.4
Example In F3,
.17;4;2/isalinearcombinationof.2;1;3/;.1;2;4/because
.17; 4; 2/ D 6.2; 1; 3/ C 5.1; 2; 4/:
.17;4;5/isnotalinearcombinationof.2;1;3/;.1;2;4/because
there do not exist numbers a1; a2 2 F such that
.17; 4; 5/ D a1.2; 1; 3/ C a2.1; 2; 4/:
In other words, the system of equations
17 D 2a1 C a2 4 D a1 2a2
5 D 3a1 C 4a2 has no solutions (as you should verify).
SECTION 2.A Span and Linear Independence 29
2.5 Definition span
The set of all linear combinations of a list of vectors v1; : : : ; vm in V is
called the span of v1; : : : ; vm, denoted span.v1; : : : ; vm/. In other words, span.v1;:::;vm/Dfa1v1 CCamvm Wa1;:::;am 2Fg:
The span of the empty list . / is defined to be f0g.
2.6 Example The previous example shows that in F3, .17;4;2/2span.2;1;3/;.1;2;4/;
.17;4;5/...span.2;1;3/;.1;2;4/.
Some mathematicians use the term linear span, which means the same as span.
Proof Suppose v1;:::;vm is a list of vectors in V.
First we show that span.v1; : : : ; vm/ is a subspace of V. The additive
identity is in span.v1; : : : ; vm/, because
0 D 0v1 C C 0vm:
Also, span.v1; : : : ; vm/ is closed under addition, because
.a1v1C Camvm/C.c1v1C Ccmvm/ D .a1Cc1/v1C C.amCcm/vm: Furthermore, span.v1; : : : ; vm/ is closed under scalar multiplication, because
.a1v1 CCamvm/Da1v1 CCamvm:
Thus span.v1; : : : ; vm/ is a subspace of V (by 1.34).
Each vj is a linear combination of v1;:::;vm (to show this, set aj D 1
and let the other a’s in 2.3 equal 0). Thus span.v1; : : : ; vm/ contains each vj . Conversely, because subspaces are closed under scalar multiplication and addition, every subspace of V containing each vj contains span.v1; : : : ; vm/. Thus span.v1; : : : ; vm/ is the smallest subspace of V containing all the vectors v1;:::;vm.
2.7 Span is the smallest containing subspace
The span of a list of vectors in V is the smallest subspace of V containing all the vectors in the list.
30 CHAPTER 2 Finite-Dimensional Vector Spaces
2.8 Definition spans
If span.v1;:::;vm/ equals V, we say that v1;:::;vm spans V.
2.9 Example Suppose n is a positive integer. Show that .1;0;:::;0/;.0;1;0;:::;0/;:::;.0;:::;0;1/
spans Fn . Here the j th vector in the list above is the n-tuple with 1 in the j th slot and 0 in all other slots.
Solution Suppose .x1; : : : ; xn/ 2 Fn. Then
.x1; : : : ; xn/ D x1.1; 0; : : : ; 0/ C x2.0; 1; 0; : : : ; 0/ C C xn.0; : : : ; 0; 1/:
Thus .x1; : : : ; xn/ 2 span.1; 0; : : : ; 0/; .0; 1; 0; : : : ; 0/; : : : ; .0; : : : ; 0; 1/, as desired.
Now we can make one of the key definitions in linear algebra.
Example 2.9 above shows that Fn is a finite-dimensional vector space for every positive integer n.
The definition of a polynomial is no doubt already familiar to you.
2.10 Definition finite-dimensional vector space
A vector space is called finite-dimensional if some list of vectors in it spans the space.
Recall that by definition every list has finite length.
2.11 Definition polynomial, P.F/
A function p W F ! F is called a polynomial with coefficients in F
if there exist a0;:::;am 2 F such that
p.z/Da0 Ca1zCa2z2 CCamzm
for all z 2 F.
P.F/ is the set of all polynomials with coefficients in F.
SECTION 2.A Span and Linear Independence 31
With the usual operations of addition and scalar multiplication, P.F/ is a vector space over F, as you should verify. In other words, P.F/ is a subspace of FF, the vector space of functions from F to F.
If a polynomial (thought of as a function from F to F) is represented by two sets of coefficients, then subtracting one representation of the polynomial from the other produces a polynomial that is identically zero as a function on F and hence has all zero coefficients (if you are unfamiliar with this fact, just believe it for now; we will prove it later—see 4.7). Conclusion: the coefficients of a polynomial are uniquely determined by the polynomial. Thus the next definition uniquely defines the degree of a polynomial.
2.12 Definition degree of a polynomial, deg p
A polynomial p 2 P.F/ is said to have degree m if there exist
scalars a0;a1;:::;am 2 F with am ¤ 0 such that p.z/Da0 Ca1zCCamzm
for all z 2 F. If p has degree m, we write deg p D m.
The polynomial that is identically 0 is said to have degree 1.
In the next definition, we use the convention that 1 < m, which means that the polynomial 0 is in Pm.F/.
To verify the next example, note that Pm.F/ D span.1; z; : : : ; zm/; here we are slightly abusing notation by letting zk denote a function.
2.14 Example Pm.F/ is a finite-dimensional vector space for each non- negative integer m.
2.13 Definition Pm.F/
For m a nonnegative integer, Pm.F/ denotes the set of all polynomials with coefficients in F and degree at most m.
2.15 Definition infinite-dimensional vector space
A vector space is called infinite-dimensional if it is not finite-dimensional.
32 CHAPTER 2 Finite-Dimensional Vector Spaces
2.16 Example Show that P.F/ is infinite-dimensional.
Solution Consider any list of elements of P.F/. Let m denote the highest degree of the polynomials in this list. Then every polynomial in the span of this list has degree at most m. Thus zmC1 is not in the span of our list. Hence no list spans P.F/. Thus P.F/ is infinite-dimensional.
Linear Independence
Suppose v1;:::;vm 2 V and v 2 span.v1;:::;vm/. By the definition of span, there exist a1;:::;am 2 F such that
v D a1v1 C C amvm:
Consider the question of whether the choice of scalars in the equation above
is unique. Suppose c1; : : : ; cm is another set of scalars such that v D c1v1 C C cmvm:
Subtracting the last two equations, we have
0D.a1 c1/v1 CC.am cm/vm:
Thus we have written 0 as a linear combination of .v1; : : : ; vm/. If the only way to do this is the obvious way (using 0 for all scalars), then each aj cj equals 0, which means that each aj equals cj (and thus the choice of scalars was indeed unique). This situation is so important that we give it a special name—linear independence—which we now define.
2.17 Definition linearly independent
A list v1; : : : ; vm of vectors in V is called linearly independent if theonlychoiceofa1;:::;am 2Fthatmakesa1v1CCamvm equal0isa1 DDam D0.
The empty list . / is also declared to be linearly independent.
The reasoning above shows that v1; : : : ; vm is linearly independent if and only if each vector in span.v1; : : : ; vm/ has only one representation as a linear combination of v1; : : : ; vm.
2.18
(a) (b)
(c) (d)
SECTION 2.A Span and Linear Independence 33
Example linearly independent lists
A list v of one vector v 2 V is linearly independent if and only if v ¤ 0.
A list of two vectors in V is linearly independent if and only if neither vector is a scalar multiple of the other.
.1; 0; 0; 0/; .0; 1; 0; 0/; .0; 0; 1; 0/ is linearly independent in F4.
The list 1; z; : : : ; zm is linearly independent in P.F/ for each nonnega-
tive integer m.
If some vectors are removed from a linearly independent list, the remaining
list is also linearly independent, as you should verify.
2.19 Definition linearly dependent
A list of vectors in V is called linearly dependent if it is not linearly
independent.
In other words, a list v1;:::;vm of vectors in V is linearly de- pendent if there exist a1;:::;am 2 F, not all 0, such that a1v1 C C amvm D 0.
2.20 Example linearly dependent lists
.2; 3; 1/; .1; 1; 2/; .7; 3; 8/ is linearly dependent in F3 because
2.2; 3; 1/ C 3.1; 1; 2/ C .1/.7; 3; 8/ D .0; 0; 0/:
The list .2; 3; 1/; .1; 1; 2/; .7; 3; c/ is linearly dependent in F3 if and
only if c D 8, as you should verify.
If some vector in a list of vectors in V is a linear combination of the other vectors, then the list is linearly dependent. (Proof: After writing one vector in the list as equal to a linear combination of the other vectors, move that vector to the other side of the equation, where it will be multiplied by 1.)
Every list of vectors in V containing the 0 vector is linearly dependent. (This is a special case of the previous bullet point.)
34 CHAPTER 2 Finite-Dimensional Vector Spaces
The lemma below will often be useful. It states that given a linearly dependent list of vectors, one of the vectors is in the span of the previous ones and furthermore we can throw out that vector without changing the span of the original list.
2.21 Linear Dependence Lemma
Suppose v1; : : : ; vm is a linearly dependent list in V. Then there exists j 2 f1;2;:::;mg such that the following hold:
(a) vj 2 span.v1;:::;vj1/;
(b) if the jth term is removed from v1;:::;vm, the span of the remain- ing list equals span.v1; : : : ; vm/.
Proof Because the list v1; : : : ; vm is linearly dependent, there exist numbers a1;:::;am 2 F, not all 0, such that
a1v1 C C amvm D 0:
Let j be the largest element of f1;:::;mg such that aj ¤ 0. Then
2.22 vj Da1v1aj1vj1; aj aj
proving (a).
To prove (b), suppose u 2 span.v1; : : : ; vm/. Then there exist numbers
c1;:::;cm 2 F such that
u D c1v1 C C cmvm:
In the equation above, we can replace vj with the right side of 2.22, which shows that u is in the span of the list obtained by removing the j th term from v1;:::;vm. Thus (b) holds.
Choosing j D 1 in the Linear Dependence Lemma above means that v1 D 0, because if j D 1 then condition (a) above is interpreted to mean that v1 2 span. /; recall that span. / D f0g. Note also that the proof of part (b) aboveneedstobemodifiedinanobviouswayifv1 D0andj D1.
In general, the proofs in the rest of the book will not call attention to special cases that must be considered involving empty lists, lists of length 1, the subspace f0g, or other trivial cases for which the result is clearly true but needs a slightly different proof. Be sure to check these special cases yourself.
Now we come to a key result. It says that no linearly independent list in V is longer than a spanning list in V.
SECTION 2.A Span and Linear Independence 35
2.23 Length of linearly independent list length of spanning list
In a finite-dimensional vector space, the length of every linearly indepen- dent list of vectors is less than or equal to the length of every spanning list of vectors.
Proof Suppose u1; : : : ; um is linearly independent in V. Suppose also that w1;:::;wn spans V. We need to prove that m n. We do so through the multi-step process described below; note that in each step we add one of the u’s and remove one of the w’s.
Step 1
Let B be the list w1; : : : ; wn, which spans V. Thus adjoining any vector in V to this list produces a linearly dependent list (because the newly adjoined vector can be written as a linear combination of the other vectors). In particular, the list
u1;w1;:::;wn
is linearly dependent. Thus by the Linear Dependence Lemma (2.21), we can remove one of the w’s so that the new list B (of length n) consisting of u1 and the remaining w’s spans V.
Step j
The list B (of length n) from step j 1 spans V. Thus adjoining any vector to this list produces a linearly dependent list. In particular, the listoflength.nC1/obtainedbyadjoininguj toB,placingitjustafter u1 ; : : : ; uj 1 , is linearly dependent. By the Linear Dependence Lemma (2.21), one of the vectors in this list is in the span of the previous ones, andbecauseu1;:::;uj islinearlyindependent,thisvectorisoneof the w’s, not one of the u’s. We can remove that w from B so that the newlistB(oflengthn)consistingofu1;:::;uj andtheremainingw’s spans V.
After step m, we have added all the u’s and the process stops. At each step as we add a u to B, the Linear Dependence Lemma implies that there is some w to remove. Thus there are at least as many w’s as u’s.
The next two examples show how the result above can be used to show, without any computations, that certain lists are not linearly independent and that certain lists do not span a given vector space.
36 CHAPTER 2 Finite-Dimensional Vector Spaces
2.24 Example Show that the list .1; 2; 3/; .4; 5; 8/; .9; 6; 7/; .3; 2; 8/ is not linearly independent in R3.
Solution The list .1; 0; 0/; .0; 1; 0/; .0; 0; 1/ spans R3. Thus no list of length larger than 3 is linearly independent in R3.
2.25 Example Show that the list .1;2;3;5/;.4;5;8;3/;.9;6;7;1/ does not span R4.
Solution The list .1; 0; 0; 0/; .0; 1; 0; 0/; .0; 0; 1; 0/; .0; 0; 0; 1/ is linearly in- dependent in R4. Thus no list of length less than 4 spans R4.
Our intuition suggests that every subspace of a finite-dimensional vector space should also be finite-dimensional. We now prove that this intuition is correct.
Proof Suppose V is finite-dimensional and U is a subspace of V. We need to prove that U is finite-dimensional. We do this through the following multi-step construction.
Step 1
If U D f0g, then U is finite-dimensional and we are done. If U ¤ f0g, then choose a nonzero vector v1 2 U.
Step j
After each step, as long as the process continues, we have constructed a list of vectors such that no vector in this list is in the span of the previous vectors. Thus after each step we have constructed a linearly independent list, by the Linear Dependence Lemma (2.21). This linearly independent list cannot be longer than any spanning list of V (by 2.23). Thus the process eventually terminates, which means that U is finite-dimensional.
2.26 Finite-dimensional subspaces
Every subspace of a finite-dimensional vector space is finite-dimensional.
If U D span.v1; : : : ; vj 1/, then U is finite-dimensional and we are done. If U ¤ span.v1;:::;vj1/, then choose a vector vj 2 U such that
vj ... span.v1;:::;vj1/:
EXERCISES 2.A
SECTION 2.A Span and Linear Independence 37
1 Suppose v1; v2; v3; v4 spans V. Prove that the list v1 v2;v2 v3;v3 v4;v4
also spans V.
2 Verify the assertions in Example 2.18.
3 Find a number t such that
.3;1;4/;.2;3;5/;.5;9;t/ is not linearly independent in R3.
4 Verify the assertion in the second bullet point in Example 2.20.
5 (a) Show that if we think of C as a vector space over R, then the list
.1 C i; 1 i/ is linearly independent.
(b) Show that if we think of C as a vector space over C, then the list
.1 C i; 1 i/ is linearly dependent.
6 Suppose v1; v2; v3; v4 is linearly independent in V. Prove that the list
v1 v2;v2 v3;v3 v4;v4 is also linearly independent.
7 Prove or give a counterexample: If v1; v2; : : : ; vm is a linearly indepen- dent list of vectors in V, then
5v1 4v2;v2;v3;:::;vm is linearly independent.
8 Prove or give a counterexample: If v1; v2; : : : ; vm is a linearly indepen- dent list of vectors in V and 2 F with ¤ 0, then v1;v2;:::;vm is linearly independent.
9 Prove or give a counterexample: If v1;:::;vm and w1;:::;wm are lin- early independent lists of vectors in V, then v1 C w1; : : : ; vm C wm is linearly independent.
10 Suppose v1; : : : ; vm is linearly independent in V and w 2 V. Prove that ifv1 Cw;:::;vm Cwislinearlydependent,thenw2span.v1;:::;vm/.
38 CHAPTER 2 Finite-Dimensional Vector Spaces
11 Suppose v1; : : : ; vm is linearly independent in V and w 2 V. Show that
v1; : : : ; vm; w is linearly independent if and only if w ... span.v1;:::;vm/:
12 Explain why there does not exist a list of six polynomials that is linearly independent in P4.F/.
13 Explain why no list of four polynomials spans P4.F/.
14 Prove that V is infinite-dimensional if and only if there is a sequence v1; v2; : : : of vectors in V such that v1; : : : ; vm is linearly independent for every positive integer m.
15 Prove that F1 is infinite-dimensional.
16 Prove that the real vector space of all continuous real-valued functions
on the interval Œ0; 1 is infinite-dimensional.
17 Suppose p0; p1; : : : ; pm are polynomials in Pm.F/ such that pj .2/ D 0 for each j . Prove that p0; p1; : : : ; pm is not linearly independent in Pm.F/.
2.B Bases
In the last section, we discussed linearly independent lists and spanning lists.
Now we bring these concepts together.
2.28 Example bases
(a) The list .1;0;:::;0/;.0;1;0;:::;0/;:::;.0;:::;0;1/ is a basis of Fn,
called the standard basis of Fn.
(b) The list .1; 2/; .3; 5/ is a basis of F2.
(c) The list .1; 2; 4/; .7; 5; 6/ is linearly independent in F3 but is not a basis of F3 because it does not span F3.
(d) The list .1; 2/; .3; 5/; .4; 13/ spans F2 but is not a basis of F2 because it is not linearly independent.
(e) Thelist.1;1;0/;.0;0;1/isabasisoff.x;x;y/2F3Wx;y2Fg.
(f) The list .1; 1; 0/; .1; 0; 1/ is a basis of
f.x; y; z/ 2 F3 W x C y C z D 0g:
(g) The list 1;z;:::;zm is a basis of Pm.F/.
In addition to the standard basis, Fn has many other bases. For example,
.7; 5/; .4; 9/ and .1; 2/; .3; 5/ are both bases of F2.
The next result helps explain why bases are useful. Recall that “uniquely” means “in only one way”.
SECTION 2.B Bases 39
2.27 Definition basis
A basis of V is a list of vectors in V that is linearly independent and spans V.
2.29 Criterion for basis
Alistv1;:::;vn ofvectorsinV isabasisofV ifandonlyifeveryv2V can be written uniquely in the form
2.30 v D a1v1 C C anvn; where a1;:::;an 2 F.
40 CHAPTER 2 Finite-Dimensional Vector Spaces
Proof First suppose that v1;:::;vn is a basis of V. Let v 2 V. Because v1;:::;vn spans V, there exist a1;:::;an 2 F such that 2.30 holds. To show that the representation in 2.30 is unique, suppose c1; : : : ; cn are scalars
such that we also have
v D c1v1 C C cnvn: Subtracting the last equation from 2.30, we get
0D.a1 c1/v1 CC.an cn/vn:
This implies that each aj cj equals 0 (because v1; : : : ; vn is linearly inde- pendent). Hence a1 D c1; : : : ; an D cn. We have the desired uniqueness, completing the proof in one direction.
For the other direction, suppose every v 2 V can be written uniquely in the form given by 2.30. Clearly this implies that v1; : : : ; vn spans V. To show that v1;:::;vn is linearly independent, suppose a1;:::;an 2 F are such that
0 D a1v1 C C anvn:
The uniqueness of the representation 2.30 (taking v D 0) now implies that a1 D D an D 0. Thus v1;:::;vn is linearly independent and hence is a basis of V.
A spanning list in a vector space may not be a basis because it is not linearly independent. Our next result says that given any spanning list, some (possibly none) of the vectors in it can be discarded so that the remaining list is linearly independent and still spans the vector space.
As an example in the vector space F2, if the procedure in the proof below is applied to the list .1; 2/; .3; 6/; .4; 7/; .5; 9/, then the second and fourth vectors will be removed. This leaves .1; 2/; .4; 7/, which is a basis of F2.
Proof Suppose v1; : : : ; vn spans V. We want to remove some of the vectors from v1; : : : ; vn so that the remaining vectors form a basis of V. We do this through the multi-step process described below.
This proof is essentially a repeti- tion of the ideas that led us to the definition of linear independence.
2.31 Spanning list contains a basis
Every spanning list in a vector space can be reduced to a basis of the vector space.
Start with B equal to the list v1;:::;vn.
Step 1
If v1 D 0, delete v1 from B. If v1 ¤ 0, leave B unchanged. Step j
If vj is in span.v1;:::;vj1/, delete vj from B. If vj is not in span.v1; : : : ; vj 1/, leave B unchanged.
Stop the process after step n, getting a list B. This list B spans V because our original list spanned V and we have discarded only vectors that were already in the span of the previous vectors. The process ensures that no vector in B is in the span of the previous ones. Thus B is linearly independent, by the Linear Dependence Lemma (2.21). Hence B is a basis of V.
Our next result, an easy corollary of the previous result, tells us that every finite-dimensional vector space has a basis.
Proof By definition, a finite-dimensional vector space has a spanning list. The previous result tells us that each spanning list can be reduced to a basis.
Our next result is in some sense a dual of 2.31, which said that every spanning list can be reduced to a basis. Now we show that given any linearly independent list, we can adjoin some additional vectors (this includes the possibility of adjoining no additional vectors) so that the extended list is still linearly independent but also spans the space.
Proof Suppose u1; : : : ; um is linearly independent in a finite-dimensional vector space V. Let w1;:::;wn be a basis of V. Thus the list
u1;:::;um;w1;:::;wn
spans V. Applying the procedure of the proof of 2.31 to reduce this list to a basis of V produces a basis consisting of the vectors u1; : : : ; um (none of the u’s get deleted in this procedure because u1; : : : ; um is linearly independent) and some of the w’s.
SECTION 2.B Bases 41
2.32 Basis of finite-dimensional vector space
Every finite-dimensional vector space has a basis.
2.33 Linearly independent list extends to a basis
Every linearly independent list of vectors in a finite-dimensional vector space can be extended to a basis of the vector space.
42 CHAPTER 2 Finite-Dimensional Vector Spaces
As an example in F3, suppose we start with the linearly independent list .2; 3; 4/; .9; 6; 8/. If we take w1; w2; w3 in the proof above to be the standard basis of F3, then the procedure in the proof above produces the list .2;3;4/;.9;6;8/;.0;1;0/, which is a basis of F3.
As an application of the result above, we now show that every subspace of a finite-dimensional vector space can be paired with another subspace to form a direct sum of the whole space.
Using the same basic ideas but considerably more advanced tools, the next result can be proved with- out the hypothesis that V is finite- dimensional.
2.34 Every subspace of V is part of a direct sum equal to V
Suppose V is finite-dimensional and U is a subspace of V. Then there is a subspaceW ofV suchthatV DU ̊W.
Because V is finite-dimensional, so is U (see 2.26). Thus there is a basis u1;:::;um of U (see 2.32). Of course u1;:::;um is a linearly in- dependent list of vectors in V. Hence this list can be extended to a basis u1;:::;um;w1;:::;wn of V (see 2.33). Let W D span.w1;:::;wn/.
ToprovethatV DU ̊W,by1.45weneedonlyshowthat VDUCW and U\WDf0g:
To prove the first equation above, suppose v 2 V. Then, because the list u1;:::;um;w1;:::;wn spans V, there exist a1;:::;am;b1;:::;bn 2 F such
Proof
that
vDa1u1 CCamumCb1w1 CCbnwn: „ ƒ‚ ...„ ƒ‚ ...
uw
Inotherwords,wehavevDuCw,whereu2U andw2W aredefinedas above. Thusv2U CW,completingtheproofthatV DU CW.
To show that U \ W D f0g, suppose v 2 U \ W. Then there exist scalars a1;:::;am;b1;:::;bn 2 F such that
Thus
vDa1u1 CCamum Db1w1 CCbnwn: a1u1 CCamum b1w1 bnwn D0:
Because u1;:::;um;w1;:::;wn is linearly independent, this implies that a1 DDam Db1 DDbn D0. ThusvD0,completingtheproof thatU \W Df0g.
EXERCISES 2.B
4 (a)
Let U be the subspace of C5 defined by
U D f.z1;z2;z3;z4;z5/ 2 C5 W 6z1 D z2 and z3C2z4C3z5 D 0g:
Find a basis of U.
(b) Extend the basis in part (a) to a basis of R5.
(c) FindasubspaceW ofR5 suchthatR5 DU ̊W.
Find a basis of U.
(b) Extend the basis in part (a) to a basis of C5.
(c) FindasubspaceW ofC5 suchthatC5 DU ̊W.
5 Prove or disprove: there exists a basis p0; p1; p2; p3 of P3.F/ such that none of the polynomials p0; p1; p2; p3 has degree 2.
6 Suppose v1; v2; v3; v4 is a basis of V. Prove that v1 Cv2;v2 Cv3;v3 Cv4;v4
is also a basis of V.
7 Prove or give a counterexample: If v1; v2; v3; v4 is a basis of V and U isasubspaceofV suchthatv1;v2 2U andv3 ...U andv4 ...U,then v1;v2 isabasisofU.
8 Suppose U and W are subspaces of V such that V D U ̊ W. Suppose also that u1;:::;um is a basis of U and w1;:::;wn is a basis of W. Prove that
is a basis of V.
u1;:::;um;w1;:::;wn
SECTION 2.B Bases 43
1 Find all vector spaces that have exactly one basis.
2 Verify all the assertions in Example 2.28.
3 (a)
Let U be the subspace of R5 defined by
U D f.x1;x2;x3;x4;x5/ 2 R5 W x1 D 3x2 and x3 D 7x4g:
44 CHAPTER 2 Finite-Dimensional Vector Spaces 2.C Dimension
Although we have been discussing finite-dimensional vector spaces, we have not yet defined the dimension of such an object. How should dimension be defined? A reasonable definition should force the dimension of Fn to equal n. Notice that the standard basis
.1;0;:::;0/;.0;1;0;:::;0/;:::;.0;:::;0;1/
of Fn has length n. Thus we are tempted to define the dimension as the length of a basis. However, a finite-dimensional vector space in general has many different bases, and our attempted definition makes sense only if all bases in a given vector space have the same length. Fortunately that turns out to be the case, as we now show.
Proof Suppose V is finite-dimensional. Let B1 and B2 be two bases of V. Then B1 is linearly independent in V and B2 spans V, so the length of B1 is at most the length of B2 (by 2.23). Interchanging the roles of B1 and B2, we also see that the length of B2 is at most the length of B1. Thus the length of B1 equals the length of B2, as desired.
Now that we know that any two bases of a finite-dimensional vector space have the same length, we can formally define the dimension of such spaces.
2.35 Basis length does not depend on basis
Any two bases of a finite-dimensional vector space have the same length.
2.36 Definition dimension, dim V
The dimension of a finite-dimensional vector space is the length of
any basis of the vector space.
The dimension of V (if V is finite-dimensional) is denoted by dim V.
2.37 Example dimensions
dim Fn D n because the standard basis of Fn has length n.
dimPm.F/ D m C 1 because the basis 1;z;:::;zm of Pm.F/ has length m C 1.
Every subspace of a finite-dimensional vector space is finite-dimensional (by 2.26) and so has a dimension. The next result gives the expected inequality about the dimension of a subspace.
Proof Suppose V is finite-dimensional and U is a subspace of V. Think of a basis of U as a linearly independent list in V, and think of a basis of V as a spanning list in V. Now use 2.23 to conclude that dim U dim V.
To check that a list of vectors in V is a basis of V, we must, according to the definition, show that the list in ques- tion satisfies two properties: it must be linearly independent and it must span V. The next two results show that if the list in question has the right length, then we need only check that it satisfies one of the two required properties. First we prove that every linearly independent list with the right length is a basis.
Proof Suppose dim V D n and v1; : : : ; vn is linearly independent in V. The list v1; : : : ; vn can be extended to a basis of V (by 2.33). However, every basis of V has length n, so in this case the extension is the trivial one, meaning that no elements are adjoined to v1;:::;vn. In other words, v1;:::;vn is a basis of V, as desired.
2.40 Example Show that the list .5; 7/; .4; 3/ is a basis of F2.
Solution This list of two vectors in F2 is obviously linearly independent (because neither vector is a scalar multiple of the other). Note that F2 has dimension 2. Thus 2.39 implies that the linearly independent list .5; 7/; .4; 3/ of length 2 is a basis of F2 (we do not need to bother checking that it spans F2).
SECTION 2.C Dimension 45
2.38 Dimension of a subspace
If V is finite-dimensional and U is a subspace of V, then dim U dim V.
The real vector space R2 has di- mension 2; the complex vector space C has dimension 1. As sets, R2 can be identified with C (and addition is the same on both spaces, as is scalar multiplication by real numbers). Thus when we talk about the dimension of a vec- tor space, the role played by the choice of F cannot be neglected.
2.39 Linearly independent list of the right length is a basis
Suppose V is finite-dimensional. Then every linearly independent list of vectors in V with length dim V is a basis of V.
46 CHAPTER 2 Finite-Dimensional Vector Spaces
2.41 Example Show that 1; .x 5/2; .x 5/3 is a basis of the subspace U of P3.R/ defined by
U Dfp2P3.R/Wp0.5/D0g:
Solution Clearly each of the polynomials 1, .x 5/2, and .x 5/3 is in U.
Suppose a; b; c 2 R and
aCb.x5/2 Cc.x5/3 D0
for every x 2 R. Without explicitly expanding the left side of the equation above, we can see that the left side has a cx3 term. Because the right side has no x3 term, this implies that c D 0. Because c D 0, we see that the left side hasabx2 term,whichimpliesthatbD0.BecausebDcD0,wecanalso conclude that a D 0.
Thus the equation above implies that a D b D c D 0. Hence the list 1; .x 5/2; .x 5/3 is linearly independent in U.
Thus dimU 3. Because U is a subspace of P3.R/, we know that dim U dim P3.R/ D 4 (by 2.38). However, dim U cannot equal 4, because otherwise when we extend a basis of U to a basis of P3.R/ we would get a list with length greater than 4. Hence dim U D 3. Thus 2.39 implies that the linearly independent list 1; .x 5/2; .x 5/3 is a basis of U.
Now we prove that a spanning list with the right length is a basis.
Proof Suppose dimV D n and v1;:::;vn spans V. The list v1;:::;vn can be reduced to a basis of V (by 2.31). However, every basis of V has length n, so in this case the reduction is the trivial one, meaning that no elements are deleted from v1;:::;vn. In other words, v1;:::;vn is a basis of V, as desired.
The next result gives a formula for the dimension of the sum of two subspaces of a finite-dimensional vector space. This formula is analogous to a familiar counting formula: the number of elements in the union of two finite sets equals the number of elements in the first set, plus the number of elements in the second set, minus the number of elements in the intersection of the two sets.
2.42 Spanning list of the right length is a basis
Suppose V is finite-dimensional. Then every spanning list of vectors in V with length dim V is a basis of V.
Proof Let u1;:::;um be a basis of U1 \ U2; thus dim.U1 \ U2/ D m. Be- cause u1; : : : ; um is a basis of U1 \ U2, it is linearly independent in U1. Hence this list can be extended to a basis u1;:::;um;v1;:::;vj of U1 (by 2.33). Thus dimU1 D m C j. Also extend u1;:::;um to a basis u1;:::;um;w1;:::;wk ofU2;thusdimU2 DmCk.
We will show that
u1;:::;um;v1;:::;vj;w1;:::;wk
is a basis of U1 CU2. This will complete the proof, because then we will have
dim.U1 CU2/DmCj Ck
D .m C j / C .m C k/ m
D dimU1 CdimU2 dim.U1 \U2/:
Clearlyspan.u1;:::;um;v1;:::;vj;w1;:::;wk/containsU1 andU2 and hence equals U1 C U2. So to show that this list is a basis of U1 C U2 we need only show that it is linearly independent. To prove this, suppose
a1u1 CCamum Cb1v1 CCbjvj Cc1w1 CCckwk D0; where all the a’s, b’s, and c’s are scalars. We need to prove that all the a’s,
b’s, and c’s equal 0. The equation above can be rewritten as
c1w1 CCckwk Da1u1 amum b1v1 bjvj;
which shows that c1w1 C C ckwk 2 U1. All the w’s are in U2, so this implies that c1w1 C C ckwk 2 U1 \ U2. Because u1;:::;um is a basis of U1 \ U2, we can write
c1w1CCckwk Dd1u1CCdmum
for some choice of scalars d1;:::;dm. But u1;:::;um;w1;:::;wk is linearly independent, so the last equation implies that all the c’s (and d’s) equal 0. Thus our original equation involving the a’s, b’s, and c’s becomes
a1u1 CCamum Cb1v1 CCbjvj D0:
Becausethelistu1;:::;um;v1;:::;vj islinearlyindependent,thisequation implies that all the a’s and b’s are 0. We now know that all the a’s, b’s, and c’s equal 0, as desired.
SECTION 2.C Dimension 47
2.43 Dimension of a sum
If U1 and U2 are subspaces of a finite-dimensional vector space, then dim.U1 CU2/DdimU1 CdimU2 dim.U1 \U2/:
48 CHAPTER 2 Finite-Dimensional Vector Spaces EXERCISES 2.C
1 Suppose V is finite-dimensional and U is a subspace of V such that dimU D dimV. Prove that U D V.
2 Show that the subspaces of R2 are precisely f0g, R2, and all lines in R2 through the origin.
3 Show that the subspaces of R3 are precisely f0g, R3, all lines in R3 through the origin, and all planes in R3 through the origin.
LetU Dfp2P4.F/Wp.6/D0g.FindabasisofU.
(b) Extend the basis in part (a) to a basis of P4.F/.
(c) Find a subspace W of P4.F/ such that P4.F/ D U ̊ W.
LetU Dfp2P4.R/Wp00.6/D0g.FindabasisofU.
(b) Extend the basis in part (a) to a basis of P4.R/.
(c) Find a subspace W of P4.R/ such that P4.R/ D U ̊ W.
LetU Dfp2P4.F/Wp.2/Dp.5/g.FindabasisofU.
(b) Extend the basis in part (a) to a basis of P4.F/.
(c) Find a subspace W of P4.F/ such that P4.F/ D U ̊ W.
LetU Dfp2P4.F/Wp.2/Dp.5/Dp.6/g.FindabasisofU.
(b) Extend the basis in part (a) to a basis of P4.F/.
(c) Find a subspace W of P4.F/ such that P4.F/ D U ̊ W.
4 (a)
5 (a)
6 (a)
7 (a)
LetU Dfp2P4.R/WR1 pD0g.FindabasisofU. 1
8 (a)
(b) Extend the basis in part (a) to a basis of P4.R/.
(c) Find a subspace W of P4.R/ such that P4.R/ D U ̊ W.
9 Suppose v1; : : : ; vm is linearly independent in V and w 2 V. Prove that dimspan.v1 Cw;:::;vm Cw/m1:
10 Suppose p0;p1;:::;pm 2 P.F/ are such that each pj has degree j. Prove that p0;p1;:::;pm is a basis of Pm.F/.
11 Suppose that U and W are subspaces of R8 such that dim U D 3, dim W D 5, and U C W D R8. Prove that R8 D U ̊ W.
SECTION 2.C Dimension 49
12 Suppose U and W are both five-dimensional subspaces of R9. Prove
thatU \W ¤f0g.
13 Suppose U and W are both 4-dimensional subspaces of C6. Prove that there exist two vectors in U \ W such that neither of these vectors is a scalar multiple of the other.
14 Suppose U1; : : : ; Um are finite-dimensional subspaces of V. Prove that U1 C C Um is finite-dimensional and
dim.U1 CCUm/dimU1 CCdimUm:
15 Suppose V is finite-dimensional, with dim V D n 1. Prove that there
exist 1-dimensional subspaces U1; : : : ; Un of V such that V DU1 ̊ ̊Un:
16 Suppose U1; : : : ; Um are finite-dimensional subspaces of V such that U1 CCUm is a direct sum. Prove that U1 ̊ ̊Um is finite- dimensional and
dimU1 ̊ ̊Um D dimU1 CCdimUm:
[The exercise above deepens the analogy between direct sums of sub- spaces and disjoint unions of subsets. Specifically, compare this exercise to the following obvious statement: if a set is written as a disjoint union of finite subsets, then the number of elements in the set equals the sum of the numbers of elements in the disjoint subsets.]
17 You might guess, by analogy with the formula for the number of ele- ments in the union of three subsets of a finite set, that if U1; U2; U3 are subspaces of a finite-dimensional vector space, then
dim.U1 C U2 C U3/
DdimU1 C dimU2 C dimU3
dim.U1 \ U2/ dim.U1 \ U3/ dim.U2 \ U3/ C dim.U1 \ U2 \ U3/:
Prove this or give a counterexample.
CHAPTER
3
Linear Maps
So far our attention has focused on vector spaces. No one gets excited about vector spaces. The interesting part of linear algebra is the subject to which we now turn—linear maps.
In this chapter we will frequently need another vector space, which we will call W, in addition to V. Thus our standing assumptions are now as follows:
German mathematician Carl Friedrich Gauss (1777–1855), who in 1809 published a method for solving systems of linear equations. This method, now called Gaussian elimination, was also used in a Chinese book published over 1600 years earlier.
3.1 Notation F, V, W
F denotes R or C.
V and W denote vector spaces over F.
LEARNING OBJECTIVES FOR THIS CHAPTER Fundamental Theorem of Linear Maps
the matrix of a linear map with respect to given bases isomorphic vector spaces
product spaces
quotient spaces
the dual space of a vector space and the dual of a linear map
© Springer International Publishing 2015 51 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__3
52 CHAPTER 3 Linear Maps
3.A The Vector Space of Linear Maps
Definition and Examples of Linear Maps
Now we are ready for one of the key definitions in linear algebra.
3.2 Definition linear map
AlinearmapfromV toW isafunctionTWV !W withthefollowing
properties:
additivity
T .u C v/ D T u C T v for all u; v 2 V ;
homogeneity
T .v/ D .T v/ for all 2 F and all v 2 V.
Note that for linear maps we often use the notation T v as well as the more standard functional notation T .v/.
Let’s look at some examples of linear maps. Make sure you verify that each of the functions defined below is indeed a linear map:
3.4 Example linear maps
zero
In addition to its other uses, we let the symbol 0 denote the function that takes each element of some vector space to the additive identity of another vector space. To be specific, 0 2 L.V; W / is defined by
0v D 0:
The 0 on the left side of the equation above is a function from V to W, whereas the 0 on the right side is the additive identity in W. As usual, the context
should allow you to distinguish between the many uses of the symbol 0.
identity
The identity map, denoted I, is the function on some vector space that takes each element to itself. To be specific, I 2 L.V; V / is defined by
Some mathematicians use the term linear transformation, which means the same as linear map.
3.3 Notation L.V; W /
The set of all linear maps from V to W is denoted L.V; W /.
Iv D v:
differentiation
Define D 2 LP.R/; P.R/ by
SECTION 3.A The Vector Space of Linear Maps 53
Dp D p0:
The assertion that this function is a linear map is another way of stating a basic result about differentiation: .f C g/0 D f 0 C g0 and .f /0 D f 0 whenever f; g are differentiable and is a constant.
integration DefineT2LP.R/;R by
Z1
0
The assertion that this function is linear is another way of stating a basic result about integration: the integral of the sum of two functions equals the sum of the integrals, and the integral of a constant times a function equals the constant times the integral of the function.
multiplication by x2
Define T 2 LP.R/; P.R/ by
.Tp/.x/ D x2p.x/
for x 2 R.
backward shift
Recall that F1 denotes the vector space of all sequences of elements of F. Define T 2 L.F1; F1/ by
T.x1;x2;x3;:::/ D .x2;x3;:::/: Define T 2 L.R3; R2/ by
Tp D
p.x/dx:
from R3 to R2
T .x; y; z/ D .2x y C 3z; 7x C 5y 6z/:
from Fn to Fm
Generalizing the previous example, let m and n be positive integers, let Aj;k 2 F for j D 1;:::;m and k D 1;:::;n, and define T 2 L.Fn;Fm/ by
T.x1;:::;xn/D.A1;1x1 CCA1;nxn;:::;Am;1x1 CCAm;nxn/: Actually every linear map from Fn to Fm is of this form.
The existence part of the next result means that we can find a linear map that takes on whatever values we wish on the vectors in a basis. The uniqueness part of the next result means that a linear map is completely determined by its values on a basis.
54 CHAPTER 3 Linear Maps
3.5 Linear maps and basis of domain
Suppose v1;:::;vn is a basis of V and w1;:::;wn 2 W. Then there exists auniquelinearmapTWV !W suchthat
for each j D 1;:::;n.
Tvj Dwj
Proof First we show the existence of a linear map T with the desired property. DefineTWV !W by
T.c1v1 CCcnvn/Dc1w1 CCcnwn;
where c1;:::;cn are arbitrary elements of F. The list v1;:::;vn is a basis of V, and thus the equation above does indeed define a function T from V to W (because each element of V can be uniquely written in the form c1v1 CCcnvn).
For each j, taking cj D 1 and the other c’s equal to 0 in the equation aboveshowsthatTvj Dwj.
Ifu;v2V withuDa1v1 CCanvn andvDc1v1 CCcnvn,then T.uCv/ D T.a Cc /v CC.a Cc /v
111 nnn
D.a1 Cc1/w1 CC.an Ccn/wn
D.a1w1 CCanwn/C.c1w1 CCcnwn/ D T u C T v:
Similarly, if 2 F and v D c1v1 C C cnvn, then
T.v/DT.c1v1 CCcnvn/ D c1w1 C C cnwn
D .c1w1 C C cnwn/ D T v:
ThusT isalinearmapfromV toW.
To prove uniqueness, now suppose that T 2 L.V; W / and that T vj D wj
for j D 1;:::;n. Let c1;:::;cn 2 F. The homogeneity of T implies that T.cjvj/Dcjwj forj D1;:::;n. TheadditivityofT nowimpliesthat
T.c1v1 CCcnvn/Dc1w1 CCcnwn:
Thus T is uniquely determined on span.v1; : : : ; vn/ by the equation above. Because v1; : : : ; vn is a basis of V, this implies that T is uniquely determined on V.
SECTION 3.A The Vector Space of Linear Maps 55 Algebraic Operations on L.V; W /
We begin by defining addition and scalar multiplication on L.V; W /.
3.6 Definition addition and scalar multiplication on L.V; W / SupposeS;T 2L.V;W/and2F. ThesumSCT andtheproduct
T are the linear maps from V to W defined by .SCT/.v/DSvCTv and .T/.v/D.Tv/
for all v 2 V.
You should verify that S CT and T as defined above are indeed linear maps. Inotherwords,ifS;T 2L.V;W/and 2F,thenSCT 2L.V;W/and T 2L.V;W/.
Because we took the trouble to de- fine addition and scalar multiplication on L.V; W /, the next result should not be a surprise.
The routine proof of the result above is left to the reader. Note that the additive identity of L.V; W / is the zero linear map defined earlier in this section.
Usually it makes no sense to multiply together two elements of a vector space, but for some pairs of linear maps a useful product exists. We will need a third vector space, so for the rest of this section suppose U is a vector space over F.
Although linear maps are perva- sive throughout mathematics, they are not as ubiquitous as imagined by some confused students who seem to think that cos is a linear map from R to R when they write that cos 2x equals 2 cos x and that cos.x C y/ equals cos x C cos y.
3.7 L.V; W / is a vector space
With the operations of addition and scalar multiplication as defined above, L.V; W / is a vector space.
3.8 Definition Product of Linear Maps
If T 2 L.U;V/ and S 2 L.V;W/, then the product ST 2 L.U;W/ is
defined by for u 2 U.
.ST /.u/ D S.T u/
56 CHAPTER 3 Linear Maps
In other words, S T is just the usual composition S ı T of two functions, but when both functions are linear, most mathematicians write ST instead of S ı T. You should verify that ST is indeed a linear map from U to W wheneverT 2L.U;V/andS 2L.V;W/.
Note that ST is defined only when T maps into the domain of S.
3.9 Algebraic properties of products of linear maps
associativity
.T1T2/T3 D T1.T2T3/
whenever T1, T2, and T3 are linear maps such that the products make sense (meaning that T3 maps into the domain of T2, and T2 maps into the domain of T1).
identity
TI D IT D T
whenever T 2 L.V; W / (the first I is the identity map on V, and the
second I is the identity map on W ). distributive properties
.S1 CS2/T DS1T CS2T and S.T1 CT2/DST1 CST2 wheneverT;T1;T2 2L.U;V/andS;S1;S2 2L.V;W/.
The routine proof of the result above is left to the reader.
Multiplication of linear maps is not commutative. In other words, it is not necessarily true that ST D TS, even if both sides of the equation make sense.
3.10 Example Suppose D 2 LP.R/; P.R/ is the differentiation map defined in Example 3.4 and T 2 LP.R/; P.R/ is the multiplication by x2 map defined earlier in this section. Show that TD ¤ DT.
Solution We have
.TD/p.x/ D x2p0.x/ but .DT /p.x/ D x2p0.x/ C 2xp.x/:
In other words, differentiating and then multiplying by x2 is not the same as multiplying by x2 and then differentiating.
SECTION 3.A The Vector Space of Linear Maps 57
3.11 Linear maps take 0 to 0
Suppose T is a linear map from V to W. Then T.0/ D 0.
Proof By additivity, we have
T.0/ D T.0C0/ D T.0/CT.0/:
Add the additive inverse of T .0/ to each side of the equation above to conclude that T .0/ D 0.
EXERCISES 3.A
1 Supposeb;c2R.DefineTWR3 !R2 by
T .x; y; z/ D .2x 4y C 3z C b; 6x C cxyz/:
ShowthatT islinearifandonlyifbDcD0.
2 Supposeb;c2R.DefineTWP.R/!R2 by Z2
x3p.x/ dx C c sin p.0/ : 3 Suppose T 2 L.Fn;Fm/. Show that there exist scalars Aj;k 2 F for
Tp D 3p.4/ C 5p0.6/ C bp.1/p.2/; ShowthatT islinearifandonlyifbDcD0.
j D 1;:::;m and k D 1;:::;n such that
T.x1;:::;xn/ D .A1;1x1CCA1;nxn;:::;Am;1x1CCAm;nxn/
for every .x1;:::;xn/ 2 Fn.
[The exercise above shows that T has the form promised in the last item of Example 3.4.]
4 SupposeT 2L.V;W/andv1;:::;vm isalistofvectorsinV suchthat Tv1;:::;Tvm is a linearly independent list in W. Prove that v1;:::;vm is linearly independent.
5 Prove the assertion in 3.7.
6 Prove the assertions in 3.9.
1
58 CHAPTER 3 Linear Maps
7 Show that every linear map from a 1-dimensional vector space to itself is multiplication by some scalar. More precisely, prove that if dim V D 1 and T 2 L.V;V/, then there exists 2 F such that Tv D v for all v 2 V.
8 Give an example of a function ' W R2 ! R such that '.av/ D a'.v/
foralla2Randallv2R2 but'isnotlinear.
[The exercise above and the next exercise show that neither homogeneity nor additivity alone is enough to imply that a function is a linear map.]
9 Give an example of a function ' W C ! C such that '.w C z/ D '.w/ C '.z/
for all w;z 2 C but ' is not linear. (Here C is thought of as a complex vector space.)
[There also exists a function ' W R ! R such that ' satisfies the additiv- ity condition above but ' is not linear. However, showing the existence of such a function involves considerably more advanced tools.]
10 SupposeU isasubspaceofV withU ¤V. SupposeS 2L.U;W/and S¤0(whichmeansthatSu¤0forsomeu2U).DefineTWV !W
by
(
TvD Sv ifv2U;
0 if v2V and v...U:
Prove that T is not a linear map on V.
11 Suppose V is finite-dimensional. Prove that every linear map on a subspace of V can be extended to a linear map on V. In other words, show that if U is a subspace of V and S 2 L.U; W /, then there exists T 2L.V;W/suchthatTuDSuforallu2U.
12 Suppose V is finite-dimensional with dim V > 0, and suppose W is infinite-dimensional. Prove that L.V; W / is infinite-dimensional.
13 Suppose v1; : : : ; vm is a linearly dependent list of vectors in V. Suppose also that W ¤ f0g. Prove that there exist w1;:::;wm 2 W such that no T 2L.V;W/satisfiesTvk Dwk foreachkD1;:::;m.
14 Suppose V is finite-dimensional with dim V 2. Prove that there exist S;T 2L.V;V/suchthatST ¤TS.
SECTION 3.B Null Spaces and Ranges 59 3.B Null Spaces and Ranges
Null Space and Injectivity
In this section we will learn about two subspaces that are intimately connected with each linear map. We begin with the set of vectors that get mapped to 0.
3.12 Definition null space, null T
For T 2 L.V; W /, the null space of T, denoted null T, is the subset of V
consisting of those vectors that T maps to 0: nullT Dfv2V WTvD0g:
3.13 Example null space
IfT isthezeromapfromV toW,inotherwordsifTvD0forevery
v2V,thennullT DV.
Suppose ‘ 2 L.C3;F/ is defined by ‘.z1;z2;z3/ D z1 C 2z2 C 3z3. Then null’ D f.z1;z2;z3/ 2 C3 W z1 C2z2 C3z3 D 0g. A basis of null ‘ is .2; 1; 0/; .3; 0; 1/.
Suppose D 2 LP.R/;P.R/ is the differentiation map defined by Dp D p0. The only functions whose derivative equals the zero function are the constant functions. Thus the null space of D equals the set of constant functions.
Suppose T 2 LP.R/; P.R/ is the multiplication by x2 map defined by .Tp/.x/ D x2p.x/. The only polynomial p such that x2p.x/ D 0 for all x 2 R is the 0 polynomial. Thus null T D f0g.
Suppose T 2 L.F1; F1/ is the backward shift defined by T.x1;x2;x3;:::/ D .x2;x3;:::/:
Clearly T.x1;x2;x3;:::/ equals 0 if and only if x2;x3;::: are all 0. Thus in this case we have nullT D f.a;0;0;:::/ W a 2 Fg.
The next result shows that the null space of each linear map is a subspace of the domain. In particular, 0 is in the null space of every linear map.
Some mathematicians use the term kernel instead of null space. The word “null” means zero. Thus the term “null space”should remind you of the connection to 0.
60 CHAPTER 3 Linear Maps
3.14 The null space is a subspace
Suppose T 2 L.V; W /. Then null T is a subspace of V.
Proof Because T is a linear map, we know that T .0/ D 0 (by 3.11). Thus 0 2 null T.
Suppose u; v 2 null T. Then
T .u C v/ D T u C T v D 0 C 0 D 0:
Hence u C v 2 null T. Thus null T is closed under addition. Suppose u 2 nullT and 2 F. Then
T .u/ D T u D 0 D 0:
Hence u 2 null T. Thus null T is closed under scalar multiplication.
We have shown that null T contains 0 and is closed under addition and scalar
multiplication. Thus null T is a sub- space of V (by 1.34).
As we will soon see, for a linear map the next definition is closely connected to the null space.
The definition above could be rephrased to say that T is injective if u ¤ v implies that T u ¤ T v. In other words, T is injective if it maps distinct inputs to distinct outputs.
The next result says that we can check whether a linear map is injective by checking whether 0 is the only vector that gets mapped to 0. As a simple application of this result, we see that of the linear maps whose null spaces we computed in 3.13, only multiplication by x2 is injective (except that the zero map is injective in the special case V D f0g).
Take another look at the null spaces that were computed in Example 3.13 and note that all of them are subspaces.
3.15 Definition injective
AfunctionTWV !W iscalledinjectiveifTuDTvimpliesuDv.
Many mathematicians use the term one-to-one, which means the same as injective.
SECTION 3.B Null Spaces and Ranges 61
3.16 Injectivity is equivalent to null space equals f0g
LetT 2L.V;W/. ThenT isinjectiveifandonlyifnullT Df0g.
Proof First suppose T is injective. We want to prove that null T D f0g. We already know that f0g null T (by 3.11). To prove the inclusion in the other direction, suppose v 2 null T. Then
T .v/ D 0 D T .0/:
Because T is injective, the equation above implies that v D 0. Thus we can conclude that null T D f0g, as desired.
To prove the implication in the other direction, now suppose null T D f0g. We want to prove that T is injective. To do this, suppose u; v 2 V and Tu D Tv. Then
0 D T u T v D T .u v/:
Thus u v is in null T, which equals f0g. Hence u v D 0, which implies
that u D v. Hence T is injective, as desired. Range and Surjectivity
Now we give a name to the set of outputs of a function.
3.17 Definition range
For T a function from V to W, the range of T is the subset of W consisting
of those vectors that are of the form T v for some v 2 V : rangeT DfTvWv2Vg:
3.18 Example range
IfT isthezeromapfromV toW,inotherwordsifTvD0forevery
v 2 V, then rangeT D f0g.
Suppose T 2 L.R2;R3/ is defined by T.x;y/ D .2x;5y;x C y/, then rangeT D f.2x;5y;x C y/ W x;y 2 Rg. A basis of rangeT is .2; 0; 1/; .0; 5; 1/.
Suppose D 2 LP.R/;P.R/ is the differentiation map defined by Dp D p0. Because for every polynomial q 2 P.R/ there exists a polynomial p 2 P.R/ such that p0 D q, the range of D is P.R/.
62 CHAPTER 3 Linear Maps
The next result shows that the range of each linear map is a subspace of the vector space into which it is being mapped.
Proof Suppose T 2 L.V; W /. Then T .0/ D 0 (by 3.11), which implies that 0 2 range T.
If w1;w2 2 rangeT, then there exist v1;v2 2 V such that Tv1 D w1 and T v2 D w2. Thus
T.v1 Cv2/ D Tv1 CTv2 D w1 Cw2:
Hence w1 C w2 2 range T. Thus range T is closed under addition. Ifw2rangeT and2F,thenthereexistsv2V suchthatTvDw.
Thus
T.v/ D Tv D w:
Hence w 2 range T. Thus range T is closed under scalar multiplication. We have shown that range T contains 0 and is closed under addition and
scalar multiplication. Thus range T is a subspace of W (by 1.34).
To illustrate the definition above, note that of the ranges we computed in 3.18, only the differentiation map is surjective (except that the zero map is surjective in the special case W D f0g.
Whether a linear map is surjective depends on what we are thinking of as the vector space into which it maps.
3.21 Example The differentiation map D 2 LP5.R/; P5.R/ defined by Dp D p0 is not surjective, because the polynomial x5 is not in the range of D. However, the differentiation map S 2 LP5.R/; P4.R/ defined by Sp D p0 is surjective, because its range equals P4.R/, which is now the vector space into which S maps.
Some mathematicians use the word image, which means the same as range.
3.19 The range is a subspace
IfT 2L.V;W/,thenrangeT isasubspaceofW.
3.20 Definition surjective
A function T W V ! W is called surjective if its range equals W.
Many mathematicians use the term onto, which means the same as sur- jective.
SECTION 3.B Null Spaces and Ranges 63 Fundamental Theorem of Linear Maps
The next result is so important that it gets a dramatic name.
3.22 Fundamental Theorem of Linear Maps
Suppose V is finite-dimensional and T 2 L.V; W /. Then range T is finite-dimensional and
dimV D dimnullT CdimrangeT:
Proof Letu1;:::;umbeabasisofnullT;thusdimnullT Dm.Thelinearly independent list u1; : : : ; um can be extended to a basis
u1;:::;um;v1;:::;vn
of V (by 2.33). Thus dim V D m C n. To complete the proof, we need only show that range T is finite-dimensional and dim range T D n. We will do this by proving that Tv1;:::;Tvn is a basis of rangeT.
Let v 2 V. Because u1;:::;um;v1;:::;vn spans V, we can write vDa1u1 CCamum Cb1v1 CCbnvn;
where the a’s and b’s are in F. Applying T to both sides of this equation, we get
Tv D b1Tv1 CCbnTvn;
where the terms of the form T uj disappeared because each uj is in null T. The last equation implies that T v1; : : : ; T vn spans range T. In particular, range T is finite-dimensional.
To show Tv1;:::;Tvn is linearly independent, suppose c1;:::;cn 2 F and
Then Hence
c1T v1 C C cnT vn D 0:
T .c1v1 C C cnvn/ D 0:
c1v1 CCcnvn 2 nullT: Because u1; : : : ; um spans null T, we can write
c1v1 CCcnvn Dd1u1 CCdmum;
where the d’s are in F. This equation implies that all the c’s (and d’s) are 0 (because u1;:::;um;v1;:::;vn is linearly independent). Thus Tv1;:::;Tvn is linearly independent and hence is a basis of range T, as desired.
64 CHAPTER 3 Linear Maps
Now we can show that no linear map from a finite-dimensional vector space to a “smaller” vector space can be injective, where “smaller” is measured by dimension.
Proof LetT2L.V;W/.Then
dimnullT D dimV dimrangeT dim V dim W
> 0;
where the equality above comes from the Fundamental Theorem of Linear Maps (3.22). The inequality above states that dim null T > 0. This means that null T contains vectors other than 0. Thus T is not injective (by 3.16).
The next result shows that no linear map from a finite-dimensional vector space to a “bigger” vector space can be surjective, where “bigger” is measured by dimension.
Proof LetT2L.V;W/.Then
dimrangeT D dimV dimnullT dim V
< dimW;
where the equality above comes from the Fundamental Theorem of Linear Maps (3.22). The inequality above states that dim range T < dim W. This means that range T cannot equal W. Thus T is not surjective.
As we will now see, 3.23 and 3.24 have important consequences in the theory of linear equations. The idea here is to express questions about systems of linear equations in terms of linear maps.
3.23 A map to a smaller dimensional space is not injective
Suppose V and W are finite-dimensional vector spaces such that dim V > dim W. Then no linear map from V to W is injective.
3.24 A map to a larger dimensional space is not surjective
Suppose V and W are finite-dimensional vector spaces such that dim V < dim W. Then no linear map from V to W is surjective.
SECTION 3.B Null Spaces and Ranges 65
3.25 Example Rephrase in terms of a linear map the question of whether a homogeneous system of linear equations has a nonzero solution.
Solution
Fix positive integers m and n, and let Aj;k 2 F for j D 1;:::;m and k D 1;:::;n. Consider the homoge- neous system of linear equations
Xn
A1;kxk D0
kD1
:
Xn
Am;kxk D0:
kD1
Obviously x1 D D xn D 0 is a solution of the system of equations above;
the question here is whether any other solutions exist. DefineTWFn !Fm by
Xn Xn T.x1;:::;xn/ D A1;kxk;:::; Am;kxk :
kD1 kD1
The equation T .x1; : : : ; xn/ D 0 (the 0 here is the additive identity in Fm,
namely, the list of length m of all 0’s) is the same as the homogeneous system of linear equations above.
Thus we want to know if null T is strictly bigger than f0g. In other words, we can rephrase our question about nonzero solutions as follows (by 3.16): What condition ensures that T is not injective?
Proof Use the notation and result from the example above. Thus T is a linear map from Fn to Fm, and we have a homogeneous system of m linear equations with n variables x1; : : : ; xn. From 3.23 we see that T is not injective if n > m.
Example of the result above: a homogeneous system of four linear equa- tions with five variables has nonzero solutions.
Homogeneous, in this context, means that the constant term on the right side of each equation below is 0.
3.26 Homogeneous system of linear equations
A homogeneous system of linear equations with more variables than equations has nonzero solutions.
66 CHAPTER 3 Linear Maps
3.27 Example Consider the question of whether an inhomogeneous sys- tem of linear equations has no solutions for some choice of the constant terms. Rephrase this question in terms of a linear map.
Solution Fix positive integers m and n, and let Aj;k 2 F for j D 1;:::;m andkD1;:::;n.Forc1;:::;cm 2F,considerthesystemoflinearequations
3.28
Xn
A1;kxk Dc1
kD1
: Am;kxk D cm:
Xn kD1
The question here is whether there is some choice of c1; : : : ; cm 2 F such that no solution exists to the system above.
DefineTWFn !Fm by
T.x1;:::;xn/ D A1;kxk;:::; Am;kxk :
Xn Xn kD1 kD1
The equation T .x1; : : : ; xn/ D .c1; : : : ; cm/ is the same as the system of equa- tions 3.28. Thus we want to know if range T ¤ Fm. Hence we can rephrase our question about not having a solution for some choice of c1; : : : ; cm 2 F as follows: What condition ensures that T is not surjective?
Proof Use the notation and result from the example above. Thus T is a lin- ear map from Fn to Fm, and we have a system of m equations with n variables x1;:::;xn. From 3.24 we see that T is not surjective if n < m.
Example of the result above: an inhomogeneous system of five linear equations with four variables has no solution for some choice of the con-
stant terms.
3.29 Inhomogeneous system of linear equations
An inhomogeneous system of linear equations with more equations than variables has no solution for some choice of the constant terms.
Our results about homogeneous systems with more variables than equations and inhomogeneous sys- tems with more equations than vari- ables (3.26 and 3.29) are often proved using Gaussian elimination. The abstract approach taken here leads to cleaner proofs.
EXERCISES 3.B
SECTION 3.B Null Spaces and Ranges 67
1 Give an example of a linear map T such that dimnullT D 3 and dimrangeT D2.
2 Suppose V is a vector space and S; T 2 L.V; V / are such that range S null T:
Provethat.ST/2 D0.
3 Supposev1;:::;vm isalistofvectorsinV. DefineT 2L.Fm;V/by
T.z1;:::;zm/Dz1v1 CCzmvm:
(a) What property of T corresponds to v1; : : : ; vm spanning V ?
(b) What property of T corresponds to v1;:::;vm being linearly independent?
4 Show that
fT 2 L.R5;R4/ W dimnullT > 2g is not a subspace of L.R5; R4/.
5 Give an example of a linear map T W R4 ! R4 such that range T D null T:
6 Prove that there does not exist a linear map T W R5 ! R5 such that range T D null T:
7 Suppose V and W are finite-dimensional with 2 dim V dim W. Show that fT 2 L.V; W / W T is not injectiveg is not a subspace of L.V;W/.
8 Suppose V and W are finite-dimensional with dim V dim W 2. Show that fT 2 L.V; W / W T is not surjectiveg is not a subspace of L.V;W/.
9 Suppose T 2 L.V; W / is injective and v1; : : : ; vn is linearly independent in V. Prove that T v1; : : : ; T vn is linearly independent in W.
68 CHAPTER 3 Linear Maps
10 Suppose v1;:::;vn spans V and T 2 L.V;W/. Prove that the list
Tv1;:::;Tvn spans rangeT.
11 Suppose S1; : : : ; Sn are injective linear maps such that S1S2 Sn
makes sense. Prove that S1S2 Sn is injective.
12 Suppose that V is finite-dimensional and that T 2 L.V; W /. Prove that there exists a subspace U of V such that U \ null T D f0g and rangeT DfTuWu2Ug.
13 Suppose T is a linear map from F4 to F2 such that
nullT D f.x1;x2;x3;x4/ 2 F4 W x1 D 5×2 and x3 D 7x4g:
Prove that T is surjective.
14 Suppose U is a 3-dimensional subspace of R8 and that T is a linear map
from R8 to R5 such that null T D U. Prove that T is surjective.
15 Prove that there does not exist a linear map from F5 to F2 whose null
space equals
f.x1;x2;x3;x4;x5/ 2 F5 W x1 D 3×2 and x3 D x4 D x5g:
16 Suppose there exists a linear map on V whose null space and range are both finite-dimensional. Prove that V is finite-dimensional.
17 Suppose V and W are both finite-dimensional. Prove that there exists an injective linear map from V to W if and only if dim V dim W.
18 Suppose V and W are both finite-dimensional. Prove that there exists a surjective linear map from V onto W if and only if dim V dim W.
19 Suppose V and W are finite-dimensional and that U is a subspace of V. ProvethatthereexistsT 2L.V;W/suchthatnullT DU ifandonlyif dimU dimV dimW.
20 Suppose W is finite-dimensional and T 2 L.V; W /. Prove that T is injective if and only if there exists S 2 L.W;V/ such that ST is the identity map on V.
21 Suppose V is finite-dimensional and T 2 L.V; W /. Prove that T is surjective if and only if there exists S 2 L.W;V/ such that TS is the identity map on W.
SECTION 3.B Null Spaces and Ranges 69
22 Suppose U and V are finite-dimensional vector spaces and S 2 L.V; W /
and T 2 L.U; V /. Prove that
dimnullST dimnullS CdimnullT:
23 Suppose U and V are finite-dimensional vector spaces and S 2 L.V; W / and T 2 L.U; V /. Prove that
dimrangeST minfdimrangeS;dimrangeTg:
24 Suppose W is finite-dimensional and T1; T2 2 L.V; W /. Prove that nullT1 nullT2 if and only if there exists S 2 L.W;W/ such that T2 D ST1.
25 Suppose V is finite-dimensional and T1; T2 2 L.V; W /. Prove that range T1 range T2 if and only if there exists S 2 L.V; V / such that T1 D T2S.
26 Suppose D 2 LP.R/; P.R/ is such that deg Dp D .deg p/ 1 for every nonconstant polynomial p 2 P.R/. Prove that D is surjective. [The notation D is used above to remind you of the differentiation map that sends a polynomial p to p0. Without knowing the formula for the derivative of a polynomial (except that it reduces the degree by 1), you can use the exercise above to show that for every polynomial q 2 P.R/, there exists a polynomial p 2 P.R/ such that p0 D q.]
27 Suppose p 2 P.R/. Prove that there exists a polynomial q 2 P.R/ such that 5q00 C 3q0 D p.
[This exercise can be done without linear algebra, but it’s more fun to do it using linear algebra.]
28 SupposeT 2L.V;W/,andw1;:::;wm isabasisofrangeT. Provethat there exist ‘1;:::;’m 2 L.V;F/ such that
Tv D ‘1.v/w1 CC’m.v/wm
for every v 2 V.
29 Suppose ‘ 2 L.V; F/. Suppose u 2 V is not in null ‘. Prove that
V Dnull’ ̊fauWa2Fg:
30 Suppose ‘1 and ‘2 are linear maps from V to F that have the same null
space. Show that there exists a constant c 2 F such that ‘1 D c’2.
31 Give an example of two linear maps T1 and T2 from R5 to R2 that have the same null space but are such that T1 is not a scalar multiple of T2.
70 CHAPTER 3 Linear Maps 3.C Matrices
Representing a Linear Map by a Matrix
Weknowthatifv1;:::;vn isabasisofV andTWV !W islinear,thenthe values of T v1; : : : ; T vn determine the values of T on arbitrary vectors in V (see 3.5). As we will soon see, matrices are used as an efficient method of recording the values of the T vj ’s in terms of a basis of W.
3.30 Definition matrix, Aj;k
Let m and n denote positive integers. An m-by-n matrix A is a rectangular
array of elements of F with m rows and n columns: 01
A1;1 ::: A1;n
A D B@ : : : : : : CA :
Am;1 ::: Am;n
The notation Aj;k denotes the entry in row j , column k of A. In other words, the first index refers to the row number and the second index refers to the column number.
Thus A2;3 refers to the entry in the second row, third column of a matrix A. 8 4 53i
3.31 Example IfAD 1 9 7 ,thenA2;3 D7. Now we come to the key definition in this section.
3.32 Definition matrix of a linear map, M.T /
SupposeT 2L.V;W/andv1;:::;vn isabasisofV andw1;:::;wm is a basis of W. The matrix of T with respect to these bases is the m-by-n matrix M.T / whose entries Aj;k are defined by
Tvk DA1;kw1CCAm;kwm:
If the bases are not clear from the context, then the notation MT;.v1;:::;vn/;.w1;:::;wm/ is used.
The matrix M.T / of a linear map T 2 L.V; W / depends on the basis v1;:::;vn of V and the basis w1;:::;wm of W, as well as on T. However, the bases should be clear from the context, and thus they are often not included in the notation.
To remember how M.T / is constructed from T, you might write across the top of the matrix the basis vectors v1; : : : ; vn for the domain and along the left the basis vectors w1; : : : ; wm for the vector space into which T maps, as follows:
0v1 :::vk :::vn 1
w1 A1;k M.T / D : B@ :
wm Am;k
In the matrix above only the kth col- umn is shown. Thus the second index of each displayed entry of the matrix above is k. The picture above should remind you that T vk can be computed from M.T / by multiplying each entry in the kth column by the correspond- ing wj from the left column, and then adding up the resulting vectors.
IfT isalinearmapfromFn toFm, then unless stated otherwise, assume the bases in question are the standard ones (where the kth basis vector is 1 in the kth slot and 0 in all the other slots). If you think of elements of Fm as columns of m numbers, then you can think of the kth column of M.T / as T applied to the kth standard basis vector.
CA :
SECTION 3.C Matrices 71
The kth column of M.T / con- sists of the scalars needed to write T vk as a linear combination of .w1;:::;wm/:
Xm jD1
Tvk D
Aj;kwj.
If T maps an n-dimensional vector space to an m-dimensional vector space, then M.T / is an m-by-n matrix.
3.33 Example Suppose T 2 L.F2; F3/ is defined by T .x; y/ D .x C 3y; 2x C 5y; 7x C 9y/:
Find the matrix of T with respect to the standard bases of F2 and F3. Solution Because T .1; 0/ D .1; 2; 7/ and T .0; 1/ D .3; 5; 9/, the matrix of
T with respect to the standard bases is the 3-by-2 matrix below:
01 31 M.T/D@2 5A:
79
72 CHAPTER 3 Linear Maps
When working with Pm.F/, use the standard basis 1; x; x2; : : : ; xm unless
the context indicates otherwise.
3.34 Example Suppose D 2 LP3.R/; P2.R/ is the differentiation map defined by Dp D p0. Find the matrix of D with respect to the standard bases of P3.R/ and P2.R/.
Solution Because .xn/0 D nxn1, the matrix of T with respect to the standard bases is the 3-by-4 matrix below:
001001 M.D/D@0 0 2 0A:
0003
Addition and Scalar Multiplication of Matrices
For the rest of this section, assume that V and W are finite-dimensional and that a basis has been chosen for each of these vector spaces. Thus for each linear map from V to W, we can talk about its matrix (with respect to the chosen bases, of course). Is the matrix of the sum of two linear maps equal to the sum of the matrices of the two maps?
Right now this question does not make sense, because although we have defined the sum of two linear maps, we have not defined the sum of two matrices. Fortunately, the obvious definition of the sum of two matrices has the right properties. Specifically, we make the following definition.
3.35 Definition matrix addition
The sum of two matrices of the same size is the matrix obtained by adding corresponding entries in the matrices:
0101
B@
CA C B@ 0ACC:::ACC1
A1;1 ::: A1;n : : : : : :
Am;1 ::: Am;n
C1;1 ::: C1;n
: : : : : : CA
In other words, .A C C/j;k D Aj;k C Cj;k.
Cm;1 ::: Cm;n 1;1 1;1
1;n 1;n
CA :
D B@ : : :
Am;1 C Cm;1
: : :
::: Am;n C Cm;n
SECTION 3.C Matrices 73 In the following result, the assumption is that the same bases are used for
all three linear maps S C T, S, and T.
The verification of the result above is left to the reader.
Still assuming that we have some bases in mind, is the matrix of a scalar times a linear map equal to the scalar times the matrix of the linear map? Again the question does not make sense, because we have not defined scalar multiplication on matrices. Fortunately, the obvious definition again has the right properties.
3.36 The matrix of the sum of linear maps
SupposeS;T 2L.V;W/.ThenM.SCT/DM.S/CM.T/.
3.37 Definition scalar multiplication of a matrix
The product of a scalar and a matrix is the matrix obtained by multiplying
each entry in the matrix by the scalar:
0101
A1;1 ::: A1;n
: : : : : : CA :
Am;1 ::: Am;n
A1;1 ::: A1;n B@ : : : : : :
CA D B@ In other words, .A/j;k D Aj;k.
Am;1 ::: Am;n
In the following result, the assumption is that the same bases are used for both linear maps T and T.
The verification of the result above is also left to the reader.
Because addition and scalar multiplication have now been defined for matrices, you should not be surprised that a vector space is about to appear. We need only a bit of notation so that this new vector space has a name.
3.38 The matrix of a scalar times a linear map
Suppose2FandT 2L.V;W/.ThenM.T/DM.T/.
3.39 Notation Fm;n
For m and n positive integers, the set of all m-by-n matrices with entries in F is denoted by Fm;n.
74 CHAPTER 3 Linear Maps
3.40 dimFm;nDmn
Suppose m and n are positive integers. With addition and scalar multipli- cation defined as above, Fm;n is a vector space with dimension mn.
Proof The verification that Fm;n is a vector space is left to the reader. Note that the additive identity of Fm;n is the m-by-n matrix whose entries all equal 0.
The reader should also verify that the list of m-by-n matrices that have 0 in all entries except for a 1 in one entry is a basis of Fm;n. There are mn such matrices, so the dimension of Fm;n equals mn.
Matrix Multiplication
Suppose, as previously, that v1;:::;vn is a basis of V and w1;:::;wm is a basis of W. Suppose also that we have another vector space U and that u1;:::;up is a basis of U.
Consider linear maps T W U ! V and S W V ! W. The composition ST is a linear map from U to W. Does M.ST / equal M.S/M.T /? This question does not yet make sense, because we have not defined the product of two matrices. We will choose a definition of matrix multiplication that forces this question to have a positive answer. Let’s see how to do this.
Suppose M.S/ D A and M.T/ D C. For 1 k p, we have
.ST/uk DS Xn
Xn Cr;kvr
rD1
D Cr;k S vr
rD1
Xn Xm
D Cr;k Aj;rwj
rD1 jD1
Xm Xn
Aj;rCr;k wj:
Thus M.ST/ is the m-by-p matrix whose entry in row j, column k, equals
Xn
Aj;rCr;k:
rD1
D
jD1 rD1
SECTION 3.C Matrices 75 Now we see how to define matrix multiplication so that the desired equation
M.S T / D M.S /M.T / holds.
3.41 Definition matrix multiplication
Suppose A is an m-by-n matrix and C is an n-by-p matrix. Then AC is defined to be the m-by-p matrix whose entry in row j , column k, is given by the following equation:
Xn rD1
In other words, the entry in row j, column k, of AC is computed by taking row j of A and column k of C , multiplying together corresponding entries, and then summing.
.AC/j;k D
Aj;rCr;k:
Note that we define the product of two matrices only when the number of columns of the first matrix equals the number of rows of the second matrix.
3.42 Example Here we multiply together a 3-by-2 matrix and a 2-by-4 matrix, obtaining a 3-by-4 matrix:
01 216 5 4 3 010 7 4 11 @34A 2101 D@2619125A:
5 6 42 31 20 9
Matrix multiplication is not commutative. In other words, AC is not necessarily equal to CA even if both products are defined (see Exercise 12). Matrix multiplication is distributive and associative (see Exercises 13 and 14).
In the following result, the assumption is that the same basis of V is used inconsideringT 2L.U;V/andS 2L.V;W/,thesamebasisofW isused inconsideringS 2L.V;W/andST 2L.U;W/,andthesamebasisofU is usedinconsideringT 2L.U;V/andST 2L.U;W/.
The proof of the result above is the calculation that was done as motivation before the definition of matrix multiplication.
You may have learned this defini- tion of matrix multiplication in an earlier course, although you may not have seen the motivation for it.
3.43 The matrix of the product of linear maps
IfT 2L.U;V/andS 2L.V;W/,thenM.ST/DM.S/M.T/.
76 CHAPTER 3 Linear Maps
In the next piece of notation, note that as usual the first index refers to a row and the second index refers to a column, with a vertically centered dot used as a placeholder.
3.44 Notation Aj; , A;k Suppose A is an m-by-n matrix.
If 1 j m, then Aj; denotes the 1-by-n matrix consisting of row j of A.
If 1 k n, then A;k denotes the m-by-1 matrix consisting of column k of A.
3.45 Example IfAD 8 4 5 ,thenA2; isrow2ofAandA;2 is 197
column 2 of A. In other words, A2;D1 9 7 and A;2D 49 :
The product of a 1-by-n matrix and an n-by-1 matrix is a 1-by-1 matrix. However, we will frequently identify a 1-by-1 matrix with its entry.
6
3.46 Example 3 4 2 D 26 because 3 6 C 4 2 D 26.
6 However, we can identify 26 with 26, writing 3 4 2 D 26.
Our next result gives another way to think of matrix multiplication: the entry in row j, column k, of AC equals (row j of A) times (column k of C).
3.47 Entry of matrix product equals row times column
Suppose A is an m-by-n matrix and C is an n-by-p matrix. Then
.AC /j;k D Aj; C;k for1j mand1kp.
The proof of the result above follows immediately from the definitions.
3.48 Example The result above and Example 3.46 show why the entry in row 2, column 1, of the product in Example 3.42 equals 26.
SECTION 3.C Matrices 77 The next result gives yet another way to think of matrix multiplication. It
states that column k of AC equals A times column k of C .
Again, the proof of the result above follows immediately from the defini- tions and is left to the reader.
3.50 Example From the result above and the equation 01215071
@3 4A 1 D@19A; 56 31
we see why column 2 in the matrix product in Example 3.42 is the right side of the equation above.
We give one more way of thinking about the product of an m-by-n matrix and an n-by-1 matrix. The following example illustrates this approach.
3.51 Example In the example above, the product of a 3-by-2 matrix and
a 2-by-1 matrix is a linear combination of the columns of the 3-by-2 matrix,
with the scalars that multiply the columns coming from the 2-by-1 matrix.
3.49 Column of matrix product equals matrix times column
Suppose A is an m-by-n matrix and C is an n-by-p matrix. Then .AC/;k DAC;k
for 1 k p.
Specifically,
071 011 021 @ 19 AD5@ 3 AC1@ 4 A:
31 5 6
The next result generalizes the example above. Again, the proof follows easily from the definitions and is left to the reader.
3.52 Linear combination of columns 0 1 c1
Suppose A is an m-by-n matrix and c D B@ : CA is an n-by-1 matrix. cn
Then
Ac D c1A;1 C C cnA;n:
In other words, Ac is a linear combination of the columns of A, with the scalars that multiply the columns coming from c.
78 CHAPTER 3 Linear Maps
Two more ways to think about matrix multiplication are given by Exercises
10 and 11.
EXERCISES 3.C
1 Suppose V and W are finite-dimensional and T 2 L.V; W /. Show that with respect to each choice of bases of V and W, the matrix of T has at least dim range T nonzero entries.
2 Suppose D 2 LP3.R/; P2.R/ is the differentiation map defined by Dp D p0. Find a basis of P3.R/ and a basis of P2.R/ such that the matrix of D with respect to these bases is
010001 @0100A:
0010
[Compare the exercise above to Example 3.34. The next exercise generalizes the exercise above.]
3 Suppose V and W are finite-dimensional and T 2 L.V; W /. Prove that there exist a basis of V and a basis of W such that with respect to these bases, all entries of M.T / are 0 except that the entries in row j , column j, equal 1 for 1 j dimrangeT.
4 Suppose v1; : : : ; vm is a basis of V and W is finite-dimensional. Suppose T 2L.V;W/. Provethatthereexistsabasisw1;:::;wn ofW suchthat all the entries in the first column of M.T / (with respect to the bases v1;:::;vm and w1;:::;wn) are 0 except for possibly a 1 in the first row, first column.
[In this exercise, unlike Exercise 3, you are given the basis of V instead of being able to choose a basis of V.]
5 Suppose w1; : : : ; wn is a basis of W and V is finite-dimensional. Suppose T 2 L.V;W/. Prove that there exists a basis v1;:::;vm of V such that all the entries in the first row of M.T / (with respect to the bases v1;:::;vm and w1;:::;wn) are 0 except for possibly a 1 in the first row, first column.
[In this exercise, unlike Exercise 3, you are given the basis of W instead of being able to choose a basis of W.]
6 Suppose V and W are finite-dimensional and T 2 L.V; W /. Prove that dimrangeT D 1 if and only if there exist a basis of V and a basis of W such that with respect to these bases, all entries of M.T / equal 1.
7 Verify 3.36.
8 Verify 3.38.
9 Prove 3.52.
10 Suppose A is an m-by-n matrix and C is an n-by-p matrix. Prove that .AC /j; D Aj; C
for1j m. Inotherwords,showthatrowjofACequals (row j of A) times C.
11 SupposeaD a a isa1-by-nmatrixandC isann-by-p 1n
matrix. Prove that
aC D a1C1; C C anCn; :
In other words, show that aC is a linear combination of the rows of C ,
with the scalars that multiply the rows coming from a.
12 Give an example with 2-by-2 matrices to show that matrix multiplication is not commutative. In other words, find 2-by-2 matrices A and C such that AC ¤ CA.
13 Prove that the distributive property holds for matrix addition and matrix multiplication. In other words, suppose A, B, C, D, E, and F are matrices whose sizes are such that A.B C C/ and .D C E/F make sense. Prove that AB C AC and DF C EF both make sense and that A.BCC/DABCAC and.DCE/F DDF CEF.
14 Prove that matrix multiplication is associative. In other words, suppose A, B, and C are matrices whose sizes are such that .AB/C makes sense. Prove that A.BC/ makes sense and that .AB/C D A.BC/.
15 Suppose A is an n-by-n matrix and 1 j; k n. Show that the entry in row j , column k, of A3 (which is defined to mean AAA) is
Xn Xn
Aj;pAp;rAr;k:
pD1rD1
SECTION 3.C Matrices 79
80 CHAPTER 3 Linear Maps
3.D Invertibility and Isomorphic Vector
Spaces
Invertible Linear Maps
We begin this section by defining the notions of invertible and inverse in the context of linear maps.
3.53 Definition invertible, inverse
A linear map T 2 L.V; W / is called invertible if there exists a linear map S 2 L.W; V / such that ST equals the identity map on V and TS equals the identity map on W.
A linear map S 2 L.W;V/ satisfying ST D I and TS D I is called an inverse of T (note that the first I is the identity map on V and the second I is the identity map on W ).
3.54 Inverse is unique
An invertible linear map has a unique inverse.
Proof Suppose T 2 L.V; W / is invertible and S1 and S2 are inverses of T. Then
S1 DS1I DS1.TS2/D.S1T/S2 DIS2 DS2: Thus S1 D S2.
Now that we know that the inverse is unique, we can give it a notation.
3.55 Notation T 1
If T is invertible, then its inverse is denoted by T 1. In other words, if T 2 L.V; W / is invertible, then T 1 is the unique element of L.W; V / suchthatT1T DI andTT1 DI.
The following result characterizes the invertible linear maps.
3.56 Invertibility is equivalent to injectivity and surjectivity
A linear map is invertible if and only if it is injective and surjective.
SECTION 3.D Invertibility and Isomorphic Vector Spaces 81
Proof Suppose T 2 L.V; W /. We need to show that T is invertible if and only if it is injective and surjective.
First suppose T is invertible. To show that T is injective, suppose u; v 2 V and Tu D Tv. Then
u D T1.Tu/ D T1.Tv/ D v;
so u D v. Hence T is injective.
We are still assuming that T is invertible. Now we want to prove that T is
surjective. To do this, let w 2 W. Then w D T .T 1w/, which shows that w is in the range of T. Thus range T D W. Hence T is surjective, completing this direction of the proof.
Now suppose T is injective and surjective. We want to prove that T is invertible. For each w 2 W, define Sw to be the unique element of V such that T.Sw/ D w (the existence and uniqueness of such an element follow from the surjectivity and injectivity of T ). Clearly T ı S equals the identity map on W.
To prove that S ı T equals the identity map on V, let v 2 V. Then T.SıT/vD.T ıS/.Tv/DI.Tv/DTv:
This equation implies that .S ı T /v D v (because T is injective). Thus S ı T equals the identity map on V.
To complete the proof, we need to show that S is linear. To do this, suppose w1, w2 2 W. Then
T.Sw1 CSw2/DT.Sw1/CT.Sw2/Dw1 Cw2:
ThusSw1 CSw2 istheuniqueelementofV thatT mapstow1 Cw2. By the definition of S, this implies that S.w1 C w2/ D Sw1 C Sw2. Hence S satisfies the additive property required for linearity.
The proof of homogeneity is similar. Specifically, if w 2 W and 2 F, then
T.Sw/ D T.Sw/ D w:
Thus Sw is the unique element of V that T maps to w. By the definition of
S, this implies that S.w/ D Sw. Hence S is linear, as desired. 3.57 Example linear maps that are not invertible
The multiplication by x2 linear map from P.R/ to P.R/ (see 3.4) is not invertible because it is not surjective (1 is not in the range).
The backward shift linear map from F1 to F1 (see 3.4) is not invertible because it is not injective [.1; 0; 0; 0; : : : / is in the null space].
82 CHAPTER 3 Linear Maps Isomorphic Vector Spaces
The next definition captures the idea of two vector spaces that are essentially the same, except for the names of the elements of the vector spaces.
3.58 Definition isomorphism, isomorphic
An isomorphism is an invertible linear map.
Two vector spaces are called isomorphic if there is an isomorphism from one vector space onto the other one.
ThinkofanisomorphismTWV !W asrelabelingv2V asTv2W.This viewpoint explains why two isomorphic vector spaces have the same vector space properties. The terms “isomorphism” and “invertible linear map” mean
the same thing. Use “isomorphism” when you want to emphasize that the two spaces are essentially the same.
Proof First suppose V and W are isomorphic finite-dimensional vector spaces. Thus there exists an isomorphism T from V onto W. Because T is invertible, we have null T D f0g and range T D W. Thus dim null T D 0 and dim range T D dim W. The formula
dimV D dimnullT CdimrangeT
(the Fundamental Theorem of Linear Maps, which is 3.22) thus becomes the equation dim V D dim W, completing the proof in one direction.
To prove the other direction, suppose V and W are finite-dimensional vector spaces with the same dimension. Let v1; : : : ; vn be a basis of V and w1;:::;wn beabasisofW. LetT 2L.V;W/bedefinedby
T.c1v1 CCcnvn/Dc1w1 CCcnwn:
Then T is a well-defined linear map because v1;:::;vn is a basis of V (see 3.5). Also, T is surjective because w1; : : : ; wn spans W. Furthermore, null T D f0g because w1; : : : ; wn is linearly independent; thus T is injective. Because T is injective and surjective, it is an isomorphism (see 3.56). Hence V and W are isomorphic, as desired.
The Greek word isos means equal; the Greek word morph means shape. Thus isomorphic literally means equal shape.
3.59 Dimension shows whether vector spaces are isomorphic
Two finite-dimensional vector spaces over F are isomorphic if and only if they have the same dimension.
SECTION 3.D Invertibility and Isomorphic Vector Spaces 83
The previous result implies that each finite-dimensional vector space V is iso- morphic to Fn, where n D dim V.
If v1;:::;vn is a basis of V and w1;:::;wm is a basis of W, then for each T 2 L.V;W/, we have a matrix M.T / 2 Fm;n. In other words, once bases have been fixed for V and W, M becomes a function from L.V; W / to Fm;n. Notice that 3.36 and 3.38 show that M is a linear map. This linear map is actually invertible, as we now show.
Proof We already noted that M is linear. We need to prove that M is injec- tive and surjective. Both are easy. We begin with injectivity. If T 2 L.V; W / and M.T/ D 0, then Tvk D 0 for k D 1;:::;n. Because v1;:::;vn is a basis of V, this implies T D 0. Thus M is injective (by 3.16).
To prove that M is surjective, suppose A 2 Fm;n. Let T be the linear map
from V to W such that
Tvk D
for k D 1;:::;n (see 3.5). Obviously M.T/ equals A, and thus the range of
M equals Fm;n, as desired.
Now we can determine the dimension of the vector space of linear maps
from one finite-dimensional vector space to another.
Because every finite-dimensional vector space is isomorphic to some Fn, why not just study Fn instead of more general vector spaces? To an- swer this question, note that an in- vestigation of Fn would soon lead to other vector spaces. For exam- ple, we would encounter the null space and range of linear maps. Al- though each of these vector spaces is isomorphic to some Fn, thinking of them that way often adds com- plexity but no new insight.
3.60 L.V; W / and Fm;n are isomorphic
Suppose v1;:::;vn is a basis of V and w1;:::;wm is a basis of W. Then M is an isomorphism between L.V; W / and Fm;n.
Xm jD1
Aj;kwj
3.61 dimL.V;W/D.dimV/.dimW/
Suppose V and W are finite-dimensional. Then L.V; W / is finite- dimensional and
dim L.V; W / D .dim V /.dim W /:
Proof This follows from 3.60, 3.59, and 3.40.
84 CHAPTER 3 Linear Maps
Linear Maps Thought of as Matrix Multiplication
Previously we defined the matrix of a linear map. Now we define the matrix of a vector.
3.62 Definition matrix of a vector, M.v/
Suppose v 2 V and v1;:::;vn is a basis of V. The matrix of v with
respect to this basis is the n-by-1 matrix 01
c1
M . v / D B@ : : : CA ;
cn where c1; : : : ; cn are the scalars such that
v D c1v1 C C cnvn:
The matrix M.v/ of a vector v 2 V depends on the basis v1;:::;vn of V, as well as on v. However, the basis should be clear from the context and thus it is not included in the notation.
3.63 Example matrix of a vector
The matrix of 2 7x C 5×3 with respect to the standard basis of P3.R/
is
021 B 7 C:
@0A 5
The matrix of a vector x 2 Fn with respect to the standard basis is
obtained by writing the coordinates of x as the entries in an n-by-1
matrix. In other words, if x D .x1;:::;xn/ 2 Fn, then 01
x1 M.x/ D B@ : CA :
xn
Occasionally we want to think of elements of V as relabeled to be n-by-1 matrices. Once a basis v1; : : : ; vn is chosen, the function M that takes v 2 V to M.v/ is an isomorphism of V onto Fn;1 that implements this relabeling.
SECTION 3.D Invertibility and Isomorphic Vector Spaces 85
Recall that if A is an m-by-n matrix, then A;k denotes the kth column of A, thought of as an m-by-1 matrix. In the next result, M.vk/ is computed with respect to the basis w1;:::;wm of W.
Proof The desired result follows immediately from the definitions of M.T / and M.vk/.
The next result shows how the notions of the matrix of a linear map, the matrix of a vector, and matrix multiplication fit together.
3.64 M.T/;k DM.vk/.
SupposeT 2L.V;W/andv1;:::;vn isabasisofV andw1;:::;wm is a basis of W. Let 1 k n. Then the kth column of M.T /, which is denoted by M.T /;k , equals M.vk /.
3.65 Linear maps act like matrix multiplication
SupposeT 2L.V;W/andv2V. Supposev1;:::;vn isabasisofV and w1;:::;wm is a basis of W. Then
M.T v/ D M.T /M.v/:
Proof
3.66
Hence
SupposevDc1v1 CCcnvn,wherec1;:::;cn 2F. Thus Tv D c1Tv1 CCcnTvn:
M.T v/ D c1M.T v1/ C C cnM.T vn/ D c1M.T/;1 CCcnM.T/;n D M.T /M.v/;
where the first equality follows from 3.66 and the linearity of M, the second equality comes from 3.64, and the last equality comes from 3.52.
Each m-by-n matrix A induces a linear map from Fn;1 to Fm;1, namely the matrix multiplication function that takes x 2 Fn;1 to Ax 2 Fm;1. The result above can be used to think of every linear map (from one finite-dimensional vector space to another finite-dimensional vector space) as a matrix multi- plication map after suitable relabeling via the isomorphisms given by M. Specifically, if T 2 L.V; W / and we identify v 2 V with M.v/ 2 Fn;1, then the result above says that we can identify T v with M.T /M.v/.
86 CHAPTER 3 Linear Maps
Because the result above allows us to think (via isomorphisms) of each linear map as multiplication on Fn;1 by some matrix A, keep in mind that the specific matrix A depends not only on the linear map but also on the choice of bases. One of the themes of many of the most important results in later chapters will be the choice of a basis that makes the matrix A as simple as possible.
In this book, we concentrate on linear maps rather than on matrices. How- ever, sometimes thinking of linear maps as matrices (or thinking of matrices as linear maps) gives important insights that we will find useful.
Operators
Linear maps from a vector space to itself are so important that they get a special name and special notation.
3.67 Definition operator, L.V /
A linear map from a vector space to itself is called an operator.
The notation L.V / denotes the set of all operators on V. In other words, L.V / D L.V; V /.
A linear map is invertible if it is injective and surjective. For an op- erator, you might wonder whether in- jectivity alone, or surjectivity alone, is enough to imply invertibility. On
infinite-dimensional vector spaces, neither condition alone implies invert- ibility, as illustrated by the next example, which uses two familiar operators from Example 3.4.
3.68 Example neither injectivity nor surjectivity implies invertibility
The multiplication by x2 operator on P.R/ is injective but not surjective. The backward shift operator on F1 is surjective but not injective.
In view of the example above, the next result is remarkable—it states that for operators on a finite-dimensional vector space, either injectivity or surjectivity alone implies the other condition. Often it is easier to check that an operator on a finite-dimensional vector space is injective, and then we get surjectivity for free.
The deepest and most important parts of linear algebra, as well as most of the rest of this book, deal with operators.
SECTION 3.D Invertibility and Isomorphic Vector Spaces 87
3.69 Injectivity is equivalent to surjectivity in finite dimensions
Suppose V is finite-dimensional and T 2 L.V /. Then the following are equivalent:
(a) T is invertible;
(b) T is injective;
(c) T is surjective.
Proof Clearly (a) implies (b).
Now suppose (b) holds, so that T is injective. Thus null T D f0g (by 3.16).
From the Fundamental Theorem of Linear Maps (3.22) we have dimrangeT D dimV dimnullT
D dim V:
Thus range T equals V. Thus T is surjective. Hence (b) implies (c).
Now suppose (c) holds, so that T is surjective. Thus range T D V. From
the Fundamental Theorem of Linear Maps (3.22) we have dimnullT D dimV dimrangeT
D 0:
Thus null T equals f0g. Thus T is injective (by 3.16), and so T is invertible (we already knew that T was surjective). Hence (c) implies (a), completing the proof.
The next example illustrates the power of the previous result. Although it is possible to prove the result in the example below without using linear algebra, the proof using linear algebra is cleaner and easier.
3.70 Example Show that for each polynomial q 2 P.R/, there exists a polynomial p 2 P.R/ with .x2 C 5x C 7/p00 D q.
Solution Example 3.68 shows that the magic of 3.69 does not apply to the infinite-dimensional vector space P.R/. However, each nonzero polynomial q has some degree m. By restricting attention to Pm.R/, we can work with a finite-dimensional vector space.
Suppose q 2 Pm.R/. Define T W Pm.R/ ! Pm.R/ by Tp D .x2 C 5x C 7/p00:
88 CHAPTER 3 Linear Maps
Multiplying a nonzero polynomial by .x2 C 5x C 7/ increases the degree by 2, and then differentiating twice reduces the degree by 2. Thus T is indeed an operator on Pm.R/.
Every polynomial whose second derivative equals 0 is of the form ax C b, where a; b 2 R. Thus null T D f0g. Hence T is injective.
Now 3.69 implies that T is surjective. Thus there exists a polynomial p 2 Pm.R/ such that .x2 C 5x C 7/p00 D q, as desired.
Exercise 30 in Section 6.A gives a similar but more spectacular application of 3.69. The result in that exercise is quite difficult to prove without using linear algebra.
EXERCISES 3.D
1 Suppose T 2 L.U; V / and S 2 L.V; W / are both invertible linear maps. Prove that ST 2 L.U; W / is invertible and that .ST /1 D T 1S1.
2 Suppose V is finite-dimensional and dim V > 1. Prove that the set of noninvertible operators on V is not a subspace of L.V /.
3 Suppose V is finite-dimensional, U is a subspace of V, and S 2 L.U; V /. Prove there exists an invertible operator T 2 L.V / such that T u D S u for every u 2 U if and only if S is injective.
4 Suppose W is finite-dimensional and T1; T2 2 L.V; W /. Prove that nullT1 D nullT2 if and only if there exists an invertible operator S 2L.W/suchthatT1 DST2.
5 Suppose V is finite-dimensional and T1; T2 2 L.V; W /. Prove that range T1 D range T2 if and only if there exists an invertible operator S 2L.V/suchthatT1 DT2S.
6 Suppose V and W are finite-dimensional and T1; T2 2 L.V; W /. Prove that there exist invertible operators R 2 L.V / and S 2 L.W / such that T1 D ST2R if and only if dimnullT1 D dimnullT2.
7 Suppose V and W are finite-dimensional. Let v 2 V. Let EDfT 2L.V;W/WTvD0g:
(a) Show that E is a subspace of L.V; W /.
(b) Suppose v ¤ 0. What is dim E?
SECTION 3.D Invertibility and Isomorphic Vector Spaces 89
8 Suppose V is finite-dimensional and T W V ! W is a surjective linear map of V onto W. Prove that there is a subspace U of V such that T jU is an isomorphism of U onto W. (Here T jU means the function T restricted to U. In other words, T jU is the function whose domain is U, withTjU definedbyTjU.u/DTuforeveryu2U.)
9 Suppose V is finite-dimensional and S; T 2 L.V /. Prove that ST is invertible if and only if both S and T are invertible.
10 SupposeV isfinite-dimensionalandS;T 2L.V/.ProvethatST DI ifandonlyifTS DI.
11 Suppose V is finite-dimensional and S; T; U 2 L.V / and ST U D I. Show that T is invertible and that T 1 D US.
12 Show that the result in the previous exercise can fail without the hypoth- esis that V is finite-dimensional.
13 Suppose V is a finite-dimensional vector space and R; S; T 2 L.V / are such that RST is surjective. Prove that S is injective.
14 Suppose v1;:::;vn is a basis of V. Prove that the map T W V ! Fn;1 defined by
Tv D M.v/
is an isomorphism of V onto Fn;1; here M.v/ is the matrix of v 2 V
with respect to the basis v1; : : : ; vn.
15 Prove that every linear map from Fn;1 to Fm;1 is given by a matrix multiplication. In other words, prove that if T 2 L.Fn;1;Fm;1/, then there exists an m-by-n matrix A such that T x D Ax for every x 2 Fn;1.
16 Suppose V is finite-dimensional and T 2 L.V /. Prove that T is a scalar multiple of the identity if and only if ST D TS for every S 2 L.V /.
17 Suppose V is finite-dimensional and E is a subspace of L.V / such that ST 2EandTS 2EforallS 2L.V/andallT 2E. Provethat E Df0gorE DL.V/.
18 Show that V and L.F; V / are isomorphic vector spaces.
19 Suppose T 2 LP.R/ is such that T is injective and deg Tp deg p
for every nonzero polynomial p 2 P.R/.
(a) Prove that T is surjective.
(b) Prove that deg Tp D deg p for every nonzero p 2 P.R/.
90 20
CHAPTER 3 Linear Maps
Suppose n is a positive integer and Ai;j 2 F for i;j D 1;:::;n. Prove that the following are equivalent (note that in both parts below, the number of equations equals the number of variables):
(a) The trivial solution x1 D D xn D 0 is the only solution to the homogeneous system of equations
Xn
A1;kxk D0
kD1
:
Xn
An;kxk D0:
kD1
(b) For every c1; : : : ; cn 2 F, there exists a solution to the system of equations
Xn
A1;kxk Dc1
kD1
:
Xn
An;kxk D cn:
kD1
SECTION 3.E Products and Quotients of Vector Spaces 91 3.E Products and Quotients of Vector Spaces
Products of Vector Spaces
As usual when dealing with more than one vector space, all the vector spaces in use should be over the same field.
3.71 Definition product of vector spaces Suppose V1; : : : ; Vm are vector spaces over F.
TheproductV1 Vm isdefinedby
V1 Vm Df.v1;:::;vm/Wv1 2V1;:::;vm 2Vmg:
AdditiononV1 Vm isdefinedby .u1;:::;um/C.v1;:::;vm/D.u1 Cv1;:::;um Cvm/:
Scalar multiplication on V1 Vm is defined by .v1;:::;vm/ D .v1;:::;vm/:
3.72 Example Elements of P2.R/ R3 are lists of length 2, with the first item in the list an element of P2.R/ and the second item in the list an elementofR3. 2 3
Forexample, 56xC4x ;.3;8;7/ 2P2.R/R .
The next result should be interpreted to mean that the product of vector spaces is a vector space with the operations of addition and scalar multiplica- tion as defined above.
The proof of the result above is left to the reader. Note that the additive identity of V1 Vm is .0;:::;0/, where the 0 in the jth slot is the additive identity of Vj . The additive inverse of .v1; : : : ; vm/ 2 V1 Vm is .v1;:::;vm/.
3.73 Product of vector spaces is a vector space
Suppose V1;:::;Vm are vector spaces over F. Then V1 Vm is a vector space over F.
92 CHAPTER 3 Linear Maps
3.74 Example Is R2 R3 equal to R5? Is R2 R3 isomorphic to R5?
Solution Elements of R2 R3 are lists .x1; x2/; .x3; x4; x5/, where x1;x2;x3;x4;x5 2R.
Elements of R5 are lists .x1; x2; x3; x4; x5/, where x1; x2; x3; x4; x5 2 R.
Although these look almost the same, they are not the same kind of object. Elements of R2 R3 are lists of length 2 (with the first item itself a list of length 2 and the second item a list of length 3), and elements of R5 are lists of length 5. Thus R2 R3 does not equal R5.
The linear map that takes a vector .x1; x2/; .x3; x4; x5/ 2 R2 R3 to .x1; x2; x3; x4; x5/ 2 R5 is clearly an isomorphism of R2 R3 onto R5. Thus these two vector spaces are isomorphic.
In this case, the isomorphism is so natural that we should think of it as a relabeling. Some people would even informally say that R2 R3 equals R5, which is not technically correct but which captures the spirit of identification via relabeling.
The next example illustrates the idea of the proof of 3.76.
3.75 Example Find a basis of P2.R/ R2.
Solution Consider this list of length 5 of elements of P2.R/ R2:
1; .0; 0/; x; .0; 0/; x2; .0; 0/; 0; .1; 0/; 0; .0; 1/:
The list above is linearly independent and it spans P2.R/ R2. Thus it is a
basis of P2.R/ R2.
3.76 Dimension of a product is the sum of dimensions
Suppose V1; : : : ; Vm are finite-dimensional vector spaces. Then V1 Vm is finite-dimensional and
dim.V1 Vm/DdimV1 CCdimVm:
Proof Choose a basis of each Vj . For each basis vector of each Vj , consider the element of V1 Vm that equals the basis vector in the jth slot and 0 in the other slots. The list of all such vectors is linearly independent and spansV1 Vm. ThusitisabasisofV1 Vm. Thelengthofthis basisisdimV1 CCdimVm,asdesired.
SECTION 3.E Products and Quotients of Vector Spaces 93 Products and Direct Sums
In the next result, the map is surjective by the definition of U1 C C Um. Thus the last word in the result below could be changed from “injective” to
“invertible”.
3.77 Products and direct sums
Suppose that U1; : : : ; Um are subspaces of V. Define a linear map W U1 Um ! U1 C C Um by
.u1;:::;um/Du1 CCum:
Then U1 C C Um is a direct sum if and only if is injective.
Proof The linear map is injective if and only if the only way to write 0 as a sumu1 CCum,whereeachuj isinUj,isbytakingeachuj equalto0. Thus 1.44 shows that is injective if and only if U1 C C Um is a direct sum, as desired.
3.78 A sum is a direct sum if and only if dimensions add up
Suppose V is finite-dimensional and U1; : : : ; Um are subspaces of V. Then U1 CCUm isadirectsumifandonlyif
dim.U1 CCUm/DdimU1 CCdimUm:
Proof The map in 3.77 is surjective. Thus by the Fundamental Theorem of Linear Maps (3.22), is injective if and only if
dim.U1 C C Um/ D dim.U1 Um/:
Combining 3.77 and 3.76 now shows that U1 C C Um is a direct sum if
and only if
as desired.
dim.U1 CCUm/DdimU1 CCdimUm;
In the special case m D 2, an alternative proof that U1 C U2 is a direct sum if and only if dim.U1 C U2/ D dimU1 C dimU2 can be obtained by combining 1.45 and 2.43.
94 CHAPTER 3 Linear Maps Quotients of Vector Spaces
We begin our approach to quotient spaces by defining the sum of a vector and a subspace.
3.79 Definition v C U
Supposev2V andU isasubspaceofV.ThenvCU isthesubsetofV defined by
vCU DfvCuWu2Ug:
3.80 Example Suppose
U Df.x;2x/2R2 Wx2Rg:
Then U is the line in R2 through the origin with slope 2. Thus
.17;20/CU
is the line in R2 that contains the point
.17; 20/ and has slope 2.
20
10, 20
U
10
17, 20
17, 20 U
17
3.81 Definition affine subset, parallel
AnaffinesubsetofV isasubsetofV oftheformvCU forsome
v2V andsomesubspaceU ofV.
Forv2V andU asubspaceofV,theaffinesubsetvCU issaidto be parallel to U.
3.82 Example parallel affine subsets
In Example 3.80 above, all the lines in R2 with slope 2 are parallel to U.
If U D f.x;y;0/ 2 R3 W x;y 2 Rg, then the affine subsets of R3 parallel to U are the planes in R3 that are parallel to the xy-plane U in the usual sense.
Important: With the definition of parallel given in 3.81, no line in R3 is considered to be an affine subset that is parallel to the plane U.
SECTION 3.E Products and Quotients of Vector Spaces 95
3.83 Definition quotient space, V=U
Suppose U is a subspace of V. Then the quotient space V=U is the set of
all affine subsets of V parallel to U. In other words, V=U DfvCU Wv2Vg:
3.84 Example quotient spaces
If U D f.x;2x/ 2 R2 W x 2 Rg, then R2=U is the set of all lines in
R2 that have slope 2.
If U is a line in R3 containing the origin, then R3=U is the set of all
lines in R3 parallel to U.
If U is a plane in R3 containing the origin, then R3=U is the set of all
planes in R3 parallel to U.
Our next goal is to make V=U into a vector space. To do this, we will
need the following result.
3.85 Two affine subsets parallel to U are equal or disjoint Suppose U is a subspace of V and v;w 2 V. Then the following are
equivalent:
(a) vw2U;
(b) vCUDwCU;
(c) .v C U / \ .w C U / ¤ ¿.
Proof Firstsuppose(a)holds,sovw2U. Ifu2U,then v C u D w C .v w/ C u 2 w C U:
ThusvCU wCU. Similarly,wCU vCU. ThusvCU DwCU, completing the proof that (a) implies (b).
Obviously (b) implies (c).
Now suppose (c) holds, so .v C U/ \ .w C U/ ¤ ¿. Thus there exist u1;u2 2U suchthat
vCu1 DwCu2:
ThusvwDu2 u1. Hencevw2U,showingthat(c)implies(a)and completing the proof.
96 CHAPTER 3 Linear Maps
Now we can define addition and scalar multiplication on V=U.
3.86 Definition addition and scalar multiplication on V=U Suppose U is a subspace of V. Then addition and scalar multiplication
are defined on V=U by
.v C U / C .w C U / D .v C w/ C U .v C U / D .v/ C U
forv;w2V and2F.
As part of the proof of the next result, we will show that the definitions above make sense.
Proof The potential problem with the definitions above of addition and scalar multiplication on V=U is that the representation of an affine subset parallel to U is not unique. Specifically, suppose v; w 2 V. Suppose also that vO; wO 2 V are such that v C U D vO C U and w C U D wO C U. To show that the definition of addition on V=U given above makes sense, we must show that .vCw/CU D.vOCwO/CU.
By 3.85, we have
vvO2U and wwO2U:
Because U is a subspace of V and thus is closed under addition, this implies that.vvO/C.wwO/2U. Thus.vCw/.vOCwO/2U. Using3.85again, we see that
. v C w / C U D . vO C wO / C U ;
as desired. Thus the definition of addition on V=U makes sense.
Similarly, suppose 2 F. Because U is a subspace of V and thus is closed under scalar multiplication, we have .v vO/ 2 U. Thus v vO 2 U. Hence 3.85 implies that .v/ C U D .vO/ C U. Thus the definition of scalar
multiplication on V=U makes sense.
Now that addition and scalar multiplication have been defined on V=U, the
verification that these operations make V=U into a vector space is straightfor- ward and is left to the reader. Note that the additive identity of V = U is 0 C U (which equals U ) and that the additive inverse of v C U is .v/ C U.
3.87 Quotient space is a vector space
Suppose U is a subspace of V. Then V=U, with the operations of addition and scalar multiplication as defined above, is a vector space.
SECTION 3.E Products and Quotients of Vector Spaces 97 The next concept will give us an easy way to compute the dimension
of V=U.
3.88 Definition quotient map,
Suppose U is a subspace of V. The quotient map is the linear map
WV !V=U definedby for v 2 V.
.v/ D v C U
The reader should verify that is indeed a linear map. Although depends on U as well as V, these spaces are left out of the notation because they should be clear from the context.
3.89 Dimension of a quotient space
Suppose V is finite-dimensional and U is a subspace of V. Then dimV=U DdimV dimU:
Proof Let be the quotient map from V to V=U. From 3.85, we see that null D U. Clearly range D V = U. The Fundamental Theorem of Linear Maps (3.22) thus tells us that
dimV DdimUCdimV=U; which gives the desired result.
Each linear map T on V induces a linear map TQ on V =.null T /, which we now define.
To show that the definition of TQ makes sense, suppose u; v 2 V are such thatuCnullT DvCnullT. By3.85,wehaveuv2nullT. Thus T.u v/ D 0. Hence Tu D Tv. Thus the definition of TQ indeed makes sense.
3.90 Definition TQ
SupposeT 2L.V;W/.DefineTQWV=.nullT/!W by
TQ .v C null T / D T v:
98 CHAPTER 3 Linear Maps
3.91 Null space and range of TQ SupposeT 2L.V;W/.Then
(a) TQ isalinearmapfromV=.nullT/toW;
(b) TQ is injective;
(c) range TQ D range T ;
(d) V =.null T / is isomorphic to range T.
Proof
(a) The routine verification that TQ is linear is left to the reader.
(b) Supposev2V andTQ.vCnullT/D0.ThenTvD0.Thusv2nullT. Hence 3.85 implies that v C null T D 0 C null T. This implies that null TQ D 0, and hence TQ is injective, as desired.
(c) The definition of TQ shows that range TQ D range T.
(d) Parts (b) and (c) imply that if we think of TQ as mapping into range T,
then TQ is an isomorphism from V =.null T / onto range T.
EXERCISES 3.E
1 Suppose T is a function from V to W. The graph of T is the subset of V W defined by
graphofT Df.v;Tv/2V W Wv2Vg:
Prove that T is a linear map if and only if the graph of T is a subspace ofV W.
[Formally,afunctionT fromV toW isasubsetT of V W suchthat for each v 2 V, there exists exactly one element .v; w/ 2 T. In other words, formally a function is what is called above its graph. We do not usually think of functions in this formal manner. However, if we do become formal, then the exercise above could be rephrased as follows: ProvethatafunctionT fromV toW isalinearmapifandonlyif T is a subspace of V W.]
SECTION 3.E Products and Quotients of Vector Spaces 99
2 SupposeV1;:::;Vm arevectorspacessuchthatV1 Vm isfinite-
dimensional. Prove that Vj is finite-dimensional for each j D 1; : : : ; m.
3 Give an example of a vector space V and subspaces U1; U2 of V such
thatU1 U2 isisomorphictoU1 CU2 butU1 CU2 isnotadirectsum. Hint: The vector space V must be infinite-dimensional.
4 SupposeV1;:::;Vmarevectorspaces.ProvethatL.V1Vm;W/ and L.V1; W / L.Vm; W / are isomorphic vector spaces.
5 Suppose W1;:::;Wm are vector spaces. Prove that L.V;W1 Wm/ and L.V; W1/ L.V; Wm/ are isomorphic vector spaces.
6 For n a positive integer, define V n by VnDVV:
„ ƒ‚ …
n times
Prove that V n and L.Fn; V / are isomorphic vector spaces.
7 Suppose v; x are vectors in V and U; W are subspaces of V such that vCU DxCW. ProvethatU DW.
8 Prove that a nonempty subset A of V is an affine subset of V if and only if v C .1 /w 2 A for all v; w 2 A and all 2 F.
9 Suppose A1 and A2 are affine subsets of V. Prove that the intersection A1 \ A2 is either an affine subset of V or the empty set.
10 Prove that the intersection of every collection of affine subsets of V is either an affine subset of V or the empty set.
11 Suppose v1;:::;vm 2 V. Let
ADf1v1CCmvm W1;:::;m 2Fand1CCm D1g:
(a) Prove that A is an affine subset of V.
(b) Prove that every affine subset of V that contains v1; : : : ; vm also
contains A.
(c) ProvethatADvCU forsomev2V andsomesubspaceU of
V withdimU m1.
12 Suppose U is a subspace of V such that V=U is finite-dimensional. Prove that V is isomorphic to U .V=U/.
100 CHAPTER 3 Linear Maps
13 Suppose U is a subspace of V and v1 CU;:::;vm CU is a basis of V=U and u1;:::;un is a basis of U. Prove that v1;:::;vm;u1;:::;un is a basis of V.
14 Suppose U D f.x1;x2;:::/ 2 F1 W xj ¤ 0 for only finitely many jg.
(a) Show that U is a subspace of F1.
(b) Prove that F1=U is infinite-dimensional.
15 Suppose ‘ 2 L.V;F/ and ‘ ¤ 0. Prove that dimV=.null’/ D 1.
16 Suppose U is a subspace of V such that dimV=U D 1. Prove that there exists ‘ 2 L.V; F/ such that null ‘ D U.
17 Suppose U is a subspace of V such that V=U is finite-dimensional. Prove that there exists a subspace W of V such that dim W D dim V = U and V D U ̊ W.
18 Suppose T 2 L.V;W/ and U is a subspace of V. Let denote the quotientmapfromV ontoV=U.ProvethatthereexistsS2L.V=U;W/ suchthatT DSıifandonlyifU nullT.
19 Find a correct statement analogous to 3.78 that is applicable to finite sets, with unions analogous to sums of subspaces and disjoint unions analogous to direct sums.
20 SupposeU isasubspaceofV.DefineWL.V=U;W/!L.V;W/by .S/ D S ı :
(a) Show that is a linear map.
(b) Show that is injective.
(c) Showthatrange DfT 2L.V;W/WTuD0foreveryu2Ug.
0
SECTION 3.F Duality 101
3.F Duality
The Dual Space and the Dual Map
Linear maps into the scalar field F play a special role in linear algebra, and thus they get a special name:
3.93 Example linear functionals
Define’WR3 !Rby’.x;y;z/D4x5yC2z.Then’isalinear
functional on R3.
Fix .c1;:::;cn/ 2 Fn. Define ‘W Fn ! F by ‘.x1;:::;xn/Dc1x1 CCcnxn:
Then ‘ is a linear functional on Fn.
Define’WP.R/!Rby’.p/D3p00.5/C7p.4/.Then’isalinear
functional on P.R/.
Define ‘W P.R/ ! R by ‘.p/ D R1p.x/dx. Then ‘ is a linear
functional on P.R/.
The vector space L.V; F/ also gets a special name and special notation:
3.92 Definition linear functional
A linear functional on V is a linear map from V to F. In other words, a linear functional is an element of L.V; F/.
3.94 Definition dual space, V 0
The dual space of V, denoted V0, is the vector space of all linear functionals on V. In other words, V 0 D L.V; F/.
3.95 dimV0 DdimV
Suppose V is finite-dimensional. Then V 0 is also finite-dimensional and dimV0 DdimV.
Proof This result follows from 3.61.
102 CHAPTER 3 Linear Maps
In the following definition, 3.5 implies that each ‘j is well defined.
3.96 Definition dual basis
If v1;:::;vn is a basis of V, then the dual basis of v1;:::;vn is the list ‘1; : : : ; ‘n of elements of V 0, where each ‘j is the linear functional on V
such that
(
‘j.vk/D 1 ifkDj; 0 ifk¤j:
3.97 Example What is the dual basis of the standard basis e1;:::;en of Fn?
Solution For 1 j n, define ‘j to be the linear functional on Fn that selects the j th coordinate of a vector in Fn . In other words,
‘j.x1;:::;xn/ D xj
for .x1;:::;xn/ 2 Fn. Clearly
(
‘j.ek/D 1 ifkDj; 0 ifk¤j:
Thus ‘1;:::;’n is the dual basis of the standard basis e1;:::;en of Fn.
The next result shows that the dual basis is indeed a basis. Thus the
terminology “dual basis” is justified.
Proof Suppose v1;:::;vn is a basis of V. Let ‘1;:::;’n denote the dual basis.
To show that ‘1; : : : ; ‘n is a linearly independent list of elements of V 0, suppose a1;:::;an 2 F are such that
a1’1 C C an’n D 0:
Now .a1’1 C C an’n/.vj/ D aj for j D 1;:::;n. The equation above thus shows that a1 D D an D 0. Hence ‘1;:::;’n is linearly independent.
Now2.39and3.95implythat’1;:::;’n isabasisofV0.
3.98 Dual basis is a basis of the dual space
Suppose V is finite-dimensional. Then the dual basis of a basis of V is a basis of V 0.
SECTION 3.F Duality 103 In the definition below, note that if T is a linear map from V to W then T 0
isalinearmapfromW0 toV0.
If T 2 L.V;W/ and ‘ 2 W0, then T0.’/ is defined above to be the composition of the linear maps ‘ and T. Thus T 0.’/ is indeed a linear map fromV toF;inotherwords,T0.’/2V0.
The verification that T 0 is a linear map from W 0 to V 0 is easy: If’; 2W0,then
T0.’C /D.’C /ıT D’ıT C ıT DT0.’/CT0. /: If 2 F and ‘ 2 W 0, then
T0.’/ D .’/ıT D .’ ıT/ D T0.’/:
In the next example, the prime notation is used with two unrelated mean- ings: D0 denotes the dual of a linear map D, and p0 denotes the derivative of a polynomial p.
3.99 Definition dual map, T 0
IfT 2L.V;W/,thenthedualmapofT isthelinearmapT0 2L.W0;V0/ definedbyT0.’/D’ıT for’2W0.
3.100
Example DefineDWP.R/!P.R/byDpDp0.
Suppose ‘ is the linear functional on P.R/ defined by ‘.p/ D p.3/.
Then D0.’/ is the linear functional on P.R/ given by D0.’/.p/ D .’ ı D/.p/ D ‘.Dp/ D ‘.p0/ D p0.3/:
In other words, D0.’/ is the linear functional on P.R/ that takes p to p0.3/.
Suppose ‘ is the linear functional on P.R/ defined by ‘.p/ D R 1 p. 00
Then D .’/ is the linear functional on P.R/ given by
Z1 D0.’/ .p/ D .’ıD/.p/ D ‘.Dp/ D ‘.p0/ D
0
In other words, D0.’/ is the linear functional on P.R/ that takes p to p.1/ p.0/.
p0 D p.1/p.0/:
104 CHAPTER 3 Linear Maps
The first two bullet points in the result below imply that the function that takesT toT0 isalinearmapfromL.V;W/toL.W0;V0/.
In the third bullet point below, note the reversal of order from ST on the left to T 0S0 on the right (here we assume that U is a vector space over F).
3.101 Algebraic properties of dual maps
.S C T /0 D S0 C T 0 for all S; T 2 L.V; W /.
.T/0 DT0 forall2FandallT 2L.V;W/.
.ST/0 DT0S0 forallT 2L.U;V/andallS 2L.V;W/.
Proof The proofs of the first two bullet points above are left to the reader. To prove the third bullet point, suppose ‘ 2 W 0. Then
.ST /0.’/ D ‘ı.ST / D .’ıS/ıT D T 0.’ıS/ D T 0S0.’/ D .T 0S0/.’/;
where the first, third, and fourth equal- ities above hold because of the defini- tion of the dual map, the second equality holds because composition of functions is associative, and the last equality fol- lows from the definition of composition.
The equality of the first and last terms above for all ‘ 2 W 0 means that .ST/0 DT0S0.
The Null Space and Range of the Dual of a Linear Map
Our goal in this subsection is to describe null T 0 and range T 0 in terms of range T and null T. To do this, we will need the following definition.
3.103 Example Suppose U is the subspace of P.R/ consisting of all polynomial multiples of x2. If ‘ is the linear functional on P.R/ defined by ‘.p/ D p0.0/, then ‘ 2 U0.
Some books use the notation V and T for duality instead of V 0 and T 0. However, here we reserve the notation T for the adjoint, which will be introduced when we study linear maps on inner product spaces in Chapter 7.
3.102 Definition annihilator, U 0
For U V, the annihilator of U, denoted U 0, is defined by
U0 Df’2V0 W’.u/D0forallu2Ug:
ForU V,theannihilatorU0 isasubsetofthedualspaceV0.ThusU0 depends on the vector space containing U, so a notation such as UV0 would be more precise. However, the containing vector space will always be clear from the context, so we will use the simpler notation U 0.
3.104 Example Let e1; e2; e3; e4; e5 denote the standard basis of R5, and let ‘1; ‘2; ‘3; ‘4; ‘5 denote the dual basis of .R5/0. Suppose
U D span.e1;e2/ D f.x1;x2;0;0;0/ 2 R5 W x1;x2 2 Rg: Show that U 0 D span.’3; ‘4; ‘5/.
Solution Recall (see 3.97) that ‘j is the linear functional on R5 that selects that j th coordinate: ‘j .x1; x2; x3; x4; x5/ D xj .
First suppose ‘ 2 span.’3; ‘4; ‘5/. Then there exist c3; c4; c5 2 R such that ‘ D c3’3 C c4’4 C c5’5. If .x1; x2; 0; 0; 0/ 2 U, then
‘.x1;x2;0;0;0/D.c3’3 Cc4’4 Cc5’5/.x1;x2;0;0;0/D0:
Thus ‘ 2 U 0. In other words, we have shown that span.’3; ‘4; ‘5/ U 0. To show the inclusion in the other direction, suppose ‘ 2 U 0. Because the dual basis is a basis of .R5/0, there exist c1; c2; c3; c4; c5 2 R such that ‘ Dc1’1 Cc2’2 Cc3’3 Cc4’4 Cc5’5. Becausee1 2U and’ 2U0,we
have
0D’.e1/D.c1’1 Cc2’2 Cc3’3 Cc4’4 Cc5’5/.e1/Dc1: Similarly, e2 2 U and thus c2 D 0. Hence ‘ D c3’3 C c4’4 C c5’5. Thus
‘ 2 span.’3; ‘4; ‘5/, which shows that U 0 span.’3; ‘4; ‘5/.
Proof Clearly 0 2 U 0 (here 0 is the zero linear functional on V ), because the zero linear functional applied to every vector in U is 0.
Suppose’; 2U0. Thus’; 2V0 and’.u/D .u/D0forevery u2U. Ifu2U,then.’C /.u/D’.u/C .u/D0C0D0. Thus ‘C 2U0.
Similarly, U 0 is closed under scalar multiplication. Thus 1.34 implies that U0 isasubspaceofV0.
SECTION 3.F Duality 105
3.105 The annihilator is a subspace
SupposeU V. ThenU0 isasubspaceofV0.
106 CHAPTER 3 Linear Maps
The next result shows that dim U 0 is the difference of dim V and dim U. For example, this shows that if U is a 2-dimensional subspace of R5, then U 0 is a 3-dimensional subspace of .R5/0, as in Example 3.104.
The next result can be proved following the pattern of Example 3.104: choose a basis u1;:::;um of U, extend to a basis u1;:::;um;:::;un of V, let’1;:::;’m;:::;’n bethedualbasisofV0,andthenshow’mC1;:::;’n is a basis of U 0, which implies the desired result.
You should construct the proof outlined in the paragraph above, even though a slicker proof is presented here.
Proof Leti 2L.U;V/betheinclusionmapdefinedbyi.u/Duforu2U. Thus i 0 is a linear map from V 0 to U 0. The Fundamental Theorem of Linear Maps (3.22) applied to i0 shows that
dimrangei0 Cdimnulli0 D dimV0:
However, null i 0 D U 0 (as can be seen by thinking about the definitions) and
dim V 0 D dim V (by 3.95), so we can rewrite the equation above as dimrangei0 CdimU0 D dimV:
If ‘ 2 U 0, then ‘ can be extended to a linear functional on V (see, for example, Exercise 11 in Section 3.A). The definition of i0 shows that i0. / D ‘. Thus ‘ 2 rangei0, which implies that rangei0 D U0. Hence dim range i 0 D dim U 0 D dim U, and the displayed equation above becomes the desired result.
The proof of part (a) of the result below does not use the hypothesis that V and W are finite-dimensional.
3.106 Dimension of the annihilator
Suppose V is finite-dimensional and U is a subspace of V. Then dim U C dim U 0 D dim V:
3.107 The null space of T 0
Suppose V and W are finite-dimensional and T 2 L.V; W /. Then
(a) nullT0 D.rangeT/0;
(b) dimnullT0 DdimnullT CdimW dimV.
Proof
(a) First suppose ‘ 2 null T 0. Thus 0 D T 0.’/ D ‘ ı T. Hence 0D.’ıT/.v/D’.Tv/ foreveryv2V:
Thus ‘ 2 .range T /0. This implies that null T 0 .range T /0.
To prove the inclusion in the opposite direction, now suppose that ‘ 2 .rangeT/0. Thus ‘.Tv/ D 0 for every vector v 2 V. Hence 0D’ıT DT0.’/. In other words, ‘ 2 nullT0, which shows that .range T /0 null T 0, completing the proof of (a).
(b) We have
dim null T 0 D dim.range T /0
D dimW dimrangeT
D dimW .dimV dimnullT/ DdimnullT CdimW dimV;
where the first equality comes from (a), the second equality comes from 3.106, and the third equality comes from the Fundamental Theorem of Linear Maps (3.22).
The next result can be useful because sometimes it is easier to verify that T 0 is injective than to show directly that T is surjective.
Proof The map T 2 L.V;W/ is surjective if and only if rangeT D W, which happens if and only if .range T /0 D f0g, which happens if and only if null T 0 D f0g [by 3.107(a)], which happens if and only if T 0 is injective.
SECTION 3.F Duality 107
3.108 T surjective is equivalent to T 0 injective
Suppose V and W are finite-dimensional and T 2 L.V; W /. Then T is surjective if and only if T 0 is injective.
3.109 The range of T 0
Suppose V and W are finite-dimensional and T 2 L.V; W /. Then
(a) dimrangeT0 D dimrangeT;
(b) rangeT0 D.nullT/0.
108
CHAPTER 3
Linear Maps
Proof
(a)
We have
(b)
where the first equality comes from the Fundamental Theorem of Linear Maps (3.22), the second equality comes from 3.95 and 3.107(a), and the third equality comes from 3.106.
First suppose ‘ 2 range T 0. Thus there exists 2 W 0 such that ‘DT0. /.Ifv2nullT,then
‘.v/DT0. /vD. ıT/.v/D .Tv/D .0/D0: Hence ‘ 2 .null T /0. This implies that range T 0 .null T /0.
We will complete the proof by showing that range T 0 and .null T /0 have the same dimension. To do this, note that
dim range T 0 D dim range T
D dimV dimnullT
D dim.null T /0;
where the first equality comes from (a), the second equality comes from the Fundamental Theorem of Linear Maps (3.22), and the third equality comes from 3.106.
The next result should be compared to 3.108.
dimrangeT0 D dimW0 dimnullT0
D dim W dim.range T /0
D dim range T;
3.110 T injective is equivalent to T 0 surjective
Suppose V and W are finite-dimensional and T 2 L.V; W /. Then T is injective if and only if T 0 is surjective.
Proof The map T 2 L.V;W/ is injective if and only if nullT D f0g, which happens if and only if .null T /0 D V 0, which happens if and only if range T 0 D V 0 [by 3.109(b)], which happens if and only if T 0 is surjective.
SECTION 3.F Duality 109 The Matrix of the Dual of a Linear Map
We now define the transpose of a matrix.
3.111 Definition transpose, At
The transpose of a matrix A, denoted At, is the matrix obtained from A by interchanging the rows and columns. More specifically, if A is an m-by-n matrix, then At is the n-by-m matrix whose entries are given by the equation
.At/k;j D Aj;k:
0 5 7 1 5 3 4 3.112 Example IfAD@ 3 8 A,thenAt D 7 8 2 .
4 2
Note that here A is a 3-by-2 matrix and At is a 2-by-3 matrix.
The transpose has nice algebraic properties: .A C C /t D At C C t and .A/t D At for all m-by-n matrices A; C and all 2 F (see Exercise 33).
The next result shows that the transpose of the product of two matrices is the product of the transposes in the opposite order.
3.113 The transpose of the product of matrices
If A is an m-by-n matrix and C is an n-by-p matrix, then .AC/t DCtAt:
Proof
Suppose1kpand1j m.Then
Thus .AC /t D C tAt, as desired.
.AC /t
D .AC /j;k
k;j
Xn
D D
Aj;r Cr;k
.C t/k;r .At/r;j
rD1 Xn
rD1
D .C tAt/k;j :
110 CHAPTER 3 Linear Maps
The setting for the next result is the assumption that we have a basis v1;:::;vn of V, along with its dual basis ‘1;:::;’n of V0. We also have a basis w1;:::;wm of W, along with its dual basis 1;:::; m of W0. Thus M.T / is computed with respect to the bases just mentioned of V and W, and M.T 0/ is computed with respect to the dual bases just mentioned of W 0 and V 0.
Proof LetA D M.T/andC D M.T0/. Suppose1 j mand 1 k n.
From the definition of M.T 0/ we have
3.114 The matrix of T 0 is the transpose of the matrix of T Suppose T 2 L.V; W /. Then M.T 0/ D M.T /t.
rD1 The left side of the equation above equals
of the equation above to vk gives
We also have
0 Xn T. j/D
Cr;j’r:
j ı T. Thus applying both sides
Cr;j’r.vk/
.jıT/.vk/D j.Tvk/
. j ıT/.vk/D
D Ck;j :
D j Xm
Xn rD1
Xm Ar;kwr
rD1
D Ar;k j.wr/
rD1 D Aj;k:
Comparingthelastlineofthelasttwosetsofequations,wehaveCk;j DAj;k. Thus C D At. In other words, M.T 0/ D M.T /t, as desired.
The Rank of a Matrix
We begin by defining two nonnegative integers that are associated with each matrix.
SECTION 3.F Duality 111
3.115 Definition row rank, column rank Suppose A is an m-by-n matrix with entries in F.
TherowrankofAisthedimensionofthespanoftherowsofAin F1;n .
The column rank of A is the dimension of the span of the columns of A in Fm;1.
3.116 Example SupposeAD 4 7 1 8 .FindtherowrankofA 3529
and the column rank of A.
Solution The row rank of A is the dimension of
span 4718;3529
in F1;4. Neither of the two vectors listed above in F1;4 is a scalar multiple of the other. Thus the span of this list of length 2 has dimension 2. In other words, the row rank of A is 2.
The column rank of A is the dimension of
!
span 43 ; 75 ; 12 ; 89
in F2;1. Neither of the first two vectors listed above in F2;1 is a scalar multiple of the other. Thus the span of this list of length 4 has dimension at least 2. The span of this list of vectors in F2;1 cannot have dimension larger than 2 because dim F2;1 D 2. Thus the span of this list has dimension 2. In other words, the column rank of A is 2.
Notice that no bases are in sight in the statement of the next result. Al- though M.T / in the next result depends on a choice of bases of V and W, the next result shows that the column rank of M.T / is the same for all such choices (because range T does not depend on a choice of basis).
112 CHAPTER 3 Linear Maps
3.117 Dimension of range T equals column rank of M.T /
Suppose V and W are finite-dimensional and T 2 L.V; W /. Then dim range T equals the column rank of M.T /.
Proof Supposev1;:::;vn isabasisofV andw1;:::;wm isabasisofW. The function that takes w 2 span.Tv1;:::;Tvn/ to M.w/ is easily seen to be an isomorphism from span.T v1; : : : ; T vn/ onto spanM.T v1/; : : : ; M.T vn/. Thus dimspan.Tv1;:::;Tvn/ D dimspanM.Tv1/;:::;M.Tvn/, where the last dimension equals the column rank of M.T /.
It is easy to see that rangeT D span.Tv1;:::;Tvn/. Thus we have dimrangeT D dimspan.Tv1;:::;Tvn/ D the column rank of M.T/, as desired.
In Example 3.116, the row rank and column rank turned out to equal each other. The next result shows that this always happens.
Proof DefineTWFn;1!Fm;1byTxDAx.ThusM.T/DA,where M.T / is computed with respect to the standard bases of Fn;1 and Fm;1. Now
column rank of A D column rank of M.T / D dim range T
D dim range T 0
D column rank of M.T 0/ D column rank of At
D row rank of A;
where the second equality above comes from 3.117, the third equality comes from 3.109(a), the fourth equality comes from 3.117 (where M.T 0/ is com- puted with respect to the dual bases of the standard bases), the fifth equality comes from 3.114, and the last equality follows easily from the definitions.
The last result allows us to dispense with the terms “row rank” and “column rank” and just use the simpler term “rank”.
3.118 Row rank equals column rank
Suppose A 2 Fm;n. Then the row rank of A equals the column rank of A.
3.119 Definition rank
The rank of a matrix A 2 Fm;n is the column rank of A.
EXERCISES 3.F
7 Suppose m is a positive integer. Show that the dual basis of the basis 1; x; : : : ; xm of Pm.R/ is ‘0; ‘1; : : : ; ‘m, where ‘j .p/ D p.j /.0/ . Here
SECTION 3.F Duality 113
1 Explain why every linear functional is either surjective or the zero map.
2 Give three distinct examples of linear functionals on RŒ0;1.
3 Suppose V is finite-dimensional and v 2 V with v ¤ 0. Prove that there exists’ 2V0 suchthat’.v/D1.
4 Suppose V is finite-dimensional and U is a subspace of V such that U ¤ V. Prove that there exists ‘ 2 V0 such that ‘.u/ D 0 for every u 2 U but ‘ ¤ 0.
5 SupposeV1;:::;Vmarevectorspaces.Provethat.V1Vm/0and V10 Vm0 are isomorphic vector spaces.
6 Suppose V is finite-dimensional and v1; : : : ; vm 2 V. Define a linear map W V 0 ! Fm by
.’/ D ’.v1/; : : : ; ‘.vm/:
(a) Prove that v1;:::;vm spans V if and only if is injective.
(b) Prove that v1; : : : ; vm is linearly independent if and only if is surjective.
jŠ
p.j / denotes the j th derivative of p, with the understanding that the 0th
derivative of p is p.
8 Suppose m is a positive integer.
(a) Showthat1;x5;:::;.x5/m isabasisofPm.R/.
(b) What is the dual basis of the basis in part (a)?
9 Suppose v1;:::;vn is a basis of V and ‘1;:::;’n is the corresponding dual basis of V 0. Suppose 2 V 0. Prove that
D .v1/’1 CC .vn/’n:
10 Prove the first two bullet points in 3.101.
114 CHAPTER 3 Linear Maps
11 Suppose A is an m-by-n matrix with A ¤ 0. Prove that the rank of A is 1 if and only if there exist .c1;:::;cm/ 2 Fm and .d1;:::;dn/ 2 Fn suchthatAj;k Dcjdk foreveryj D1;:::;mandeverykD1;:::;n.
12 Show that the dual map of the identity map on V is the identity map on V 0.
13 DefineTWR3 !R2 byT.x;y;z/D.4xC5yC6z;7xC8yC9z/. Suppose ‘1; ‘2 denotes the dual basis of the standard basis of R2 and
1; (a)
2; 3 denotes the dual basis of the standard basis of R3. Describe the linear functionals T 0.’1/ and T 0.’2/.
Write T 0.’1/ and T 0.’2/ as linear combinations of 1; 2; 3. 14 DefineTWP.R/!P.R/by.Tp/.x/Dx2p.x/Cp00.x/forx2R.
(a) Suppose ‘ 2 P.R/0 is defined by ‘.p/ D p0.4/. Describe the
p.x/ dx. Evaluate
15 Suppose W is finite-dimensional and T 2 L.V; W /. Prove that T 0 D 0 ifandonlyifT D0.
16 Suppose V and W are finite-dimensional. Prove that the map that takes T 2 L.V;W/ to T0 2 L.W0;V0/ is an isomorphism of L.V;W/ onto L.W0;V0/.
17 SupposeU V.ExplainwhyU0 Df’2V0 WU null’g.
18 Suppose V is finite-dimensional and U V. Show that U D f0g if and
onlyifU0 DV0.
19 Suppose V is finite-dimensional and U is a subspace of V. Show that
U DV ifandonlyifU0 Df0g.
20 SupposeU andW aresubsetsofV withU W. ProvethatW0 U0.
21 Suppose V is finite-dimensional and U and W are subspaces of V with W 0 U 0. Prove that U W.
22 SupposeU;WaresubspacesofV.Showthat.UCW/0DU0\W0.
(b)
linear functional T 0.’/ on P.R/. R 1 (b) Suppose ‘ 2 P.R/0 is defined by ‘.p/ D
03 0 T .’/ .x /.
SECTION 3.F Duality 115
23 Suppose V is finite-dimensional and U and W are subspaces of V. Prove
that .U \ W /0 D U 0 C W 0.
24 Prove 3.106 using the ideas sketched in the discussion before the state-
ment of 3.106.
25 Suppose V is finite-dimensional and U is a subspace of V. Show that
U Dfv2V W’.v/D0forevery’2U0g:
26 Suppose V is finite-dimensional and is a subspace of V 0. Show that
Dfv2V W’.v/D0forevery’ 2g0:
27 Suppose T 2 LP5.R/; P5.R/ and null T 0 D span.’/, where ‘ is the linear functional on P5.R/ defined by ‘.p/ D p.8/. Prove that rangeT Dfp2P5.R/Wp.8/D0g.
28 Suppose V and W are finite-dimensional, T 2 L.V; W /, and there exists ‘ 2 W0 such that nullT0 D span.’/. Prove that rangeT D null’.
29 Suppose V and W are finite-dimensional, T 2 L.V; W /, and there exists ‘ 2 V0 such that rangeT0 D span.’/. Prove that nullT D null’.
30 Suppose V is finite-dimensional and ‘1; : : : ; ‘m is a linearly independent list in V 0. Prove that
dim.null’1/\\.null’m/ D .dimV/m:
31 Suppose V is finite-dimensional and ‘1; : : : ; ‘n is a basis of V 0. Show
that there exists a basis of V whose dual basis is ‘1; : : : ; ‘n.
32 SupposeT 2L.V/,andu1;:::;unandv1;:::;vnarebasesofV.Prove
that the following are equivalent:
(a) T is invertible.
(b) The columns of M.T / are linearly independent in Fn;1.
(c) The columns of M.T / span Fn;1.
(d) The rows of M.T / are linearly independent in F1;n.
(e) The rows of M.T / span F1;n.
Here M.T/ means MT;.u1;:::;un/;.v1;:::;un/.
116 CHAPTER 3 Linear Maps
33 Suppose m and n are positive integers. Prove that the function that takes A to At is a linear map from Fm;n to Fn;m. Furthermore, prove that this linear map is invertible.
34 The double dual space of V, denoted V 00, is defined to be the dual space ofV0.Inotherwords,V00 D.V0/0.DefineƒWV !V00 by
.ƒv/.’/ D ‘.v/
forv2V and’2V0.
(a) ShowthatƒisalinearmapfromV toV00.
(b) ShowthatifT 2L.V/,thenT00ıƒDƒıT,whereT00 D.T0/0.
(c) Show that if V is finite-dimensional, then ƒ is an isomorphism from V onto V 00.
[Suppose V is finite-dimensional. Then V and V 0 are isomorphic, but finding an isomorphism from V onto V 0 generally requires choosing a basis of V. In contrast, the isomorphism ƒ from V onto V 00 does not require a choice of basis and thus is considered more natural.]
35 Show that P.R/0 and R1 are isomorphic.
36 Suppose U is a subspace of V. Let iW U ! V be the inclusion map
definedbyi.u/Du. Thusi0 2L.V0;U0/.
(a) Showthatnulli0DU0.
(b) Prove that if V is finite-dimensional, then range i 0 D U 0.
(c) Prove that if V is finite-dimensional, then ei0 is an isomorphism fromV0=U0 ontoU0.
[The isomorphism in part (c) is natural in that it does not depend on a choice of basis in either vector space.]
37 Suppose U is a subspace of V. Let W V ! V=U be the usual quotient map.Thus0 2L.V=U/0;V0.
(a) Show that 0 is injective.
(b) Show that range 0 D U 0.
(c) Conclude that 0 is an isomorphism from .V = U /0 onto U 0 .
[The isomorphism in part (c) is natural in that it does not depend on a choice of basis in either vector space. In fact, there is no assumption here that any of these vector spaces are finite-dimensional.]
CHAPTER
4
Polynomials
This short chapter contains material on polynomials that we will need to understand operators. Many of the results in this chapter will already be familiar to you from other courses; they are included here for completeness.
Because this chapter is not about linear algebra, your instructor may go through it rapidly. You may not be asked to scrutinize all the proofs. Make sure, however, that you at least read and understand the statements of all the results in this chapter—they will be used in later chapters.
The standing assumption we need for this chapter is as follows:
Statue of Persian mathematician and poet Omar Khayyám (1048–1131), whose algebra book written in 1070 contained the first serious study of cubic polynomials.
4.1 Notation F F denotes R or C.
LEARNING OBJECTIVES FOR THIS CHAPTER Division Algorithm for Polynomials
factorization of polynomials over C
factorization of polynomials over R
© Springer International Publishing 2015 117 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__4
118 CHAPTER 4 Polynomials
Complex Conjugate and Absolute Value
Before discussing polynomials with complex or real coefficients, we need to learn a bit more about the complex numbers.
4.2 Definition Re z, Im z
Suppose z D a C bi, where a and b are real numbers.
The real part of z, denoted Rez, is defined by Rez D a.
The imaginary part of z, denoted Im z, is defined by Im z D b.
Thus for every complex number z, we have z D Re z C .Im z/i:
4.3 Definition complex conjugate, zN, absolute value, jzj Suppose z 2 C.
The complex conjugate of z 2 C, denoted zN, is defined by zN D Re z .Im z/i:
The absolute value of a complex number z, denoted jzj, is defined
by
q
jzjD .Rez/2C.Imz/2:
4.4
Example Suppose z D 3 C 2i. Then RezD3andImzD2;
zN D 3 2 i ;
jzj D p32 C 22 D p13.
Note that jzj is a nonnegative number for every z 2 C.
The real and imaginary parts, com-
plex conjugate, and absolute value have the following properties:
You should verify that z D zN if and only if z is a real number.
Proof Except for the last item, the routine verifications of the assertions above are left to the reader. To verify the last item, we have
j w C z j 2 D . w C z / . wN C zN /
D w wN C z zN C w zN C z wN
D j w j 2 C j z j 2 C w zN C w zN D jwj2 C jzj2 C 2 Re.wzN/ jwj2 C jzj2 C 2jwzNj
D jwj2 C jzj2 C 2jwj jzj D .jwj C jzj/2:
Taking the square root of both sides of the inequality jw C zj2 .jwj C jzj/2 now gives the desired inequality.
CHAPTER 4 Polynomials 119
4.5 Properties of complex numbers
Suppose w; z 2 C. Then sum of z and zN
z C zN D 2 Re z; difference of z and zN
zzN D2.Imz/i; product of z and zN
z zN D j z j 2 ;
additivity and multiplicativity of complex conjugate
w C z D wN C zN and wz D wN zN ; conjugate of conjugate
zN D z ;
real and imaginary parts are bounded by jzj
jRezj jzj and jImzj jzj absolute value of the complex conjugate
jzNj D jzj;
multiplicativity of absolute value
jwzj D jwj jzj;
Triangle Inequality
jw C zj jwj C jzj.
w
wz
z
120 CHAPTER 4 Polynomials
Uniqueness of Coefficients for Polynomials
Recall that a function p W F ! F is called a polynomial with coefficients in F if there exist a0;:::;am 2 F such that
4.6 p.z/Da0 Ca1zCa2z2 CCamzm for all z 2 F.
4.7 If a polynomial is the zero function, then all coefficients are 0 Suppose a0;:::;am 2 F. If
a0 C a1z C C amzm D 0 foreveryz2F,thena0 DDam D0.
Proof We will prove the contrapositive. If not all the coefficients are 0, then by changing m we can assume am ¤ 0. Let
z D ja0jCja1jCCjam1j C1: jam j
Notethatz1,andthuszj zm1forjD0;1;:::;m1.Usingthe Triangle Inequality, we have
ja0 Ca1zCCam1zm1j.ja0jCja1jCCjam1j/zm1 < jamzmj:
Thus a0 C a1z C C am1zm1 ¤ amzm. Hence we conclude that a0 Ca1zCCam1zm1 Camzm ¤0.
The result above implies that the coefficients of a polynomial are uniquely determined (because if a polynomial had two different sets of coefficients, then subtracting the two representations of the polynomial would give a contradiction to the result above).
Recall that if a polynomial p can be written in the form 4.6 with am ¤ 0, then we say that p has degree m and we write degp D m.
The degree of the 0 polynomial is defined to be 1. When necessary, use the obvious arithmetic with 1. For example, 1 < m and 1Cm D 1 for every integer m.
The 0 polynomial is declared to have degree 1 so that excep- tions are not needed for various reasonable results. For example, deg.pq/ D degp C degq even if p D 0.
CHAPTER 4 Polynomials 121 The Division Algorithm for Polynomials
If p and s are nonnegative integers, with s ¤ 0, then there exist nonnegative integers q and r such that
p D sq C r
and r < s. Think of dividing p by s, getting quotient q with remainder r. Our next task is to prove an analogous result for polynomials.
The result below is often called the Division Algorithm for Polynomials, al- though as stated here it is not really an algorithm, just a useful result.
Recall that P.F/ denotes the vector space of all polynomials with co- efficients in F and that Pm.F/ is the subspace of P.F/ consisting of the polynomials with coefficients in F and degree at most m.
The next result can be proved without linear algebra, but the proof given here using linear algebra is appropriate for a linear algebra textbook.
Think of the Division Algorithm for Polynomials as giving the remain- der r when p is divided by s.
4.8 Division Algorithm for Polynomials
Suppose that p;s 2 P.F/, with s ¤ 0. Then there exist unique polynomials q; r 2 P.F/ such that
and deg r < deg s.
p D sq C r
Proof LetnDdegpandmDdegs.Ifn
polynomial with degree m 1 has at most m 1 distinct zeros. If p has no zerosinF,thenwearedone. Ifphasazero2F,thenby4.11thereisa polynomial q such that
p.z/ D .z /q.z/
for all z 2 F. Clearly deg q D m 1. The equation above shows that if p.z/D0,theneitherzDorq.z/D0. Inotherwords,thezerosofp consist of and the zeros of q. By our induction hypothesis, q has at most m 1 distinct zeros in F. Thus p has at most m distinct zeros in F.
Factorization of Polynomials over C
So far we have been handling polynomials with complex coefficients and polynomials with real coefficients simultaneously through our convention that F denotes R or C. Now we will see some differences between these two cases. First we treat polynomials with complex coefficients. Then we will use our results about polynomials with complex coefficients to prove corresponding results for polynomials with real coefficients.
The next result, although called the Fundamental Theorem of Algebra, uses analysis its proof. The short proof pre- sented here uses tools from complex analysis. If you have not had a course in complex analysis, this proof will almost certainly be meaningless to you. In that case, just accept the Fundamental The- orem of Algebra as something that we need to use but whose proof requires more advanced tools that you may learn in later courses.
CHAPTER 4 Polynomials 123
4.12 A polynomial has at most as many zeros as its degree
Suppose p 2 P.F/ is a polynomial with degree m 0. Then p has at most m distinct zeros in F.
The Fundamental Theorem of Al- gebra is an existence theorem. Its proof does not lead to a method for finding zeros. The quadratic for- mula gives the zeros explicitly for polynomials of degree 2. Similar but more complicated formulas ex- ist for polynomials of degree 3 and 4. No such formulas exist for poly- nomials of degree 5 and above.
124 CHAPTER 4 Polynomials
4.13 Fundamental Theorem of Algebra
Every nonconstant polynomial with complex coefficients has a zero.
Proof Let p be a nonconstant polynomial with complex coefficients. Sup- pose p has no zeros. Then 1=p is an analytic function on C. Furthermore, jp.z/j ! 1 as jzj ! 1, which implies that 1=p ! 0 as jzj ! 1. Thus 1=p is a bounded analytic function on C. By Liouville’s theorem, every such function is constant. But if 1=p is constant, then p is constant, contradicting our assumption that p is nonconstant.
Although the proof given above is probably the shortest proof of the Fundamental Theorem of Algebra, a web search can lead you to several other proofs that use different techniques. All proofs of the Fundamental Theorem of Algebra need to use some analysis, because the result is not true if C is replaced,forexample,withthesetofnumbersoftheformcCdi wherec;d are rational numbers.
Remarkably, mathematicians have proved that no formula exists for the ze- ros of polynomials of degree 5 or higher. But computers and calculators can use clever numerical methods to find good approximations to the zeros of any poly- nomial, even when exact zeros cannot be found.
For example, no one will ever be able to give an exact formula for a zero of the polynomial p defined by
p.x/ D x55×46x3C17x2C4x7:
However, a computer or symbolic cal- culator can find approximate zeros of this polynomial.
The Fundamental Theorem of Alge- bra leads to the following factorization result for polynomials with complex co- efficients. Note that in this factorization, the numbers 1; : : : ; m are precisely the zeros of p, for these are the only values of z for which the right side of the equation in the next result equals 0.
The cubic formula, which was discovered in the 16th century, is presented below for your amusement only. Do not memorize it.
Suppose
p.x/Dax3 Cbx2 CcxCd; where a ¤ 0. Set
uD 9abc2b3 27a2d 54a3
and then set
2 3ac b2 3 vDuC 9a2 :
Suppose v 0. Then
b q3 p q3 p
3aC uC vC u v is a zero of p.
Proof Let p 2 P.C/ and let m D degp. We will use induction on m. If m D 1, then clearly the desired factorization exists and is unique. So assume that m > 1 and that the desired factorization exists and is unique for all polynomials of degree m 1.
First we will show that the desired factorization of p exists. By the Fundamental Theorem of Algebra (4.13), p has a zero . By 4.11, there is a polynomial q such that
p.z/ D .z /q.z/
for all z 2 C. Because deg q D m 1, our induction hypothesis implies that q has the desired factorization, which when plugged into the equation above gives the desired factorization of p.
Now we turn to the question of uniqueness. Clearly c is uniquely deter- mined as the coefficient of zm in p. So we need only show that except for the order, there is only one way to choose 1; : : : ; m. If
.z 1/ .z m/ D .z 1/ .z m/
for all z 2 C, then because the left side of the equation above equals 0 when z D 1, one of the ’s on the right side equals 1. Relabeling, we can assume that 1 D 1. Now for z ¤ 1, we can divide both sides of the equation above by z 1, getting
.z 2/ .z m/ D .z 2/ .z m/
for all z 2 C except possibly z D 1. Actually the equation above holds for all z 2 C, because otherwise by subtracting the right side from the left side we would get a nonzero polynomial that has infinitely many zeros. The equation above and our induction hypothesis imply that except for the order, the ’s are the same as the ’s, completing the proof of uniqueness.
CHAPTER 4 Polynomials 125
4.14 Factorization of a polynomial over C
If p 2 P.C/ is a nonconstant polynomial, then p has a unique factoriza-
tion (except for the order of the factors) of the form
p.z/ D c.z 1/ .z m/; where c;1;:::;m 2 C.
126 CHAPTER 4 Polynomials Factorization of Polynomials over R
A polynomial with real coefficients may have no real zeros. For example, the polynomial 1 C x2 has no real zeros.
To obtain a factorization theorem over R, we will use our factorization theorem over C. We begin with the fol- lowing result.
The failure of the Fundamental Theorem of Algebra for R accounts for the differences between oper- ators on real and complex vector spaces, as we will see in later chapters.
4.15 Polynomials with real coefficients have zeros in pairs
Suppose p 2 P.C/ is a polynomial with real coefficients. If 2 C is a zero of p, then so is N.
Proof Let
p.z/Da0 Ca1zCCamzm;
where a0;:::;am are real numbers. Suppose 2 C is a zero of p. Then
a0 C a1 C C amm D 0:
Take the complex conjugate of both sides of this equation, obtaining
a0 Ca1N CCamNm D0;
where we have used basic properties of complex conjugation (see 4.5). The
equation above shows that N is a zero of p.
We want a factorization theorem for polynomials with real coefficients. First we need to characterize the polynomi- als of degree 2 with real coefficients that can be written as the product of two polynomials of degree 1 with real coefficients.
Think about the connection be- tween the quadratic formula and 4.16.
4.16 Factorization of a quadratic polynomial
Suppose b; c 2 R. Then there is a polynomial factorization of the form x2 C bx C c D .x 1/.x 2/
with1;2 2Rifandonlyifb2 4c.
Proof Notice that
xCbxCcDxC2 Cc4:
2 b2 b2
CHAPTER 4 Polynomials 127
First suppose b2 < 4c. Then clearly the right side of the equation above is positive for every x 2 R. Hence the polynomial x2 C bx C c has no real zeros and thus cannot be factored in the form.x1/.x2/with1;2 2R.
Conversely, now suppose b2 4c. Then there is a real number d such thatd2Db2 c.Fromthedisplayedequationabove,wehave
4
2 b22
x CbxCcD xC2 d
b b
D xC2Cd xC2d ; which gives the desired factorization.
The next result gives a factorization of a polynomial over R. The idea of the proof is to use the factorization 4.14 of p as a polynomial with complex coefficients. Complex but nonreal zeros of p come in pairs; see 4.15. Thus if the factorization of p as an element of P.C/ includes terms of the form .x / with a nonreal complex number, then .x N / is also a term in the factorization. Multiplying together these two terms, we get
x2 2.Re /x C jj2;
which is a quadratic term of the required form.
The idea sketched in the paragraph above almost provides a proof of the
existence of our desired factorization. However, we need to be careful about one point. Suppose is a nonreal complex number and .x / is a term in the factorization of p as an element of P.C/. We are guaranteed by 4.15 that .x N / also appears as a term in the factorization, but 4.15 does not state that these two factors appear the same number of times, as needed to make the idea above work. However, the proof works around this point.
In the next result, either m or M may equal 0. The numbers 1; : : : ; m are precisely the real zeros of p, for these are the only real values of x for which the right side of the equation in the next result equals 0.
The equation above is the basis of the technique called completing the square.
128 CHAPTER 4 Polynomials
4.17 Factorization of a polynomial over R
Suppose p 2 P.R/ is a nonconstant polynomial. Then p has a unique
factorization (except for the order of the factors) of the form
p.x/Dc.x1/.xm/.x2 Cb1xCc1/.x2 CbMxCcM/;
where c;1;:::;m;b1;:::;bM;c1;:::;cM 2 R, with bj2 < 4cj for each j .
Proof Think of p as an element of P.C/. If all the (complex) zeros of p are real, then we are done by 4.14. Thus suppose p has a zero 2 C with ... R. By 4.15, N is a zero of p. Thus we can write
p.x/ D .x /.x N /q.x/
D x2 2.Re /x C jj2q.x/
for some polynomial q 2 P.C/ with degree two less than the degree of p. If we can prove that q has real coefficients, then by using induction on the degree of p, we can conclude that .x / appears in the factorization of p exactly as many times as .x N /.
To prove that q has real coefficients, we solve the equation above for q, getting
q.x/ D p.x/
x2 2.Re /x C jj2
for all x 2 R. The equation above implies that q.x/ 2 R for all x 2 R. Writing
q.x/Da0 Ca1xCCan2xn2; where n D degp and a0;:::;an2 2 C, we thus have
0 D Im q.x/ D .Im a0/ C .Im a1/x C C .Im an2/xn2
for all x 2 R. This implies that Ima0;:::;Iman2 all equal 0 (by 4.7). Thus all the coefficients of q are real, as desired. Hence the desired factorization exists.
Now we turn to the question of uniqueness of our factorization. A factor ofpoftheformx2CbjxCcj withbj2 <4cj canbeuniquelywritten as .x j/.x j/ with j 2 C. A moment’s thought shows that two different factorizations of p as an element of P.R/ would lead to two different factorizations of p as an element of P.C/, contradicting 4.14.
EXERCISES 4
f0g [ fp 2 P.F/ W deg p is eveng
p.zj / D wj
[This result can be proved without using linear algebra. However, try to
forj D1;:::;mC1.
find the clearer, shorter proof that uses some linear algebra.]
6 Suppose p 2 P.C/ has degree m. Prove that p has m distinct zeros if and only if p and its derivative p0 have no zeros in common.
7 Prove that every polynomial of odd degree with real coefficients has a real zero.
8 DefineTWP.R/!RR by
8ˆ
Although null T and range T are invariant under T, they do not necessarily provide easy answers to the question about the existence of invariant subspaces other than f0g and V , because null T may equal f0g and range T may equal V (this happens when T is invertible).
5.4 Example Suppose that T 2 LP.R/ is defined by Tp D p0. Then P4.R/, which is a subspace of P.R/, is invariant under T because if p 2 P.R/ has degree at most 4, then p0 also has degree at most 4.
Eigenvalues and Eigenvectors
We will return later to a deeper study of invariant subspaces. Now we turn to an investigation of the simplest possible nontrivial invariant subspaces—invariant subspaces with dimension 1.
Take any v 2 V with v ¤ 0 and let U equal the set of all scalar multiples of v:
U DfvW2FgDspan.v/:
Then U is a 1-dimensional subspace of V (and every 1-dimensional subspace of V is of this form for an appropriate choice of v). If U is invariant under an operatorT 2L.V/,thenTv2U,andhencethereisascalar2Fsuchthat
T v D v:
Conversely, if T v D v for some 2 F, then span.v/ is a 1-dimensional subspace of V invariant under T.
134 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces The equation
T v D v;
which we have just seen is intimately connected with 1-dimensional invariant subspaces, is important enough that the vectors v and scalars satisfying it are given special names.
The comments above show that T has a 1-dimensional invariant subspace if and only if T has an eigenvalue.
In the definition above, we require that v ¤ 0 because every scalar 2 F satisfies T 0 D 0.
5.5 Definition eigenvalue
Suppose T 2 L.V /. A number 2 F is called an eigenvalue of T if thereexistsv2V suchthatv¤0andTvDv.
The word eigenvalue is half- German, half-English. The Ger- man adjective eigen means “own” in the sense of characterizing an in- trinsic property. Some mathemati- cians use the term characteristic value instead of eigenvalue.
5.6 Equivalent conditions to be an eigenvalue
Suppose V is finite-dimensional, T 2 L.V /, and 2 F . Then the following are equivalent:
(a) is an eigenvalue of T ;
(b) T I is not injective;
(c) T I is not surjective;
(d) T I is not invertible.
Recall that I 2 L.V / is the iden- tity operator defined by I v D v for all v 2 V.
Conditions (a) and (b) are equivalent because the equation T v D v is equivalent to the equation .T I /v D 0. Conditions (b), (c), and (d) are equivalent by 3.69.
Proof
5.7 Definition eigenvector
SupposeT 2L.V/and2FisaneigenvalueofT. Avectorv2V is called an eigenvector of T corresponding to if v ¤ 0 and T v D v.
SECTION 5.A Invariant Subspaces 135
BecauseTvDvifandonlyif.TI/vD0,avectorv2V withv¤0 is an eigenvector of T corresponding to if and only if v 2 null.T I /.
Example Suppose T 2 L.F2/ is defined by T.w;z/ D .z;w/:
Find the eigenvalues and eigenvectors of T if F D R. Find the eigenvalues and eigenvectors of T if F D C.
(a) If F D R, then T is a counterclockwise rotation by 90ı about the origin in R2. An operator has an eigenvalue if and only if there exists a nonzero vector in its domain that gets sent by the operator to a scalar multiple of itself. A 90ı counterclockwise rotation of a nonzero vector in R2 obviously never equals a scalar multiple of itself. Conclusion: if F D R, then T has no eigenvalues (and thus has no eigenvectors).
(b) To find eigenvalues of T, we must find the scalars such that T.w;z/ D .w;z/
has some solution other than w D z D 0. The equation above is equivalent to the simultaneous equations
5.9 zDw; wDz:
Substituting the value for w given by the second equation into the first
equation gives
z D 2z:
Now z cannot equal 0 [otherwise 5.9 implies that w D 0; we are looking for solutions to 5.9 where .w; z/ is not the 0 vector], so the equation above leads to the equation
1 D 2:
The solutions to this equation are D i and D i. You should be able to verify easily that i and i are eigenvalues of T. Indeed, the eigenvectors corresponding to the eigenvalue i are the vectors of the form .w;wi/, with w 2 C and w ¤ 0, and the eigenvectors corresponding to the eigenvalue i are the vectors of the form .w; wi /, with w 2 C and w ¤ 0.
5.8
(a)
(b) Solution
136 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
Now we show that eigenvectors corresponding to distinct eigenvalues are
linearly independent.
Proof Suppose v1; : : : ; vm is linearly dependent. Let k be the smallest posi- tive integer such that
5.11 vk 2span.v1;:::;vk1/I
the existence of k with this property follows from the Linear Dependence
Lemma (2.21). Thus there exist a1; : : : ; ak1 2 F such that 5.12 vk Da1v1 CCak1vk1: Apply T to both sides of this equation, getting
kvk Da11v1CCak1k1vk1:
Multiply both sides of 5.12 by k and then subtract the equation above, getting 0 D a1.k 1/v1 C C ak1.k k1/vk1:
Because we chose k to be the smallest positive integer satisfying 5.11, v1; : : : ; vk1 is linearly independent. Thus the equation above implies that all the a’s are 0 (recall that k is not equal to any of 1; : : : ; k1). However, this means that vk equals 0 (see 5.12), contradicting our hypothesis that vk is an eigenvector. Therefore our assumption that v1; : : : ; vm is linearly dependent was false.
The corollary below states that an operator cannot have more distinct eigenvalues than the dimension of the vector space on which it acts.
Proof Let T 2 L.V /. Suppose 1; : : : ; m are distinct eigenvalues of T. Let v1; : : : ; vm be corresponding eigenvectors. Then 5.10 implies that the list v1; : : : ; vm is linearly independent. Thus m dim V (see 2.23), as desired.
5.10 Linearly independent eigenvectors
Let T 2 L.V /. Suppose 1; : : : ; m are distinct eigenvalues of T and v1; : : : ; vm are corresponding eigenvectors. Then v1; : : : ; vm is linearly independent.
5.13 Number of eigenvalues
Suppose V is finite-dimensional. Then each operator on V has at most dim V distinct eigenvalues.
SECTION 5.A Invariant Subspaces 137 Restriction and Quotient Operators
If T 2 L.V / and U is a subspace of V invariant under T, then U determines two other operators TjU 2 L.U/ and T=U 2 L.V=U/ in a natural way, as defined below.
5.14 Definition TjU andT=U
Suppose T 2 L.V / and U is a subspace of V invariant under T.
The restriction operator T jU 2 L.U / is defined by T jU .u/ D T u
for u 2 U.
The quotient operator T=U 2 L.V=U/ is defined by
for v 2 V.
.T = U /.v C U / D T v C U
For both the operators defined above, it is worthwhile to pay attention to their domains and to spend a moment thinking about why they are well defined as operators on their domains. First consider the restriction operator T jU 2 L.U /, which is T with its domain restricted to U, thought of as mapping into U instead of into V. The condition that U is invariant under T is what allows us to think of T jU as an operator on U, meaning a linear map into the same space as the domain, rather than as simply a linear map from one vector space to another vector space.
To show that the definition above of the quotient operator makes sense, weneedtoverifythatifvCU DwCU,thenTvCU DTwCU. Hence supposevCU DwCU. Thusvw2U (see3.85). BecauseU isinvariant under T, we also have T .v w/ 2 U, which implies that T v T w 2 U, which impliesthatTvCU DTwCU,asdesired.
Suppose T is an operator on a finite-dimensional vector space V and U is a subspace of V invariant under T, with U ¤ f0g and U ¤ V. In some sense, wecanlearnaboutT bystudyingtheoperatorsTjU andT=U,eachofwhich is an operator on a vector space with smaller dimension than V. For example, proof 2 of 5.27 makes nice use of T=U.
However, sometimes TjU and T=U do not provide enough information aboutT.Inthenextexample,bothTjU andT=Uare0eventhoughTisnot the 0 operator.
138 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
5.15 Example Define an operator T 2 L.F2/ by T .x; y/ D .y; 0/. Let U Df.x;0/Wx2Fg.Showthat
(a) U is invariant under T and TjU is the 0 operator on U;
(b) there does not exist a subspace W of F2 that is invariant under T and
suchthatF2 DU ̊W;
(c) T=U is the 0 operator on F2=U.
Solution
(a) For .x;0/ 2 U, we have T.x;0/ D .0;0/ 2 U. Thus U is invariant underTandTjU isthe0operatoronU.
(b) Suppose W is a subspace of V such that F2 D U ̊ W. Because dimF2 D 2 and dimU D 1, we have dimW D 1. If W were invariant under T, then each nonzero vector in W would be an eigenvector of T. However, it is easy to see that 0 is the only eigenvalue of T and that all eigenvectors of T are in U. Thus W is not invariant under T.
(c) For .x; y/ 2 F2, we have
.T=U/.x;y/CU D T.x;y/CU D .y; 0/ C U
D 0 C U;
where the last equality holds because .y; 0/ 2 U. The equation above
shows that T=U is the 0 operator.
EXERCISES 5.A
1 SupposeT 2L.V/andU isasubspaceofV.
(a) Prove that if U null T, then U is invariant under T.
(b) Prove that if range T U, then U is invariant under T.
2 Suppose S;T 2 L.V/ are such that ST D TS. Prove that nullS is invariant under T.
SECTION 5.A Invariant Subspaces 139
3 Suppose S;T 2 L.V/ are such that ST D TS. Prove that rangeS is
invariant under T.
4 Suppose that T 2 L.V / and U1; : : : ; Um are subspaces of V invariant
under T. Prove that U1 C C Um is invariant under T.
5 Suppose T 2 L.V /. Prove that the intersection of every collection of
subspaces of V invariant under T is invariant under T.
6 Prove or give a counterexample: if V is finite-dimensional and U is a subspace of V that is invariant under every operator on V, then U D f0g or U D V.
7 Suppose T 2 L.R2/ is defined by T .x; y/ D .3y; x/. eigenvalues of T.
8 Define T 2 L.F2 / by
T.w;z/ D .z;w/: Find all eigenvalues and eigenvectors of T.
9 Define T 2 L.F3 / by
T .z1; z2; z3/ D .2z2; 0; 5z3/:
Find all eigenvalues and eigenvectors of T.
10 Define T 2 L.Fn / by
T.x1;x2;x3;:::;xn/ D .x1;2×2;3×3;:::;nxn/:
(a) Find all eigenvalues and eigenvectors of T.
Find the
(b) Find all invariant subspaces of T.
11 Define T W P.R/ ! P.R/ by Tp D p0. Find all eigenvalues and
eigenvectors of T.
12 Define T 2 LP4.R/ by
.Tp/.x/ D xp0.x/
for all x 2 R. Find all eigenvalues and eigenvectors of T.
13 Suppose V is finite-dimensional, T 2 L.V /, and 2 F. Prove that there
exists ̨2Fsuchthatj ̨j< 1 andT ̨I isinvertible. 1000
140 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
14 Suppose V D U ̊ W, where U and W are nonzero subspaces of V. DefineP 2L.V/byP.uCw/Duforu2U andw2W. Findall eigenvalues and eigenvectors of P .
15 Suppose T 2 L.V /. Suppose S 2 L.V / is invertible.
(a) Prove that T and S1TS have the same eigenvalues.
(b) What is the relationship between the eigenvectors of T and the eigenvectors of S1TS?
16 Suppose V is a complex vector space, T 2 L.V /, and the matrix of T with respect to some basis of V contains only real entries. Show that if is an eigenvalue of T, then so is N .
17 Give an example of an operator T 2 L.R4/ such that T has no (real) eigenvalues.
18 Show that the operator T 2 L.C1/ defined by T.z1;z2;:::/ D .0;z1;z2;:::/
has no eigenvalues.
19 Suppose n is a positive integer and T 2 L.Fn/ is defined by
T.x1;:::;xn/D.x1 CCxn;:::;x1 CCxn/I
in other words, T is the operator whose matrix (with respect to the standard basis) consists of all 1’s. Find all eigenvalues and eigenvectors of T.
20 Find all eigenvalues and eigenvectors of the backward shift operator T 2 L.F1/ defined by
T.z1;z2;z3;:::/ D .z2;z3;:::/:
21 Suppose T 2 L.V / is invertible.
(a) Suppose2Fwith¤0.ProvethatisaneigenvalueofT if and only if 1 is an eigenvalue of T 1.
(b) Prove that T and T 1 have the same eigenvectors.
SECTION 5.A Invariant Subspaces 100p2 22 Suppose T 2 L.V / and there exist nonzero vectors v and w in V such
that
Prove that 3 or 3 is an eigenvalue of T.
TvD3w and TwD3v:
23 Suppose V is finite-dimensional and S; T 2 L.V /. Prove that ST and
TS have the same eigenvalues.
24 Suppose A is an n-by-n matrix with entries in F. Define T 2 L.Fn/ by T x D Ax, where elements of Fn are thought of as n-by-1 column vectors.
(a) Suppose the sum of the entries in each row of A equals 1. Prove that 1 is an eigenvalue of T.
(b) Suppose the sum of the entries in each column of A equals 1. Prove that 1 is an eigenvalue of T.
25 Suppose T 2 L.V/ and u;v are eigenvectors of T such that u C v is also an eigenvector of T. Prove that u and v are eigenvectors of T corresponding to the same eigenvalue.
26 Suppose T 2 L.V / is such that every nonzero vector in V is an eigen- vector of T. Prove that T is a scalar multiple of the identity operator.
27 Suppose V is finite-dimensional and T 2 L.V / is such that every sub- space of V with dimension dim V 1 is invariant under T. Prove that T is a scalar multiple of the identity operator.
28 Suppose V is finite-dimensional with dim V 3 and T 2 L.V / is such that every 2-dimensional subspace of V is invariant under T. Prove that T is a scalar multiple of the identity operator.
29 Suppose T 2 L.V/ and dimrangeT D k. Prove that T has at most k C 1 distinct eigenvalues.
30 Suppose T 2 L.R3/ and 4, 5, and p7 are eigenvalues of T. Prove that thereexistsx2R3 suchthatTx9xD.4;5;p7/.
31 Suppose V is finite-dimensional and v1; : : : ; vm is a list of vectors in V. Prove that v1; : : : ; vm is linearly independent if and only if there exists T 2 L.V / such that v1; : : : ; vm are eigenvectors of T corresponding to distinct eigenvalues.
142 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
32 Suppose 1; : : : ; n is a list of distinct real numbers. Prove that the list e1x;:::;enx is linearly independent in the vector space of real-valued functions on R.
Hint:LetV Dspane1x;:::;enx,anddefineanoperatorT 2L.V/ by Tf D f 0. Find eigenvalues and eigenvectors of T .
33 SupposeT2L.V/.ProvethatT=.rangeT/D0.
34 Suppose T 2 L.V /. Prove that T =.null T / is injective if and only if
.nullT/\.rangeT/ D f0g.
35 Suppose V is finite-dimensional, T 2 L.V /, and U is invariant under T. Prove that each eigenvalue of T=U is an eigenvalue of T.
[The exercise below asks you to verify that the hypothesis that V is finite-dimensional is needed for the exercise above.]
36 Give an example of a vector space V, an operator T 2 L.V /, and a subspace U of V that is invariant under T such that T=U has an eigenvalue that is not an eigenvalue of T.
SECTION 5.B Eigenvectors and Upper-Triangular Matrices 143 5.B Eigenvectors and Upper-Triangular
Matrices
Polynomials Applied to Operators
The main reason that a richer theory exists for operators (which map a vector space into itself) than for more general linear maps is that operators can be raised to powers. We begin this section by defining that notion and the key concept of applying a polynomial to an operator.
If T 2 L.V /, then T T makes sense and is also in L.V /. We usually write T 2 instead of T T. More generally, we have the following definition.
5.16 Definition T m
Suppose T 2 L.V / and m is a positive integer.
Tm is defined by
TmDTT: „ƒ‚...
m times
T 0 is defined to be the identity operator I on V.
If T is invertible with inverse T 1, then T m is defined by Tm D.T1/m:
You should verify that if T is an operator, then
TmTn D TmCn and .Tm/n D Tmn;
where m and n are allowed to be arbitrary integers if T is invertible and nonnegative integers if T is not invertible.
5.17 Definition p.T /
Suppose T 2 L.V / and p 2 P.F/ is a polynomial given by
p.z/Da0 Ca1zCa2z2 CCamzm for z 2 F. Then p.T / is the operator defined by
p.T/Da0I Ca1T Ca2T2 CCamTm:
This is a new use of the symbol p because we are applying it to operators, not just elements of F.
144 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
5.18 Example Suppose D 2 LP.R/ is the differentiation operator defined by Dq D q0 and p is the polynomial defined by p.x/ D 73xC5x2. Thenp.D/D7I 3DC5D2;thus
p.D/q D 7q 3q0 C 5q00
If we fix an operator T 2 L.V /, then the function from P.F/ to L.V /
for every q 2 P.R/.
given by p 7! p.T / is linear, as you should verify.
5.19 Definition product of polynomials
If p; q 2 P.F/, then pq 2 P.F/ is the polynomial defined by
for z 2 F.
.pq/.z/ D p.z/q.z/
Any two polynomials of an operator commute, as shown below.
5.20 Multiplicative properties
Supposep;q2P.F/andT 2L.V/. Then
(a) .pq/.T / D p.T /q.T /;
(b) p.T /q.T / D q.T /p.T /.
Part (a) holds because when ex- panding a product of polynomials using the distributive property, it does not matter whether the sym- bol is z or T.
Proof
(a) Suppose p.z/ D PmjD0 aj zj and q.z/ D PnkD0 bkzk for z 2 F.
Then
Thus
Xm Xn jD0kD0
Xm Xn
.pq/.T/ D ajbkTjCk
jD0kD0
Xm Xn D ajTj bkTk
jD0 kD0 D p.T /q.T /:
.pq/.z/ D
ajbkzjCk:
(b) Part (a) implies p.T /q.T / D .pq/.T / D .qp/.T / D q.T /p.T /.
SECTION 5.B Eigenvectors and Upper-Triangular Matrices 145 Existence of Eigenvalues
Now we come to one of the central results about operators on complex vector spaces.
Proof Suppose V is a complex vector space with dimension n > 0 and T 2L.V/. Choosev2V withv¤0. Then
v;Tv;T2v;:::;Tnv
is not linearly independent, because V has dimension n and we have n C 1
vectors. Thus there exist complex numbers a0; : : : ; an, not all 0, such that 0 D a0vCa1TvCCanTnv:
Note that a1; : : : ; an cannot all be 0, because otherwise the equation above would become 0 D a0v, which would force a0 also to be 0.
Make the a’s the coefficients of a polynomial, which by the Fundamental Theorem of Algebra (4.14) has a factorization
a0 Ca1zCCanzn Dc.z1/.zm/;
where c is a nonzero complex number, each j is in C, and the equation holds for all z 2 C (here m is not necessarily equal to n, because an may equal 0). We then have
0 D a0vCa1TvCCanTnv D .a0I C a1T C C anT n/v Dc.T 1I/.T mI/v:
ThusT jI isnotinjectiveforatleastonej. Inotherwords,T hasan eigenvalue.
The proof above depends on the Fundamental Theorem of Algebra, which is typical of proofs of this result. See Exercises 16 and 17 for possible ways to rewrite the proof above using the idea of the proof in a slightly different form.
5.21 Operators on complex vector spaces have an eigenvalue
Every operator on a finite-dimensional, nonzero, complex vector space has an eigenvalue.
146 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces Upper-Triangular Matrices
In Chapter 3 we discussed the matrix of a linear map from one vector space to another vector space. That matrix depended on a choice of a basis of each of the two vector spaces. Now that we are studying operators, which map a vector space to itself, the emphasis is on using only one basis.
5.22 Definition matrix of an operator, M.T /
SupposeT 2L.V/andv1;:::;vn isabasisofV. Thematrixof T with
respect to this basis is the n-by-n matrix 01
A1;1 ::: A1;n M.T / D B@ : :
An;1 ::: An;n whose entries Aj;k are defined by
CA
Tvk DA1;kv1CCAn;kvn:
If the basis is not clear from the context, then the notation MT;.v1;:::;vn/ is used.
Note that the matrices of operators are square arrays, rather than the more general rectangular arrays that we considered earlier for linear maps.
If T is an operator on Fn and no basis is specified, assume that the basis in question is the standard one (where the jth basis vector is 1 in the jth slot and 0 in all the other slots). You can
then think of the j th column of M.T / as T applied to the j th basis vector. 5.23 Example DefineT 2L.F3/byT.x;y;z/D.2xCy;5yC3z;8z/.
The kth column of the matrix M.T / is formed from the coeffi- cients used to write T vk as a linear combination of v1; : : : ; vn.
Then
02101 M.T/D@0 5 3A:
008
A central goal of linear algebra is to show that given an operator T 2 L.V /, there exists a basis of V with respect to which T has a reasonably simple matrix. To make this vague formulation a bit more precise, we might try to choose a basis of V such that M.T / has many 0’s.
SECTION 5.B Eigenvectors and Upper-Triangular Matrices 147
If V is a finite-dimensional complex vector space, then we already know enough to show that there is a basis of V with respect to which the matrix of T has 0’s everywhere in the first column, except possibly the first entry. In other words, there is a basis of V with respect to which the matrix of T looks like
01
B0 C
B@ : : : CA I 0
here the denotes the entries in all the columns other than the first column. To prove this, let be an eigenvalue of T (one exists by 5.21) and let v be a corresponding eigenvector. Extend v to a basis of V. Then the matrix of T with respect to this basis has the form above.
Soon we will see that we can choose a basis of V with respect to which the matrix of T has even more 0’s.
For example, the diagonal of the matrix in 5.23 consists of the entries 2;5;8.
For example, the matrix in 5.23 is upper triangular.
Typically we represent an upper-triangular matrix in the form
0 1 1
5.24 Definition diagonal of a matrix
The diagonal of a square matrix consists of the entries along the line from the upper left corner to the bottom right corner.
5.25 Definition upper-triangular matrix
A matrix is called upper triangular if all the entries below the diagonal equal 0.
B@ : : :
0 n
the 0 in the matrix above indicates that all entries below the diagonal in this n-by-n matrix equal 0. Upper- triangular matrices can be considered reasonably simple—for n large, almost half its entries in an n-by-n upper- triangular matrix are 0.
CA I
We often use to denote matrix en- tries that we do not know about or that are irrelevant to the questions being discussed.
148 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
The following proposition demonstrates a useful connection between
upper-triangular matrices and invariant subspaces.
5.26 Conditions for upper-triangular matrix
SupposeT 2L.V/andv1;:::;vn isabasisofV. Thenthefollowingare equivalent:
(a) the matrix of T with respect to v1; : : : ; vn is upper triangular;
(b) Tvj 2span.v1;:::;vj/foreachj D1;:::;n;
(c) span.v1;:::;vj/isinvariantunderT foreachj D1;:::;n.
Proof The equivalence of (a) and (b) follows easily from the definitions and a moment’s thought. Obviously (c) implies (b). Hence to complete the proof, we need only prove that (b) implies (c).
Thus suppose (b) holds. Fix j 2 f1; : : : ; ng. From (b), we know that Tv1 2 span.v1/ span.v1;:::;vj/I
Tv2 2 span.v1;v2/ span.v1;:::;vj/I :
Tvj 2 span.v1;:::;vj/:
Thus if v is a linear combination of v1;:::;vj , then
Tv 2 span.v1;:::;vj/:
In other words, span.v1; : : : ; vj / is invariant under T, completing the proof.
Now we can prove that for each operator on a finite-dimensional com- plex vector space, there is a basis of the vector space with respect to which the matrix of the operator has only 0’s be- low the diagonal. In Chapter 8 we will improve even this result.
Sometimes more insight comes from seeing more than one proof of a theo- rem. Thus two proofs are presented of the next result. Use whichever appeals more to you.
The next result does not hold on real vector spaces, because the first vector in a basis with respect to which an operator has an upper- triangular matrix is an eigenvector of the operator. Thus if an opera- tor on a real vector space has no eigenvalues [see 5.8(a) for an ex- ample], then there is no basis with respect to which the operator has an upper-triangular matrix.
SECTION 5.B Eigenvectors and Upper-Triangular Matrices 149
5.27 Over C, every operator has an upper-triangular matrix
Suppose V is a finite-dimensional complex vector space and T 2 L.V /. Then T has an upper-triangular matrix with respect to some basis of V.
Proof 1 We will use induction on the dimension of V. Clearly the desired result holds if dim V D 1.
Suppose now that dim V > 1 and the desired result holds for all complex vector spaces whose dimension is less than the dimension of V. Let be any eigenvalue of T (5.21 guarantees that T has an eigenvalue). Let
U D range.T I /:
Because T I is not surjective (see 3.69), dim U < dim V. Furthermore,
U is invariant under T. To prove this, suppose u 2 U. Then T u D .T I /u C u:
Obviously .T I/u 2 U (because U equals the range of T I) and u 2 U. Thus the equation above shows that T u 2 U. Hence U is invariant under T, as claimed.
Thus T jU is an operator on U. By our induction hypothesis, there is a basis u1; : : : ; um of U with respect to which T jU has an upper-triangular matrix. Thus for each j we have (using 5.26)
5.28 Tuj D.TjU/.uj/2span.u1;:::;uj/:
Extend u1;:::;um to a basis u1;:::;um;v1;:::;vn of V. For each k, we
have
Tvk D.T I/vk Cvk:
The definition of U shows that .T I /vk 2 U D span.u1; : : : ; um/. Thus
the equation above shows that
5.29 Tvk 2 span.u1;:::;um;v1;:::;vk/:
From 5.28 and 5.29, we conclude (using 5.26) that T has an upper- triangular matrix with respect to the basis u1;:::;um;v1;:::;vn of V, as desired.
150 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
Proof 2 We will use induction on the dimension of V. Clearly the desired result holds if dim V D 1.
Suppose now that dim V D n > 1 and the desired result holds for all complex vector spaces whose dimension is n 1. Let v1 be any eigenvector of T (5.21 guarantees that T has an eigenvector). Let U D span.v1/. Then U is an invariant subspace of T and dim U D 1.
Because dim V = U D n 1 (see 3.89), we can apply our induction hy- pothesis to T=U 2 L.V=U/. Thus there is a basis v2 C U;:::;vn C U of V=U such that T=U has an upper-triangular matrix with respect to this basis. Hence by 5.26,
.T=U/.vj CU/2span.v2CU;:::;vj CU/
for each j D 2; : : : ; n. Unraveling the meaning of the inclusion above, we
see that
Tvj 2 span.v1;:::;vj/
for each j D 1; : : : ; n. Thus by 5.26, T has an upper-triangular matrix with respect to the basis v1; : : : ; vn of V, as desired (it is easy to verify that v1;:::;vn is a basis of V ; see Exercise 13 in Section 3.E for a more general result).
How does one determine from looking at the matrix of an operator whether the operator is invertible? If we are fortunate enough to have a basis with respect to which the matrix of the operator is upper triangular, then this problem becomes easy, as the following proposition shows.
5.30 Determination of invertibility from upper-triangular matrix
Suppose T 2 L.V / has an upper-triangular matrix with respect to some basis of V. Then T is invertible if and only if all the entries on the diagonal of that upper-triangular matrix are nonzero.
Proof Suppose v1;:::;vn is a basis of V with respect to which T has an upper-triangular matrix
5.31 M.T / D B@ : : : CA : 0 n
0 1 1 B2 C
We need to prove that T is invertible if and only if all the j ’s are nonzero.
SECTION 5.B Eigenvectors and Upper-Triangular Matrices 151
First suppose the diagonal entries 1; : : : ; n are all nonzero. The upper- triangular matrix in 5.31 implies that T v1 D 1v1. Because 1 ¤ 0, we have T .v1=1/ D v1; thus v1 2 range T.
Now
T .v2=2/ D av1 C v2
for some a 2 F. The left side of the equation above and av1 are both in range T ; thus v2 2 range T.
Similarly, we see that
T .v3=3/ D bv1 C cv2 C v3
for some b; c 2 F. The left side of the equation above and bv1; cv2 are all in range T ; thus v3 2 range T.
Continuing in this fashion, we conclude that v1; : : : ; vn 2 range T. Be- cause v1; : : : ; vn is a basis of V, this implies that range T D V. In other words, T is surjective. Hence T is invertible (by 3.69), as desired.
To prove the other direction, now suppose that T is invertible. This implies that 1 ¤ 0, because otherwise we would have T v1 D 0.
Let1
fp.T / W p 2 P.F/g ¤ L.V /:
20 Suppose V is a finite-dimensional complex vector space and T 2 L.V /. Prove that T has an invariant subspace of dimension k for each k D 1;:::;dimV.
SECTION 5.C Eigenspaces and Diagonal Matrices 155 5.C Eigenspaces and Diagonal Matrices
5.34 Definition diagonal matrix
A diagonal matrix is a square matrix that is 0 everywhere except possibly along the diagonal.
5.35 Example
is a diagonal matrix.
08001 @050A 005
Obviously every diagonal matrix is upper triangular. In general, a diagonal matrix has many more 0’s than an upper-triangular matrix.
If an operator has a diagonal matrix with respect to some basis, then the entries along the diagonal are precisely the eigenvalues of the operator; this follows from 5.32 (or find an easier proof for diagonal matrices).
5.36 Definition eigenspace, E.; T /
Suppose T 2 L.V / and 2 F. The eigenspace of T corresponding to ,
denoted E.; T /, is defined by E.;T/Dnull.T I/:
In other words, E.; T / is the set of all eigenvectors of T corresponding to , along with the 0 vector.
For T 2 L.V/ and 2 F, the eigenspace E.;T/ is a subspace of V (because the null space of each linear map on V is a subspace of V ). The definitions imply that is an eigenvalue of T if and only if E.; T / ¤ f0g.
5.37 Example Suppose the matrix of an operator T 2 L.V / with respect to a basis v1; v2; v3 of V is the matrix in Example 5.35 above. Then
E.8; T / D span.v1/; E.5; T / D span.v2; v3/:
If is an eigenvalue of an operator T 2 L.V /, then T restricted to E.; T / is just the operator of multiplication by .
156 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
5.38 Sum of eigenspaces is a direct sum
Suppose V is finite-dimensional and T 2 L.V /. Suppose also that 1; : : : ; m are distinct eigenvalues of T. Then
E.1;T/CCE.m;T/ is a direct sum. Furthermore,
dim E.1; T / C C dim E.m; T / dim V:
Proof ToshowthatE.1;T/CCE.m;T/isadirectsum,suppose u1 C C um D 0;
whereeachuj isinE.;T/.Becauseeigenvectorscorrespondingtodistinct eigenvalues are linearly independent (see 5.10), this implies that each uj equals 0. This implies (using 1.44) that E.1; T / C C E.m; T / is a direct sum, as desired.
Now
dim E.1; T / C C dim E.m; T / D dimE.1; T / ̊ ̊ E.m; T /
dim V;
where the equality above follows from Exercise 16 in Section 2.C.
5.40 Example Define T 2 L.R2/ by
T .x; y/ D .41x C 7y; 20x C 74y/:
The matrix of T with respect to the standard basis of R2 is
matrix of T with respect to the basis .1; 4/; .7; 5/ is
5.39 Definition diagonalizable
An operator T 2 L.V / is called diagonalizable if the operator has a diagonal matrix with respect to some basis of V.
as you should verify.
41 7 20 74
;
which is not a diagonal matrix. However, T is diagonalizable, because the
69 0 0 46
;
Proof
An operator T 2 L.V / has a diagonal matrix 01
1 0
B@ : : : CA
0 n
SECTION 5.C Eigenspaces and Diagonal Matrices 157
5.41 Conditions equivalent to diagonalizability
Suppose V is finite-dimensional and T 2 L.V /. Let 1; : : : ; m denote the distinct eigenvalues of T. Then the following are equivalent:
(a) T is diagonalizable;
(b) V has a basis consisting of eigenvectors of T ;
(c) there exist 1-dimensional subspaces U1; : : : ; Un of V, each invariant under T, such that
V DU1 ̊ ̊UnI
(d) V DE.1;T/ ̊ ̊E.m;T/;
(e) dimV DdimE.1;T/CCdimE.m;T/.
withrespecttoabasisv1;:::;vn ofV ifandonlyifTvj Djvj foreachj. Thus (a) and (b) are equivalent.
Suppose (b) holds; thus V has a basis v1; : : : ; vn consisting of eigenvectors of T. For each j , let Uj D span.vj /. Obviously each Uj is a 1-dimensional subspace of V that is invariant under T. Because v1; : : : ; vn is a basis of V, each vector in V can be written uniquely as a linear combination of v1; : : : ; vn. In other words, each vector in V can be written uniquely as a sum u1C Cun, whereeachuj isinUj.ThusV DU1 ̊ ̊Un.Hence(b)implies(c).
Suppose now that (c) holds; thus there are 1-dimensional subspaces U1;:::;Un of V, each invariant under T, such that V D U1 ̊ ̊ Un. For each j , let vj be a nonzero vector in Uj . Then each vj is an eigenvector of T. Because each vector in V can be written uniquely as a sum u1 C Cun, where each uj is in Uj (so each uj is a scalar multiple of vj ), we see that v1;:::;vn is a basis of V. Thus (c) implies (b).
At this stage of the proof we know that (a), (b), and (c) are all equivalent. We will finish the proof by showing that (b) implies (d), that (d) implies (e), and that (e) implies (b).
158 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
Suppose (b) holds; thus V has a basis consisting of eigenvectors of T. Hence every vector in V is a linear combination of eigenvectors of T, which implies that
V DE.1;T/CCE.m;T/:
Now 5.38 shows that (d) holds.
That (d) implies (e) follows immediately from Exercise 16 in Section 2.C. Finally, suppose (e) holds; thus
5.42 dimV DdimE.1;T/CCdimE.m;T/:
Choose a basis of each E.j ; T /; put all these bases together to form a list v1;:::;vn of eigenvectors of T, where n D dimV (by 5.42). To show that this list is linearly independent, suppose
a1v1 C C anvn D 0;
where a1;:::;an 2 F. For each j D 1;:::;m, let uj denote the sum of all
thetermsakvk suchthatvk 2E.j;T/.Thuseachuj isinE.j;T/,and u1 C C um D 0:
Because eigenvectors corresponding to distinct eigenvalues are linearly inde- pendent (see 5.10), this implies that each uj equals 0. Because each uj is a sum of terms ak vk , where the vk ’s were chosen to be a basis of E.j ; T /, this implies that all the ak’s equal 0. Thus v1;:::;vn is linearly independent and hence is a basis of V (by 2.39). Thus (e) implies (b), completing the proof.
Unfortunately not every operator is diagonalizable. This sad state of affairs can arise even on complex vector spaces, as shown by the next example.
5.43 Example Show that the operator T 2 L.C2/ defined by T.w;z/ D .z;0/
is not diagonalizable.
Solution As you should verify, 0 is the only eigenvalue of T and furthermore E.0;T/Df.w;0/2C2 Ww2Cg.
Thus conditions (b), (c), (d), and (e) of 5.41 are easily seen to fail (of course, because these conditions are equivalent, it is only necessary to check that one of them fails). Thus condition (a) of 5.41 also fails, and hence T is not diagonalizable.
SECTION 5.C Eigenspaces and Diagonal Matrices 159 The next result shows that if an operator has as many distinct eigenvalues
as the dimension of its domain, then the operator is diagonalizable.
Proof Suppose T 2 L.V / has dim V distinct eigenvalues 1; : : : ; dim V . For each j , let vj 2 V be an eigenvector corresponding to the eigenvalue j . Because eigenvectors corresponding to distinct eigenvalues are linearly inde- pendent(see5.10),v1;:::;vdimV islinearlyindependent.Alinearlyindepen- dent list of dimV vectors in V is a basis of V (see 2.39); thus v1;:::;vdimV is a basis of V. With respect to this basis consisting of eigenvectors, T has a diagonal matrix.
5.45 Example DefineT 2L.F3/byT.x;y;z/D.2xCy;5yC3z;8z/. Find a basis of F3 with respect to which T has a diagonal matrix.
Solution With respect to the standard basis, the matrix of T is
02101 @053A:
008
The matrix above is upper triangular. Thus by 5.32, the eigenvalues of T are 2, 5, and 8. Because T is an operator on a vector space with dimension 3 and T has three distinct eigenvalues, 5.44 assures us that there exists a basis of F3 with respect to which T has a diagonal matrix.
To find this basis, we only have to find an eigenvector for each eigenvalue. In other words, we have to find a nonzero solution to the equation
T.x;y;z/ D .x;y;z/
for D 2, then for D 5, and then for D 8. These simple equations are easy to solve: for D 2 we have the eigenvector .1; 0; 0/; for D 5 we have the eigenvector .1; 3; 0/; for D 8 we have the eigenvector .1; 6; 6/.
Thus .1; 0; 0/; .1; 3; 0/; .1; 6; 6/ is a basis of F3, and with respect to this
5.44 Enough eigenvalues implies diagonalizability
If T 2 L.V / has dim V distinct eigenvalues, then T is diagonalizable.
basis the matrix of T is
02001 @050A:
008
160 CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces
The converse of 5.44 is not true. For example, the operator T defined on
the three-dimensional space F3 by
T .z1; z2; z3/ D .4z1; 4z2; 5z3/
has only two eigenvalues (4 and 5), but this operator has a diagonal matrix with respect to the standard basis.
In later chapters we will find additional conditions that imply that certain operators are diagonalizable.
EXERCISES 5.C
1 Suppose T 2 L.V / is diagonalizable. Prove that V D null T ̊ range T.
2 Prove the converse of the statement in the exercise above or give a
counterexample to the converse.
3 Suppose V is finite-dimensional and T 2 L.V /. Prove that the following are equivalent:
(a) V D null T ̊ range T.
(b) V D null T C range T.
(c) null T \ range T D f0g.
4 Give an example to show that the exercise above is false without the hypothesis that V is finite-dimensional.
5 Suppose V is a finite-dimensional complex vector space and T 2 L.V /. Prove that T is diagonalizable if and only if
V D null.T I / ̊ range.T I /
for every 2 C.
6 Suppose V is finite-dimensional, T 2 L.V / has dim V distinct eigenval- ues, and S 2 L.V / has the same eigenvectors as T (not necessarily with the same eigenvalues). Prove that ST D TS.
7 Suppose T 2 L.V / has a diagonal matrix A with respect to some basis of V and that 2 F. Prove that appears on the diagonal of A precisely dim E.; T / times.
8 SupposeT 2L.F5/anddimE.8;T/D4.ProvethatT2IorT6I is invertible.
SECTION 5.C Eigenspaces and Diagonal Matrices 161 9 SupposeT 2L.V/isinvertible. ProvethatE.;T/DE.1;T1/for
10 Suppose that V is finite-dimensional and T 2 L.V /. Let 1; : : : ; m
every 2 F with ¤ 0.
denote the distinct nonzero eigenvalues of T. Prove that
dim E.1; T / C C dim E.m; T / dim range T:
11 Verify the assertion in Example 5.40.
12 Suppose R; T 2 L.F3/ each have 2, 6, 7 as eigenvalues. Prove that there
exists an invertible operator S 2 L.F3/ such that R D S1TS.
13 Find R; T 2 L.F4/ such that R and T each have 2, 6, 7 as eigenvalues, R and T have no other eigenvalues, and there does not exist an invertible operator S 2 L.F4/ such that R D S1TS.
14 Find T 2 L.C3/ such that 6 and 7 are eigenvalues of T and such that T does not have a diagonal matrix with respect to any basis of C3.
15 Suppose T 2 L.C3/ is such that 6 and 7 are eigenvalues of T. Fur-
thermore, suppose T does not have a diagonal matrix with respect
to any basis of C3. Prove that there exists .x;y;z/ 2 F3 such that p
T.x;y;z/D.17C8x; 5C8y;2C8z/.
16 The Fibonacci sequence F1 ; F2 ; : : : is defined by
F1 D1; F2 D1; and Fn DFn2 CFn1 forn3: DefineT 2L.R2/byT.x;y/D.y;xCy/.
(a) Show that T n .0; 1/ D .Fn ; FnC1 / for each positive integer n.
(b) Find the eigenvalues of T.
(c) Find a basis of R2 consisting of eigenvectors of T.
(d) Use the solution to part (c) to compute T n.0; 1/. Conclude that
1 1Cp5n 1p5n FnDp5 2 2
for each positive integer n.
(e) Use part (d) to conclude that for each positive integer n, the
Fibonacci number Fn is the integer that is closest to
1 1Cp5n p5 2 :
CHAPTER
6
Inner Product Spaces
In making the definition of a vector space, we generalized the linear structure (addition and scalar multiplication) of R2 and R3. We ignored other important features, such as the notions of length and angle. These ideas are embedded in the concept we now investigate, inner products.
Our standing assumptions are as follows:
Woman teaching geometry, from a fourteenth-century edition of Euclid’s geometry book.
6.1 Notation F, V
F denotes R or C.
V denotes a vector space over F.
LEARNING OBJECTIVES FOR THIS CHAPTER Cauchy–Schwarz Inequality
Gram–Schmidt Procedure
linear functionals on inner product spaces calculating minimum distance to a subspace
© Springer International Publishing 2015 163 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__6
164 CHAPTER 6 Inner Product Spaces
6.A Inner Products and Norms
Inner Products
The length of this vector x is p22
To motivate the concept of inner prod-
uct, think of vectors in R2 and R3 as
arrows with initial point at the origin.
The length of a vector x in R2 or R3
is called the norm of x, denoted kxk.
Thus for x D .x ;x / 2 R2, we have p12
x1, x2 x
x1 Cx2 .
kxkD x12Cx22. Similarly,ifxD.x ;x ;x /2R3,
p
123 then kxk D x12 C x22 C x32.
Even though we cannot draw pictures in higher dimensions, the gener- alization to Rn is obvious: we define the norm of x D .x1;:::;xn/ 2 Rn
by
introduce the dot product.
p
kxkD x12CCxn2:
The norm is not linear on Rn. To inject linearity into the discussion, we
6.2 Definition dot product
For x; y 2 Rn, the dot product of x and y, denoted x y, is defined by
x y D x1y1 C C xnyn; where x D .x1;:::;xn/ and y D .y1;:::;yn/.
If we think of vectors as points in- stead of arrows, then kxk should be interpreted as the distance from the origin to the point x.
xx0forallx2Rn;
xxD0ifandonlyifxD0;
Note that the dot product of two vec- tors in Rn is a number, not a vector. Ob- viously x x D kxk2 for all x 2 Rn. The dot product on Rn has the follow- ing properties:
fory2Rn fixed,themapfromRn toRthatsendsx2Rn toxyis linear;
xyDyxforallx;y2Rn.
SECTION 6.A Inner Products and Norms 165
An inner product is a generalization of the dot product. At this point you may be tempted to guess that an inner product is defined by abstracting the properties of the dot product discussed in the last paragraph. For real vector spaces, that guess is correct. However, so that we can make a definition that will be useful for both real and complex vector spaces, we need to examine the complex case before making the definition.
Recall that if D a C bi, where a; b 2 R, then
the absolute value of , denoted jj, is defined by jj D pa2 C b2; thecomplexconjugateof,denotedN,isdefinedbyN Dabi;
j j 2 D N .
See Chapter 4 for the definitions and the basic properties of the absolute value and complex conjugate.
For z D .z1;:::;zn/ 2 Cn, we define the norm of z by q
kzkD jz1j2CCjznj2:
The absolute values are needed because we want kzk to be a nonnegative
number. Note that
kzk2 D z1z1 C C znzn:
We want to think of kzk2 as the inner product of z with itself, as we did in Rn. The equation above thus suggests that the inner product of w D .w1;:::;wn/ 2 Cn with z should equal
w1z1 CCwnzn:
If the roles of the w and z were interchanged, the expression above would be replaced with its complex conjugate. In other words, we should expect that the inner product of w with z equals the complex conjugate of the inner product of z with w. With that motivation, we are now ready to define an inner product on V, which may be a real or a complex vector space.
Two comments about the notation used in the next definition:
If is a complex number, then the notation 0 means that is real and nonnegative.
We use the common notation hu; vi, with angle brackets denoting an inner product. Some people use parentheses instead, but then .u; v/ becomes ambiguous because it could denote either an ordered pair or an inner product.
166 CHAPTER 6
Inner Product Spaces
6.3 Definition inner product
An inner product on V is a function that takes each ordered pair .u; v/ of
elements of V to a number hu; vi 2 F and has the following properties: positivity
hv; vi 0 for all v 2 V ; definiteness
hv; vi D 0 if and only if v D 0; additivity in first slot
hu C v; wi D hu; wi C hv; wi for all u; v; w 2 V ; homogeneity in first slot
hu; vi D hu; vi for all 2 F and all u; v 2 V ;
conjugate symmetry
hu;vi D hv;ui for all u;v 2 V.
Every real number equals its com- plex conjugate. Thus if we are dealing with a real vector space, then in the last condition above we can dispense with the complex conjugate and simply state that hu;vi D hv;ui for all v;w 2 V.
6.4 Example
(a) The Euclidean inner product on Fnis defined by
h.w1;:::;wn/;.z1;:::;zn/iDw1z1 CCwnzn:
(b) If c1; : : : ; cn are positive numbers, then an inner product can be defined on Fn by
h.w1;:::;wn/;.z1;:::;zn/iDc1w1z1 CCcnwnzn:
(c) An inner product can be defined on the vector space of continuous
real-valued functions on the interval Œ1; 1 by Z1
hf; gi D
(d) An inner product can be defined on P.R/ by
Although most mathematicians de- fine an inner product as above, many physicists use a definition that requires homogeneity in the second slot instead of the first slot.
inner products
hp; qi D
p.x/q.x/ex dx:
1
Z1 0
f .x/g.x/ dx:
SECTION 6.A Inner Products and Norms 167
6.5 Definition inner product space
An inner product space is a vector space V along with an inner product on V.
The most important example of an inner product space is Fn with the Euclidean inner product given by part (a) of the last example. When Fn is referred to as an inner product space, you should assume that the inner product is the Euclidean inner product unless explicitly told otherwise.
So that we do not have to keep repeating the hypothesis that V is an inner product space, for the rest of this chapter we make the following assumption:
Note the slight abuse of language here. An inner product space is a vector space along with an inner product on that vector space. When we say that a vector space V is an inner product space, we are also thinking that an inner product on V is lurking nearby or is obvious from the context (or is the Euclidean inner product if the vector space is Fn).
6.6 Notation V
For the rest of this chapter, V denotes an inner product space over F.
6.7 Basic properties of an inner product
(a) For each fixed u 2 V, the function that takes v to hv; ui is a linear map from V to F.
(b) h0;uiD0foreveryu2V.
(c) hu;0iD0foreveryu2V.
(d) hu;vCwiDhu;viChu;wiforallu;v;w2V.
(e) hu;viDNhu;viforall2Fandu;v2V.
Proof
(a) Part (a) follows from the conditions of additivity in the first slot and homogeneity in the first slot in the definition of an inner product.
(b) Part (b) follows from part (a) and the result that every linear map takes 0 to 0.
168 CHAPTER 6 Inner Product Spaces
(c) Part (c) follows from part (a) and the conjugate symmetry property in
the definition of an inner product.
(d) Suppose u; v; w 2 V. Then
hu; v C wi D hv C w; ui
D hv; ui C hw; ui
D hv; ui C hw; ui D hu; vi C hu; wi:
(e) Suppose2Fandu;v2V.Then
hu; vi D hv; ui D hv; ui D N h v ; u i
D N hu; vi;
as desired.
Norms
Our motivation for defining inner products came initially from the norms of vectors on R2 and R3. Now we see that each inner product determines a norm.
6.9 Example norms
(a) If .z1; : : : ; zn/ 2 Fn (with the Euclidean inner product), then
q
kfkD
6.8 Definition norm, kvk
For v 2 V, the norm of v, denoted kvk, is defined by p
kvk D hv; vi:
jz1j2 CCjznj2: inner product given as in part (c) of 6.4], we have
k.z1;:::;zn/kD
(b) In the vector space of continuous real-valued functions on Œ1; 1 [with
sZ 1 2 f.x/ dx:
1
SECTION 6.A Inner Products and Norms 169
6.10 Basic properties of the norm
Suppose v 2 V.
(a) kvk D 0 if and only if v D 0.
(b) kvk D jjkvk for all 2 F.
Proof
(a) (b)
The desired result holds because hv; vi D 0 if and only if v D 0. Suppose 2 F. Then
kvk2 D hv; vi D hv; vi D N h v ; v i
D jj2kvk2: Taking square roots now gives the desired equality.
The proof above of part (b) illustrates a general principle: working with norms squared is usually easier than working directly with norms.
Now we come to a crucial definition.
In the definition above, the order of the vectors does not matter, because hu;vi D 0 if and only if hv;ui D 0. Instead of saying that u and v are orthogonal, sometimes we say that u is orthogonal to v.
Exercise 13 asks you to prove that if u; v are nonzero vectors in R2, then hu; vi D kukkvk cos ;
where is the angle between u and v (thinking of u and v as arrows with initial point at the origin). Thus two vectors in R2 are orthogonal (with respect to the usual Euclidean inner product) if and only if the cosine of the angle between them is 0, which happens if and only if the vectors are perpendicular in the usual sense of plane geometry. Thus you can think of the word orthogonal as a fancy word meaning perpendicular.
6.11 Definition orthogonal
Two vectors u; v 2 V are called orthogonal if hu; vi D 0.
170 CHAPTER 6 Inner Product Spaces
We begin our study of orthogonality with an easy result.
6.12 Orthogonality and 0
(a) 0 is orthogonal to every vector in V.
(b) 0 is the only vector in V that is orthogonal to itself.
Proof
(a) (b)
Part (b) of 6.7 states that h0;ui D 0 for every u 2 V.
If v 2 V and hv;vi D 0, then v D 0 (by definition of inner product).
For the special case V D R2, the next theorem is over 2,500 years old. Of course, the proof below is not the original proof.
The word orthogonal comes from the Greek word orthogonios, which means right-angled.
6.13 Pythagorean Theorem
Suppose u and v are orthogonal vectors in V. Then ku C vk2 D kuk2 C kvk2:
Proof
We have
as desired.
kuCvk2 DhuCv;uCvi
D hu; ui C hu; vi C hv; ui C hv; vi
D kuk2 C kvk2;
Supposeu;v2V,withv¤0. We would like to write u as a scalar multiple of v plus a vector w orthogonal to v, as suggested in the next picture.
The proof given above of the Pythagorean Theorem shows that the conclusion holds if and only if hu;vi C hv;ui, which equals 2 Rehu; vi, is 0. Thus the converse of the Pythagorean Theorem holds in real inner product spaces.
SECTION 6.A Inner Products and Norms 171
u
w
cv v
0
An orthogonal decomposition.
To discover how to write u as a scalar multiple of v plus a vector orthogonal
to v, let c 2 F denote a scalar. Then
u D cv C .u cv/:
Thus we need to choose c so that v is orthogonal to .u cv/. In other words, we want
0 D hu cv; vi D hu; vi ckvk2:
The equation above shows that we should choose c to be hu; vi=kvk2. Making
this choice of c, we can write
hu;vi hu;vi
uDkvk2vC ukvk2v :
As you should verify, the equation above writes u as a scalar multiple of v plus a vector orthogonal to v. In other words, we have proved the following result.
6.14 An orthogonal decomposition
Suppose u;v 2 V, with v ¤ 0. Set c D hu;vi and w D u hu;viv. Then kvk2 kvk2
hw;viD0 and uDcvCw:
The orthogonal decomposition 6.14 will be used in the proof of the Cauchy– Schwarz Inequality, which is our next result and is one of the most important inequalities in mathematics.
French mathematician Augustin- Louis Cauchy (1789–1857) proved 6.17(a) in 1821. German mathe- matician Hermann Schwarz (1843– 1921) proved 6.17(b) in 1886.
172 CHAPTER 6 Inner Product Spaces
6.15 Cauchy–Schwarz Inequality
Suppose u; v 2 V. Then
jhu; vij kuk kvk:
This inequality is an equality if and only if one of u; v is a scalar multiple of the other.
Proof If v D 0, then both sides of the desired inequality equal 0. Thus we can assume that v ¤ 0. Consider the orthogonal decomposition
uD hu;vivCw kvk2
given by 6.14, where w is orthogonal to v. By the Pythagorean Theorem, hu;vi 2
kuk2Dkvk2v Ckwk2 jhu; vij2 2
D kvk2 Ckwk jhu; vij2
6.16
Multiplying both sides of this inequality by kvk2 and then taking square roots gives the desired inequality.
Looking at the proof in the paragraph above, note that the Cauchy–Schwarz Inequality is an equality if and only if 6.16 is an equality. Obviously this happensifandonlyifwD0. ButwD0ifandonlyifuisamultipleofv (see 6.14). Thus the Cauchy–Schwarz Inequality is an equality if and only if u is a scalar multiple of v or v is a scalar multiple of u (or both; the phrasing has been chosen to cover cases in which either u or v equals 0).
6.17 Example examples of the Cauchy–Schwarz Inequality
(a) If x1;:::;xn;y1;:::;yn 2 R, then
jx1y1 CCxnynj2 .x12 CCxn2/.y12 CCyn2/:
(b) If f; g are continuous real-valued functions on Œ1; 1, then
ˇZ 1 ˇ2 Z 1 2 Z 1 2
ˇ f.x/g.x/dxˇ f.x/ dx g.x/ dx : 1 1 1
kvk2 :
The next result, called the Triangle Inequality, has the geometric interpreta- tion that the length of each side of a tri- angle is less than the sum of the lengths of the other two sides.
Note that the Triangle Inequality im- plies that the shortest path between two points is a line segment.
Proof We have
kuCvk2 DhuCv;uCvi
SECTION 6.A
Inner Products and Norms 173
u
uv
v
6.18 Triangle Inequality
Suppose u; v 2 V. Then
kuCvk kukCkvk:
This inequality is an equality if and only if one of u; v is a nonnegative multiple of the other.
6.19 6.20
D hu; ui C hv; vi C hu; vi C hv; ui D hu; ui C hv; vi C hu; vi C hu; vi Dkuk2 Ckvk2 C2Rehu;vi
kuk2 C kvk2 C 2jhu; vij kuk2Ckvk2C2kukkvk
D .kuk C kvk/2;
where 6.20 follows from the Cauchy–Schwarz Inequality (6.15). Taking square roots of both sides of the inequality above gives the desired inequality. The proof above shows that the Triangle Inequality is an equality if and
only if we have equality in 6.19 and 6.20. Thus we have equality in the Triangle Inequality if and only if
6.21 hu; vi D kukkvk:
If one of u;v is a nonnegative multiple of the other, then 6.21 holds, as you should verify. Conversely, suppose 6.21 holds. Then the condition for equality in the Cauchy–Schwarz Inequality (6.15) implies that one of u; v is a scalar multiple of the other. Clearly 6.21 forces the scalar in question to be nonnegative, as desired.
174 CHAPTER 6 Inner Product Spaces
The next result is called the parallelogram equality because of its geometric interpretation: in every parallelogram, the sum of the squares of the lengths of the diagonals equals the sum of the squares of the lengths of the four sides.
u uv
vv uv
u
Proof
We have
as desired.
The parallelogram equality.
6.22 Parallelogram Equality
Suppose u; v 2 V. Then
kuCvk2 Ckuvk2 D2.kuk2 Ckvk2/:
kuCvk2 Ckuvk2 DhuCv;uCviChuv;uvi Dkuk2 Ckvk2 Chu;viChv;ui
Ckuk2 Ckvk2 hu;vihv;ui D 2.kuk2 C kvk2/;
Law professor Richard Friedman presenting a case before the U.S. Supreme Court in 2010:
Mr. Friedman: I think that issue is entirely orthogonal to the issue here because the Commonwealth is acknowledging—
Chief Justice Roberts: I’m sorry. Entirely what?
Mr. Friedman: Orthogonal. Right angle. Unrelated. Irrelevant.
Chief Justice Roberts: Oh.
Justice Scalia: What was that adjective? I liked that.
Mr. Friedman: Orthogonal.
Chief Justice Roberts: Orthogonal.
Mr. Friedman: Right, right.
Justice Scalia: Orthogonal, ooh. (Laughter.)
Justice Kennedy: I knew this case presented us a problem. (Laughter.)
p
2I is invertible.
SECTION 6.A Inner Products and Norms 175
EXERCISES 6.A
1 Show that the function that takes .x1; x2/; .y1; y2/ 2 R2 R2 to jx1y1j C jx2y2j is not an inner product on R2.
2 Show that the function that takes .x1; x2; x3/; .y1; y2; y3/ 2 R3 R3 to x1y1 C x3y3 is not an inner product on R3.
3 Suppose F D R and V ¤ f0g. Replace the positivity condition (which states that hv; vi 0 for all v 2 V ) in the definition of an inner product (6.3) with the condition that hv; vi > 0 for some v 2 V. Show that this change in the definition does not change the set of functions from V V to R that are inner products on V.
4 Suppose V is a real inner product space.
(a) ShowthathuCv;uviDkuk2 kvk2 foreveryu;v2V.
(b) Show that if u; v 2 V have the same norm, then uCv is orthogonal to u v.
(c) Use part (b) to show that the diagonals of a rhombus are perpen- dicular to each other.
5 SupposeT 2L.V/issuchthatkTvkkvkforeveryv2V. Provethat
T
6 Suppose u;v 2 V. Prove that hu;vi D 0 if and only if
kuk kuCavk
for all a 2 F.
7 Supposeu;v2V.ProvethatkauCbvkDkbuCavkforalla;b2R
if and only if kuk D kvk.
8 Supposeu;v2V andkukDkvkD1andhu;viD1. ProvethatuDv.
9 Supposeu;v2V andkuk1andkvk1. Provethat qq
1kuk2 1kvk2 1jhu;vij:
10 Find vectors u; v 2 R2 such that u is a scalar multiple of .1; 3/, v is orthogonal to .1; 3/, and .1; 2/ D u C v.
176 CHAPTER 6 Inner Product Spaces
11 Prove that
1 1 1 1 16.aCbCcCd/ aCbCcCd
for all positive numbers a; b; c; d .
12 Prove that
.x1 CCxn/2 n.x12 CCxn2/ for all positive integers n and all real numbers x1; : : : ; xn.
13 Suppose u; v are nonzero vectors in R2. Prove that hu; vi D kukkvk cos ;
where is the angle between u and v (thinking of u and v as arrows with initial point at the origin).
Hint: Draw the triangle formed by u, v, and u v; then use the law of cosines.
14 The angle between two vectors (thought of as arrows with initial point at the origin) in R2 or R3 can be defined geometrically. However, geometry is not as clear in Rn for n > 3. Thus the angle between two nonzero vectors x; y 2 Rn is defined to be
arccos hx; yi ; kxkkyk
where the motivation for this definition comes from the previous exercise. Explain why the Cauchy–Schwarz Inequality is needed to show that this definition makes sense.
15 Prove that
Xn 2 Xn Xn b j 2 ajbj jaj2
jD1 jD1 jD1 j for all real numbers a1;:::;an and b1;:::;bn.
16 Suppose u; v 2 V are such that
kukD3; kuCvkD4; kuvkD6:
What number does kvk equal?
SECTION 6.A Inner Products and Norms 177 17 Prove or disprove: there is an inner product on R2 such that the associated
associated norm is given by
k.x;y/kD.xp Cyp/1=p forall.x;y/2R2 ifandonlyifpD2.
19 Suppose V is a real inner product space. Prove that hu;viD kuCvk2 kuvk2
4
20 Suppose V is a complex inner product space. Prove that
4
for all u; v 2 V.
21 AnormonavectorspaceU isafunctionkkWU ! Œ0;1/such that kuk D 0 if and only if u D 0, k ̨uk D j ̨jkuk for all ̨ 2 F and all u 2 U, and kuCvk kukCkvk for all u;v 2 U. Prove that a norm satisfying the parallelogram equality comes from an inner product (in other words, show that if k k is a norm on U satisfying the parallelogram equality, then there is an inner product h ; i on U such that kuk D hu;ui1=2 for all u 2 U).
22 Show that the square of an average is less than or equal to the average of the squares. More precisely, show that if a1; : : : ; an 2 R, then the square of the average of a1; : : : ; an is less than or equal to the average of a12;:::;an2.
23 Suppose V1; : : : ; Vm are inner product spaces. Show that the equation h.u1; : : : ; um/; .v1; : : : ; vm/i D hu1; v1i C C hum; vmi
definesaninnerproductonV1 Vm.
[In the expression above on the right, hu1; v1i denotes the inner product on V1, . . . , hum; vmi denotes the inner product on Vm. Each of the spaces V1; : : : ; Vm may have a different inner product, even though the same notation is used here.]
norm is given by for all .x; y/ 2 R2.
k.x; y/k D maxfx; yg
18 Suppose p > 0. Prove that there is an inner product on R2 such that the
for all u; v 2 V.
hu; vi D ku C vk2 ku vk2 C ku C ivk2i ku ivk2i
178 CHAPTER 6 Inner Product Spaces
24 Suppose S 2 L.V / is an injective operator on V. Define h; i1 by hu;vi1 D hSu;Svi
for u; v 2 V. Show that h; i1 is an inner product on V.
25 Suppose S 2 L.V / is not injective. Define h; i1 as in the exercise above.
Explain why h; i1 is not an inner product on V.
26 Suppose f; g are differentiable functions from R to Rn.
(a) Show that
hf.t/;g.t/i0 Dhf0.t/;g.t/iChf.t/;g0.t/i:
(b) Supposec>0andkf.t/kDcforeveryt2R.Showthat
hf0.t/;f.t/iD0foreveryt 2R.
(c) Interpret the result in part (b) geometrically in terms of the tangent
vector to a curve lying on a sphere in Rn centered at the origin.
[For the exercise above, a function f W R ! Rn is called differentiable if there exist differentiable functions f1; : : : ; fn from R to R such that f.t/ D f1.t/;:::;fn.t/ for each t 2 R. Furthermore, for each t 2 R, the derivative f 0.t/ 2 Rn is defined by f 0.t/ D f10.t/; : : : ; fn0.t/.]
27 Suppose u; v; w 2 V. Prove that
1 2 kwuk2 Ckwvk2 kuvk2
kw2.uCv/k D 2 4 :
28 Suppose C is a subset of V with the property that u; v 2 C implies
1.uCv/2C. Letw2V. ShowthatthereisatmostonepointinC 2
that is closest to w. In other words, show that there is at most one u 2 C such that
kwukkwvk forallv2C.
Hint: Use the previous exercise.
29 Foru;v2V,defined.u;v/Dkuvk.
(a) Show that d is a metric on V.
(b) Show that if V is finite-dimensional, then d is a complete metric
on V (meaning that every Cauchy sequence converges).
(c) Show that every finite-dimensional subspace of V is a closed subset of V (with respect to the metric d ).
SECTION 6.A Inner Products and Norms 179 30 Fix a positive integer n. The Laplacian p of a twice differentiable
function p on Rn is the function on Rn defined by p D @2p CC @2p:
@x12 @xn2 The function p is called harmonic if p D 0.
A polynomial on Rn is a linear combination of functions of the form x1m1 xnmn , where m1; : : : ; mn are nonnegative integers.
Suppose q is a polynomial on Rn. Prove that there exists a harmonic polynomial p on Rn such that p.x/ D q.x/ for every x 2 Rn with kxk D 1.
[The only fact about harmonic functions that you need for this exercise isthatifpisaharmonicfunctiononRn andp.x/D0forallx2Rn with kxk D 1, then p D 0.]
Hint: A reasonable guess is that the desired harmonic polynomial p is of the form q C .1 kxk2/r for some polynomial r. Prove that there is a polynomial r on Rn such that q C .1 kxk2/r is harmonic by defining an operator T on a suitable vector space by
T r D .1 kxk2/r
and then showing that T is injective and hence surjective.
31 Use inner products to prove Apollonius’s Identity: In a triangle with sides of length a, b, and c, let d be the length of the line segment from the midpoint of the side of length c to the opposite vertex. Then
a2 Cb2 D 1c2 C2d2: 2
adb
c
180 CHAPTER 6 Inner Product Spaces 6.B Orthonormal Bases
6.23 Definition orthonormal
A list of vectors is called orthonormal if each vector in the list has
norm 1 and is orthogonal to all the other vectors in the list.
In other words, a list e1; : : : ; em of vectors in V is orthonormal if
(
hej;ekiD 1 ifjDk, 0 ifj¤k.
6.24
(a) (b)
(c)
Example orthonormal lists
The standard basis in Fn is an orthonormal list.
11111 3 p3; p3; p3 ; p2; p2;0 is an orthonormal list in F .
1111111 2
p3; p3; p3 ; p2; p2;0 ; p6; p6;p6 is an orthonormal list
in F3.
Orthonormal lists are particularly easy to work with, as illustrated by the
next result.
6.25 The norm of an orthonormal linear combination
If e1; : : : ; em is an orthonormal list of vectors in V, then ka1e1 CCamemk2 Dja1j2 CCjamj2
for all a1;:::;am 2 F.
Proof Because each ej has norm 1, this follows easily from repeated appli- cations of the Pythagorean Theorem (6.13).
The result above has the following important corollary.
6.26 An orthonormal list is linearly independent
Every orthonormal list of vectors is linearly independent.
SECTION 6.B Orthonormal Bases 181 Proof Suppose e1;:::;em is an orthonormal list of vectors in V and
a1;:::;am 2 F are such that
a1e1 C C amem D 0:
Thenja1j2 CCjamj2 D0(by6.25),whichmeansthatalltheaj’sare0. Thus e1; : : : ; em is linearly independent.
For example, the standard basis is an orthonormal basis of Fn.
Proof By 6.26, any such list must be linearly independent; because it has the right length, it is a basis—see 2.39.
6.29 Example Show that
1; 1; 1; 1;1; 1;1;1;1;1;1; 1;1; 1;1; 1
222222222222 2222 is an orthonormal basis of F4.
6.27 Definition orthonormal basis
An orthonormal basis of V is an orthonormal list of vectors in V that is also a basis of V.
6.28 An orthonormal list of the right length is an orthonormal basis
Every orthonormal list of vectors in V with length dim V is an orthonormal basis of V.
Solution We have
q 2 2 2 2
1;1;1;1 D 1 C 1 C 1 C 1 D1: 2222 2222
Similarly, the other three vectors in the list above also have norm 1. We have
̋ 1 ; 1 ; 1 ; 1 ; 1 ; 1 ; 1 ; 1 ̨ D 1 1 C 1 1 C 1 1 C 1 1 D 0: 2222222222222222
Similarly, the inner product of any two distinct vectors in the list above also equals 0.
Thus the list above is orthonormal. Because we have an orthonormal list of length four in the four-dimensional vector space F4, this list is an orthonormal basis of F4 (by 6.28).
182 CHAPTER 6 Inner Product Spaces
In general, given a basis e1;:::;en of V and a vector v 2 V, we know that
there is some choice of scalars a1;:::;an 2 F such that v D a1e1 C C anen:
Computing the numbers a1; : : : ; an that satisfy the equation above can be diffi- cult for an arbitrary basis of V. The next result shows, however, that this is easy for an orthonormal basis—just take aj Dhv;eji.
The importance of orthonormal bases stems mainly from the next result.
6.30 Writing a vector as linear combination of orthonormal basis
Suppose e1;:::;en is an orthonormal basis of V and v 2 V. Then vDhv;e1ie1 CChv;enien
and
kvk2 Djhv;e1ij2 CCjhv;enij2:
Because e1;:::;en is a basis of V, there exist scalars a1;:::;an such v D a1e1 C C anen:
Because e1; : : : ; en is orthonormal, taking the inner product of both sides of thisequationwithej giveshv;ejiDaj.Thusthefirstequationin6.30holds. The second equation in 6.30 follows immediately from the first equation
and 6.25.
Now that we understand the usefulness of orthonormal bases, how do we go about finding them? For example, does Pm.R/, with inner product given by integration on Œ1; 1 [see 6.4(c)], have an orthonormal basis? The next result will lead to answers to these questions.
The algorithm used in the next proof is called the Gram–Schmidt Procedure. It gives a method for turning a linearly independent list into an orthonormal list with the same span as the original list.
Proof
that
Danish mathematician Jørgen Gram (1850–1916) and German mathematician Erhard Schmidt (1876–1959) popularized this algo- rithm that constructs orthonormal lists.
SECTION 6.B Orthonormal Bases 183
6.31 Gram–Schmidt Procedure
Suppose v1; : : : ; vm is a linearly independent list of vectors in V. Let e1 D v1=kv1k. For j D 2;:::;m, define ej inductively by
ej D vj hvj;e1ie1 hvj;ej1iej1 : kvj hvj;e1ie1 hvj;ej1iej1k
Then e1; : : : ; em is an orthonormal list of vectors in V such that span.v1;:::;vj/ D span.e1;:::;ej/
for j D 1;:::;m.
Proof We will show by induction on j that the desired conclusion holds. To get started with j D 1, note that span.v1/ D span.e1/ because v1 is a positive multiple of e1.
Suppose 1 < j < m and we have verified that
6.32 span.v1;:::;vj1/ D span.e1;:::;ej1/:
Note that vj ... span.v1; : : : ; vj 1/ (because v1; : : : ; vm is linearly indepen- dent).Thusvj ...span.e1;:::;ej1/.Hencewearenotdividingby0inthe definition of ej given in 6.31. Dividing a vector by its norm produces a new vector with norm 1; thus kej k D 1.
Let 1 k < j . Then
v hv ;e ie hv ;e ie
hej;ekiD j j 1 1 j j1 j1 ;ek kvj hvj;e1ie1 hvj;ej1iej1k
D hvj;ekihvj;eki
kvj hvj;e1ie1 hvj;ej1iej1k
D 0:
Thuse1;:::;ej isanorthonormallist.
Fromthedefinitionofej givenin6.31,weseethatvj 2span.e1;:::;ej/.
Combining this information with 6.32 shows that span.v1;:::;vj/ span.e1;:::;ej/:
Both lists above are linearly independent (the v’s by hypothesis, the e’s by orthonormality and 6.26). Thus both subspaces above have dimension j , and hence they are equal, completing the proof.
184 CHAPTER 6 Inner Product Spaces
6.33 Example Find an orthonormal basis of P2.R/, where the inner prod- uct is given by hp; qi D R 1 p.x/q.x/ dx.
Solution We will apply the Gram–Schmidt Procedure (6.31) to the basis 1;x;x2.
To get started, with this inner product we have
k1k2 D
p q1
1
12dxD2: Now the numerator in the expression for e2 is
Z1 1
Thusk1kD 2,andhencee1 D 2.
Z1 q q
x hx; e1ie1 D x x 1 dx
1 2 2
1 D x:
We have
Thus kxk D
Now the numerator in the expression for e3 is
x2 hx2; e1ie1 hx2; e2ie2
We have
kxk2 D qq
x2 dx D 2: 2, and hence e2 D 3x.
Z1
1 3
32
Z1 q q Z1 x2 1 dx 1
q q
x2 3xdx 3x 2 2
dxD 8: 45
D x2
D x2 1:
1 2 2 1 Z1
3
kx21k2D x42x2C1 3 1 3 9
2 1 q8 q452 1 Thuskx 3kD 45,andhencee3D 8 x 3 .
Thus
qqq
1; 3x; 45x21 2283
is an orthonormal list of length 3 in P2.R/. Hence this orthonormal list is an orthonormal basis of P2.R/ by 6.28.
SECTION 6.B Orthonormal Bases 185 Now we can answer the question about the existence of orthonormal bases.
Proof Suppose V is finite-dimensional. Choose a basis of V. Apply the Gram–Schmidt Procedure (6.31) to it, producing an orthonormal list with length dim V. By 6.28, this orthonormal list is an orthonormal basis of V.
Sometimes we need to know not only that an orthonormal basis exists, but also that every orthonormal list can be extended to an orthonormal basis. In the next corollary, the Gram–Schmidt Procedure shows that such an extension is always possible.
Proof Suppose e1; : : : ; em is an orthonormal list of vectors in V. Then e1; : : : ; em is linearly independent (by 6.26). Hence this list can be extended to a basis e1;:::;em;v1;:::;vn of V (see 2.33). Now apply the Gram–Schmidt Procedure (6.31) to e1; : : : ; em; v1; : : : ; vn, producing an orthonormal list
6.36 e1;:::;em;f1;:::;fnI
here the formula given by the Gram–Schmidt Procedure leaves the first m vectors unchanged because they are already orthonormal. The list above is an orthonormal basis of V by 6.28.
Recall that a matrix is called upper triangular if all entries below the diagonal equal 0. In other words, an upper-triangular matrix looks like this:
0 1 B@ : : : CA ;
0
where the 0 in the matrix above indicates that all entries below the diagonal equal 0, and asterisks are used to denote entries on and above the diagonal.
6.34 Existence of orthonormal basis
Every finite-dimensional inner product space has an orthonormal basis.
6.35 Orthonormal list extends to orthonormal basis
Suppose V is finite-dimensional. Then every orthonormal list of vectors in V can be extended to an orthonormal basis of V.
186 CHAPTER 6 Inner Product Spaces
In the last chapter we showed that if V is a finite-dimensional complex vector space, then for each operator on V there is a basis with respect to which the matrix of the operator is upper triangular (see 5.27). Now that we are dealing with inner product spaces, we would like to know whether there exists an orthonormal basis with respect to which we have an upper-triangular matrix.
The next result shows that the existence of a basis with respect to which T has an upper-triangular matrix implies the existence of an orthonormal basis with this property. This result is true on both real and complex vector spaces (although on a real vector space, the hypothesis holds only for some operators).
Proof Suppose T has an upper-triangular matrix with respect to some basis v1;:::;vn of V. Thus span.v1;:::;vj/ is invariant under T for each j D 1;:::;n (see 5.26).
Apply the Gram–Schmidt Procedure to v1; : : : ; vn, producing an orthonor- mal basis e1;:::;en of V. Because
span.e1;:::;ej/ D span.v1;:::;vj/
for each j (see 6.31), we conclude that span.e1; : : : ; ej / is invariant under T for each j D 1; : : : ; n. Thus, by 5.26, T has an upper-triangular matrix with respect to the orthonormal basis e1; : : : ; en.
The next result is an important appli- cation of the result above.
6.37 Upper-triangular matrix with respect to orthonormal basis
Suppose T 2 L.V /. If T has an upper-triangular matrix with respect to some basis of V, then T has an upper-triangular matrix with respect to some orthonormal basis of V.
German mathematician Issai Schur (1875–1941) published the first proof of the next result in 1909.
6.38 Schur’s Theorem
Suppose V is a finite-dimensional complex vector space and T 2 L.V /. Then T has an upper-triangular matrix with respect to some orthonormal basis of V.
Proof Recall that T has an upper-triangular matrix with respect to some basis of V (see 5.27). Now apply 6.37.
SECTION 6.B Orthonormal Bases 187 Linear Functionals on Inner Product Spaces
Because linear maps into the scalar field F play a special role, we defined a special name for them in Section 3.F. That definition is repeated below in case you skipped Section 3.F.
6.40 Example The function ' W F3 ! F defined by '.z1;z2;z3/D2z1 5z2 Cz3
is a linear functional on F3. We could write this linear functional in the form '.z/ D hz; ui
for every z 2 F3, where u D .2;5;1/.
6.41 Example The function ' W P2.R/ ! R defined by
6.39 Definition linear functional
A linear functional on V is a linear map from V to F. In other words, a linear functional is an element of L.V; F/.
Z1 p.t/ cos.t/ dt
1
is a linear functional on P2.R/ (here the inner product on P2.R/ is multi- plication followed by integration on Œ1; 1; see 6.33). It is not obvious that there exists u 2 P2.R/ such that
'.p/ D hp; ui
for every p 2 P2.R/ [we cannot take u.t/ D cos.t/ because that function
is not an element of P2.R/].
If u 2 V, then the map that sends v to hv;ui is a linear functional on V. The next result shows that every linear functional on V is of this form. Ex- ample 6.41 above illustrates the power of the next result because for the linear functional in that example, there is no obvious candidate for u.
'.p/ D
The next result is named in honor of Hungarian mathematician Frigyes Riesz (1880–1956), who proved several results early in the twen- tieth century that look very much like the result below.
188 CHAPTER 6 Inner Product Spaces
6.42 Riesz Representation Theorem
Suppose V is finite-dimensional and ' is a linear functional on V. Then there is a unique vector u 2 V such that
for every v 2 V.
'.v/ D hv; ui
Proof First we show there exists a vector u 2 V such that '.v/ D hv; ui for every v 2 V. Let e1;:::;en be an orthonormal basis of V. Then
'.v/D'.hv;e1ie1 CChv;enien/
D hv; e1i'.e1/ C C hv; eni'.en/ D hv; '.e1/e1 C C '.en/eni
for every v 2 V, where the first equality comes from 6.30. Thus setting
6.43 uD'.e1/e1 CC'.en/en;
we have '.v/ D hv; ui for every v 2 V, as desired.
Now we prove that only one vector u 2 V has the desired behavior.
Suppose u1; u2 2 V are such that
'.v/ D hv;u1i D hv;u2i
for every v 2 V. Then
0Dhv;u1ihv;u2iDhv;u1 u2i
for every v 2 V. Taking v D u1 u2 shows that u1 u2 D 0. In other words, u1 D u2, completing the proof of the uniqueness part of the result.
6.44 Example Find u 2 P2.R/ such that Z1Z1
1 for every p 2 P2.R/.
p.t/ cos.t/ dt D p.t/u.t/dt 1
SECTION 6.B Orthonormal Bases 189 Solution Let '.p/ D R 1 p.t/cos.t/ dt. Applying formula 6.43 from
1
the proof above, and using the orthonormal basis from Example 6.33, we have
Z 1 q q Z 1 q q u.x/ D 1 cos.t/ dt 1 C 3t cos.t/ dt 3x
12 212 2 Z1q q
C 45 t21 cos.t/ dt 45 x21 : 183 83
A bit of calculus shows that
u.x/D45 x21: 22 3
Suppose V is finite-dimensional and ' a linear functional on V. Then 6.43 gives a formula for the vector u that satisfies '.v/ D hv;ui for all v 2 V. Specifically, we have
uD'.e1/e1 CC'.en/en:
The right side of the equation above seems to depend on the orthonormal basis e1; : : : ; en as well as on '. However, 6.42 tells us that u is uniquely determined by '. Thus the right side of the equation above is the same regardless of which orthonormal basis e1; : : : ; en of V is chosen.
EXERCISES 6.B
1 (a) Suppose 2 R. Show that .cos ; sin /; . sin ; cos / and .cos ; sin /; .sin ; cos / are orthonormal bases of R2.
(b) Show that each orthonormal basis of R2 is of the form given by one of the two possibilities of part (a).
2 Suppose e1; : : : ; em is an orthonormal list of vectors in V. Let v 2 V. Prove that
kvk2 Djhv;e1ij2 CCjhv;emij2 if and only if v 2 span.e1;:::;em/.
3 Suppose T 2 L.R3/ has an upper-triangular matrix with respect to the basis .1; 0; 0/, (1, 1, 1), .1; 1; 2/. Find an orthonormal basis of R3 (use the usual inner product on R3) with respect to which T has an upper-triangular matrix.
190 CHAPTER 6 Inner Product Spaces
4 Suppose n is a positive integer. Prove that
1 cosx cos2x cosnx sinx sin2x sinnx p2 ; p ; p ; : : : ; p ; p ; p ; : : : ; p
is an orthonormal list of vectors in C Œ; , the vector space of contin- uous real-valued functions on Œ; with inner product
ena such as tides.]
5 On P2.R/, consider the inner product given by
6 Find an orthonormal basis of P2.R/ (with inner product as in Exercise 5) such that the differentiation operator (the operator that takes p to p0) on P2.R/ has an upper-triangular matrix with respect to this basis.
7 Find a polynomial q 2 P2.R/ such that
1 Z1
p 2 D
8 Find a polynomial q 2 P2.R/ such that
Z
hf; gi D
[The orthonormal list above is often used for modeling periodic phenom-
Z1 0
hp; qi D
Apply the Gram–Schmidt Procedure to the basis 1; x; x2 to produce an
orthonormal basis of P2.R/.
f .x/g.x/ dx:
p.x/q.x/ dx:
p.x/q.x/dx
for every p 2 P2.R/.
Z1 Z1
p.x/.cos x/ dx D 00
for every p 2 P2.R/.
p.x/q.x/ dx
0
9 What happens if the Gram–Schmidt Procedure is applied to a list of vectors that is not linearly independent?
SECTION 6.B Orthonormal Bases 191
10 Suppose V is a real inner product space and v1; : : : ; vm is a linearly inde- pendent list of vectors in V. Prove that there exist exactly 2m orthonormal lists e1;:::;em of vectors in V such that
span.v1;:::;vj/ D span.e1;:::;ej/ for all j 2 f1;:::;mg.
11 Suppose h; i1 and h; i2 are inner products on V such that hv; wi1 D 0 if and only if hv; wi2 D 0. Prove that there is a positive number c such that hv;wi1 D chv;wi2 for every v;w 2 V.
12 Suppose V is finite-dimensional and h; i1, h; i2 are inner products on V with corresponding norms k k1 and k k2. Prove that there exists a positive number c such that
kvk1 ckvk2
13 Suppose v1; : : : ; vm is a linearly independent list in V. Show that there
for every v 2 V.
existsw2V suchthathw;vji>0forallj 2f1;:::;mg.
14 Suppose e1;:::;en is an orthonormal basis of V and v1;:::;vn are vectors in V such that
1 kej vj k < pn
for each j. Prove that v1;:::;vn is a basis of V.
15 Suppose CR .Œ1; 1/ is the vector space of continuous real-valued func-
tions on the interval Œ1; 1 with inner product given by Z1
hf; gi D
for f; g 2 CR .Œ1; 1/. Let ' be the linear functional on CR .Œ1; 1/ defined by '.f / D f .0/. Show that there does not exist g 2 CR .Œ1; 1/ such that
'.f / D hf; gi
foreveryf 2CR.Œ1;1/.
[The exercise above shows that the Riesz Representation Theorem (6.42) does not hold on infinite-dimensional vector spaces without additional hypotheses on V and '.]
1
f .x/g.x/ dx
192 16
17
CHAPTER 6 Inner Product Spaces
Suppose F D C, V is finite-dimensional, T 2 L.V /, all the eigenvalues of T have absolute value less than 1, and > 0. Prove that there exists a positive integer m such that kT mvk kvk for every v 2 V.
For u 2 V, let ˆu denote the linear functional on V defined by .ˆu/.v/ D hv; ui
for v 2 V.
(a) ShowthatifFDR,thenˆisalinearmapfromVtoV0.(Recall from Section 3.F that V 0 D L.V; F/ and that V 0 is called the dual space of V.)
(b) ShowthatifFDCandV ¤f0g,thenˆisnotalinearmap.
(c) Show that ˆ is injective.
(d) Suppose F D R and V is finite-dimensional. Use parts (a) and (c) and a dimension-counting argument (but without using 6.42) to show that ˆ is an isomorphism from V onto V 0.
[Part (d) gives an alternative proof of the Riesz Representation Theorem (6.42) when F D R. Part (d) also gives a natural isomorphism (meaning that it does not depend on a choice of basis) from a finite-dimensional real inner product space onto its dual space.]
SECTION 6.C Orthogonal Complements and Minimization Problems 193 6.C Orthogonal Complements and
Minimization Problems
Orthogonal Complements
6.45 Definition orthogonal complement, U ?
If U is a subset of V, then the orthogonal complement of U, denoted U ?,
is the set of all vectors in V that are orthogonal to every vector in U : U? Dfv2V Whv;uiD0foreveryu2Ug:
For example, if U is a line in R3, then U? is the plane containing the origin that is perpendicular to U. If U is a plane in R3, then U ? is the line containing the origin that is perpendicular to U.
6.46 Basic properties of orthogonal complement
(a) IfU isasubsetofV,thenU? isasubspaceofV.
(b) f0g? D V.
(c) V?Df0g.
(d) IfU isasubsetofV,thenU \U? f0g.
(e) IfU andW aresubsetsofV andU W,thenW? U?.
Proof
(a) SupposeU isasubsetofV. Thenh0;uiD0foreveryu2U;thus 0 2 U?.
Suppose v;w 2 U?. If u 2 U, then
hv C w; ui D hv; ui C hw; ui D 0 C 0 D 0:
Thus v C w 2 U ?. In other words, U ? is closed under addition. Similarly, suppose 2 F and v 2 U?. If u 2 U, then
hv; ui D hv; ui D 0 D 0:
Thus v 2 U ?. In other words, U ? is closed under scalar multiplica- tion. Thus U ? is a subspace of V.
194
(b) (c) (d) (e)
CHAPTER 6 Inner Product Spaces
Suppose v 2 V. Then hv; 0i D 0, which implies that v 2 f0g?. Thus
f0g? D V.
Suppose v 2 V?. Then hv;vi D 0, which implies that v D 0. Thus
V ? D f0g. SupposeUisasubsetofVandv2U\U?.Thenhv;viD0,which
impliesthatvD0. ThusU \U? f0g.
SupposeU andW aresubsetsofV andU W. Supposev2W?. Then hv;ui D 0 for every u 2 W, which implies that hv;ui D 0 for everyu2U. Hencev2U?. ThusW? U?.
Recall that if U; W are subspaces of V, then V is the direct sum of U and W (written V D U ̊ W ) if each element of V can be written in exactly one way as a vector in U plus a vector in W (see 1.40).
The next result shows that every finite-dimensional subspace of V leads to a natural direct sum decomposition of V.
6.47 Direct sum of a subspace and its orthogonal complement
Suppose U is a finite-dimensional subspace of V. Then V DU ̊U?:
Proof First we will show that 6.48
V DUCU?:
To do this, suppose v 2 V. Let e1;:::;em be an orthonormal basis of U.
Obviously
6.49 vDhv;e1ie1 CChv;emiemCvhv;e1ie1 hv;emiem: „ ƒ‚ …„ ƒ‚ …
uw
Let u and w be defined as in the equation above. Clearly u 2 U. Because
e1;:::;em is an orthonormal list, for each j D 1;:::;m we have hw;eji D hv;ejihv;eji
D 0:
Thus w is orthogonal to every vector in span.e1; : : : ; em/. In other words, w2U?. ThuswehavewrittenvDuCw,whereu2U andw2U?, completing the proof of 6.48.
From 6.46(d), we know that U \ U ? D f0g. Along with 6.48, this implies that V D U ̊ U ? (see 1.45).
SECTION 6.C Orthogonal Complements and Minimization Problems 195 Now we can see how to compute dim U ? from dim U.
Proof The formula for dim U ? follows immediately from 6.47 and 3.78. The next result is an important consequence of 6.47.
6.50 Dimension of the orthogonal complement
Suppose V is finite-dimensional and U is a subspace of V. Then dimU? D dimV dimU:
6.51 The orthogonal complement of the orthogonal complement
Suppose U is a finite-dimensional subspace of V. Then U D.U?/?:
Proof First we will show that
6.52
U .U?/?:
Todothis,supposeu2U. Thenhu;viD0foreveryv2U? (bythe
definition of U ?). Because u is orthogonal to every vector in U ?, we have u 2 .U ?/?, completing the proof of 6.52.
To prove the inclusion in the other direction, suppose v 2 .U?/?. By 6.47,wecanwritevDuCw,whereu2Uandw2U?. Wehave vuDw2U?. Becausev2.U?/? andu2.U?/? (from6.52),we havevu2.U?/?. Thusvu2U? \.U?/?,whichimpliesthatvu is orthogonal to itself, which implies that v u D 0, which implies that v D u, which implies that v 2 U. Thus .U ?/? U, which along with 6.52 completes the proof.
We now define an operator PU for each finite-dimensional subspace of V.
6.53 Definition orthogonal projection, PU
Suppose U is a finite-dimensional subspace of V. The orthogonal projection of V onto U is the operator PU 2 L.V / defined as follows: Forv2V,writevDuCw,whereu2U andw2U?. ThenPUvDu.
196 CHAPTER 6 Inner Product Spaces
The direct sum decomposition V D U ̊ U ? given by 6.47 shows that eachv2V canbeuniquelywrittenintheformvDuCwwithu2U and w 2 U?. Thus PU v is well defined.
6.54 Example Suppose x 2 V with x ¤ 0 and U D span.x/. Show that PUvD hv;xix
kxk2
for every v 2 V.
Solution Suppose v 2 V. Then
hv;xi hv;xi vDkxk2xC vkxk2x;
where the first term on the right is in span.x/ (and thus in U ) and the second term on the right is orthogonal to x (and thus is in U ?/. Thus PU v equals the first term on the right, as desired.
6.55 Properties of the orthogonal projection PU
Suppose U is a finite-dimensional subspace of V and v 2 V. Then
(a) PU 2L.V/;
(b) PUuDuforeveryu2U;
(c) PUwD0foreveryw2U?;
(d) rangePU DU;
(e) nullPU DU?;
(f) vPUv2U?;
(g) PU2 DPU;
(h) kPU vk kvk;
(i) for every orthonormal basis e1; : : : ; em of U,
PUvDhv;e1ie1 CChv;emiem:
SECTION 6.C Orthogonal Complements and Minimization Problems 197
Proof
(a) To show that PU is a linear map on V, suppose v1; v2 2 V. Write v1Du1Cw1 and v2Du2Cw2
with u1; u2 2 U and w1; w2 2 U ?. Thus PU v1 D u1 and PU v2 D u2. Now
v1 Cv2 D .u1 Cu2/C.w1 Cw2/; whereu1 Cu2 2U andw1 Cw2 2U?. Thus
PU.v1 Cv2/Du1 Cu2 DPUv1 CPUv2:
Similarly,suppose2F. TheequationvDuCwwithu2U and w2U? impliesthatvDuCwwithu2U andw2U?. Thus PU .v/ D u D PU v.
Hence PU is a linear map from V to V.
(b) Supposeu2U. WecanwriteuDuC0,whereu2U and02U?.
Thus PU u D u.
(c) Supposew2U?.WecanwritewD0Cw,where02Uandw2U?.
Thus PU w D 0.
(d) The definition of PU implies that range PU U. Part (b) implies that
U range PU. Thus range PU D U.
(e) Part (c) implies that U ? null PU. To prove the inclusion in the other direction,notethatifv2nullPU thenthedecompositiongivenby6.47 mustbevD0Cv,where02U andv2U?. ThusnullPU U?.
(f) IfvDuCwwithu2U andw2U?,then
v PU v D v u D w 2 U ?:
(g) IfvDuCwwithu2U andw2U?,then
.PU 2/v D PU .PU v/ D PU u D u D PU v:
(h) IfvDuCwwithu2U andw2U?,then
kPU vk2 D kuk2 kuk2 C kwk2 D kvk2;
where the last equality comes from the Pythagorean Theorem.
(i) The formula for PU v follows from equation 6.49 in the proof of 6.47.
198 CHAPTER 6 Inner Product Spaces Minimization Problems
The following problem often arises: given a subspace U of V and a point v 2 V, find a point u 2 U such that kv uk is as small as possible. The next proposition shows that this mini- mization problem is solved by taking u D PU v.
The remarkable simplicity of the so- lution to this minimization problem has led to many important applica- tions of inner product spaces out- side of pure mathematics.
6.56 Minimizing the distance to a subspace
Suppose U is a finite-dimensional subspace of V, v 2 V, and u 2 U. Then kvPUvk kvuk:
Furthermore, the inequality above is an equality if and only if u D PU v.
Proof
6.57
We have
kvPUvk2 kvPUvk2 CkPUvuk2 D k.v PU v/ C .PU v u/k2
D kv uk2;
where the first line above holds because 0 kPU v uk2, the second line above comes from the Pythagorean Theorem [which applies because vPUv2U? by6.55(f),andPUvu2U],andthethirdlineaboveholds by simple algebra. Taking square roots gives the desired inequality.
Our inequality above is an equality if and only if 6.57 is an equality, which happens if and only if kPU v uk D 0, which happens if and only if u D PU v.
v
U
Pv U
0
PU v is the closest point in U to v.
SECTION 6.C Orthogonal Complements and Minimization Problems 199 The last result is often combined with the formula 6.55(i) to compute
explicit solutions to minimization problems.
6.58 Example Find a polynomial u with real coefficients and degree at most 5 that approximates sin x as well as possible on the interval Œ; , in
the sense that
Z
j sin x u.x/j2 dx
is as small as possible. Compare this result to the Taylor series approximation.
Solution Let CR Œ; denote the real inner product space of continuous real-valued functions on Œ; with inner product
Z
Let v 2 CRŒ; be the function defined by v.x/ D sinx. Let U denote the subspace of CR Œ; consisting of the polynomials with real coefficients and degree at most 5. Our problem can now be reformulated as follows:
Find u 2 U such that kv uk is as small as possible.
To compute the solution to our ap-
proximation problem, first apply the
Gram–Schmidt Procedure (using the in-
ner product given by 6.59) to the basis 1; x; x2; x3; x4; x5 of U, producing an orthonormal basis e1; e2; e3; e4; e5; e6 of U. Then, again using the inner product given by 6.59, compute PU v using 6.55(i) (with m D 6). Doing this computation shows that PU v is the function u defined by
6.60 u.x/ D 0:987862x 0:155271×3 C 0:00564312×5;
where the ’s that appear in the exact answer have been replaced with a good decimal approximation.
By 6.56, the polynomial u above is the best approximation to sin x on
6.59 hf; gi D
f .x/g.x/ dx:
Œ; using polynomials of degree at most 5 (here “best approximation”
means in the sense of minimizing R j sin x u.x/j2 dx). To see how good
this approximation is, the next figure shows the graphs of both sin x and our approximation u.x/ given by 6.60 over the interval Œ; .
A computer that can perform inte- grations is useful here.
200 CHAPTER 6 Inner Product Spaces
1
3
1
3
Graphs on Œ; of sin x (blue) and its approximation u.x/ (red) given by 6.60.
Our approximation 6.60 is so accurate that the two graphs are almost identical—our eyes may see only one graph! Here the blue graph is placed almost exactly over the red graph. If you are viewing this on an electronic device, try enlarging the picture above, especially near 3 or 3, to see a small gap between the two graphs.
Another well-known approximation to sin x by a polynomial of degree 5 is given by the Taylor polynomial
x3 x5 6.61 x 3Š C 5Š:
To see how good this approximation is, the next picture shows the graphs of both sin x and the Taylor polynomial 6.61 over the interval Œ; .
1
3
1
3
Graphs on Œ; of sin x (blue) and the Taylor polynomial 6.61 (red).
The Taylor polynomial is an excellent approximation to sin x for x near 0. But the picture above shows that for jxj > 2, the Taylor polynomial is not so accurate, especially compared to 6.60. For example, taking x D 3, our approximation 6.60 estimates sin 3 with an error of about 0:001, but the Taylor series 6.61 estimates sin 3 with an error of about 0:4. Thus at x D 3, the error in the Taylor series is hundreds of times larger than the error given by 6.60. Linear algebra has helped us discover an approximation to sin x that improves upon what we learned in calculus!
SECTION 6.C Orthogonal Complements and Minimization Problems 201 EXERCISES 6.C
1 Suppose v1;:::;vm 2 V. Prove that
fv1;:::;vmg? D span.v1;:::;vm/?:
2 Suppose U is a finite-dimensional subspace of V. Prove that U ? D f0g ifandonlyifU DV.
[Exercise 14(a) shows that the result above is not true without the hy- pothesis that U is finite-dimensional.]
3 Suppose U is a subspace of V with basis u1;:::;um and u1;:::;um;w1;:::;wn
is a basis of V. Prove that if the Gram–Schmidt Procedure is applied to the basis of V above, producing a list e1;:::;em;f1;:::;fn, then e1;:::;em is an orthonormal basis of U and f1;:::;fn is an orthonor- mal basis of U ?.
4 Suppose U is the subspace of R4 defined by
U D span.1; 2; 3; 4/; .5; 4; 3; 2/:
Find an orthonormal basis of U and an orthonormal basis of U ? .
5 Suppose V is finite-dimensional and U is a subspace of V. Show that
PU? DIPU,whereIistheidentityoperatoronV.
6 Suppose U and W are finite-dimensional subspaces of V. Prove that
PUPW D0ifandonlyifhu;wiD0forallu2U andallw2W.
7 Suppose V is finite-dimensional and P 2 L.V / is such that P 2 D P and every vector in null P is orthogonal to every vector in range P . Prove that there exists a subspace U of V such that P D PU.
8 Suppose V is finite-dimensional and P 2 L.V / is such that P 2 D P and
kP vk kvk
for every v 2 V. Prove that there exists a subspace U of V such that
P D PU.
9 Suppose T 2 L.V / and U is a finite-dimensional subspace of V. Prove that U is invariant under T if and only if PU TPU D TPU.
202 CHAPTER 6 Inner Product Spaces
10 Suppose V is finite-dimensional, T 2 L.V /, and U is a subspace of V. Prove that U and U? are both invariant under T if and only ifPUT DTPU.
11 In R4 , let
Findu2U suchthatku.1;2;3;4/kisassmallaspossible.
U D span.1; 1; 0; 0/; .1; 1; 1; 2/: 12 Find p 2 P3.R/ such that p.0/ D 0, p0.0/ D 0, and
Z1
j2 C 3x p.x/j2 dx
0
is as small as possible.
13 Find p 2 P5.R/ that makes
Z
j sin x p.x/j2 dx
as small as possible.
[The polynomial 6.60 is an excellent approximation to the answer to this exercise, but here you are asked to find the exact solution, which involves powers of . A computer that can perform symbolic integration will be useful.]
14 Suppose CR .Œ1; 1/ is the vector space of continuous real-valued func- tions on the interval Œ1; 1 with inner product given by
U Dff 2CR.Œ1;1/Wf.0/D0g:
(a) Show that U ? D f0g.
(b) Show that 6.47 and 6.51 do not hold without the finite-dimensional hypothesis.
Z1 1
hf; gi D
for f; g 2 CR .Œ1; 1/. Let U be the subspace of CR .Œ1; 1/ defined
by
f .x/g.x/ dx
CHAPTER
7
Isaac Newton (1642–1727), as envisioned by British poet and artist William Blake in this 1795 painting.
Operators on Inner Product Spaces
The deepest results related to inner product spaces deal with the subject to which we now turn—operators on inner product spaces. By exploiting properties of the adjoint, we will develop a detailed description of several important classes of operators on inner product spaces.
A new assumption for this chapter is listed in the second bullet point below:
7.1 Notation F, V
F denotes R or C.
V and W denote finite-dimensional inner product spaces over F.
LEARNING OBJECTIVES FOR THIS CHAPTER adjoint
Spectral Theorem
positive operators
isometries
Polar Decomposition
Singular Value Decomposition
© Springer International Publishing 2015 203 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__7
204 CHAPTER 7 Operators on Inner Product Spaces
7.A
Adjoints
Self-Adjoint and Normal Operators
7.2 Definition adjoint, T
Suppose T 2 L.V;W/. The adjoint of T is the function TW W ! V
such that
foreveryv2V andeveryw2W.
hTv;wi D hv;Twi
To see why the definition above makes sense, suppose T 2 L.V; W /. Fix w 2 W. Consider the linear func- tionalonV thatmapsv2V tohTv;wi; this linear functional depends on T and w. By the Riesz Representation Theo- rem (6.42), there exists a unique vector in V such that this linear functional is
The word adjoint has another meaning in linear algebra. We do not need the second meaning in this book. In case you encounter the second meaning for adjoint elsewhere, be warned that the two meanings for adjoint are unrelated to each other.
given by taking the inner product with it. We call this unique vector T w. In other words, T w is the unique vector in V such that hT v; wi D hv; T wi for every v 2 V.
7.3 Example Define T W R3 ! R2 by
T .x1; x2; x3/ D .x2 C 3×3; 2×1/:
Find a formula for T .
Solution Here T will be a function from R2 to R3. To compute T , fix a
point .y1; y2/ 2 R2. Then for every .x1; x2; x3/ 2 R3 we have
h.x1; x2; x3/; T .y1; y2/i D hT .x1; x2; x3/; .y1; y2/i D h.x2 C 3×3; 2×1/; .y1; y2/i
Thus
D x2y1 C 3x3y1 C 2x1y2
D h.x1; x2; x3/; .2y2; y1; 3y1/i:
T .y1; y2/ D .2y2; y1; 3y1/:
SECTION 7.A Self-Adjoint and Normal Operators 205
7.4 Example Fixu2V andx2W.DefineT 2L.V;W/by Tv D hv;uix
for every v 2 V. Find a formula for T .
Solution
Thus
Fix w 2 W. Then for every v 2 V we have
hv;Twi D hTv;wi
D ̋hv; uix; w ̨
D hv; uihx; wi D ̋v; hw; xiu ̨:
T w D hw; xiu:
In the two examples above, T turned out to be not just a function but a linear map. This is true in general, as shown by the next result.
The proofs of the next two results use a common technique: flip T from one side of an inner product to become T on the other side.
Proof SupposeT 2L.V;W/. Fixw1;w2 2W. Ifv2V,then
hv;T.w1 Cw2/i D hTv;w1 Cw2i
D hTv;w1iChTv;w2i
D hv;Tw1iChv;Tw2i D hv; T w1 C T w2i;
whichshowsthatT.w1 Cw2/DTw1 CTw2. Fixw2W and2F. Ifv2V,then
hv; T .w/i D hT v; wi D N h T v ; w i
D N h v ; T w i D hv; T wi;
which shows that T .w/ D T w. Thus T is a linear map, as desired.
7.5 The adjoint is a linear map
IfT 2L.V;W/,thenT 2L.W;V/.
206 CHAPTER 7 Operators on Inner Product Spaces
7.6 Properties of the adjoint
(a) .SCT/DSCTforallS;T2L.V;W/;
(b) .T/ DNT forall2FandT 2L.V;W/;
(c) .T/ DT forallT 2L.V;W/;
(d) IDI,whereIistheidentityoperatoronV;
(e) .ST/ DTS forallT 2L.V;W/andS 2L.W;U/(hereU is an inner product space over F).
Proof
(a) SupposeS;T 2L.V;W/. Ifv2V andw2W,then
hv;.S CT/wi D h.S CT/v;wi
D hSv;wiChTv;wi
D hv; Swi C hv; T wi D hv; Sw C T wi:
Thus .S C T /w D Sw C T w, as desired.
(b) Suppose2FandT 2L.V;W/.Ifv2V andw2W,then
hv;.T/wi D hTv;wi D hTv;wi D hv;Twi D hv;NTwi: Thus .T /w D N T w, as desired.
(c) SupposeT 2L.V;W/. Ifv2V andw2W,then
hw;.T/vi D hTw;vi D hv;Twi D hTv;wi D hw;Tvi:
Thus .T /v D T v, as desired.
(d) Ifv;u2V,then
hv;Iui D hIv;ui D hv;ui: Thus Iu D u, as desired.
(e) SupposeT 2L.V;W/andS 2L.W;U/. Ifv2V andu2U,then hv;.ST/ui D hSTv;ui D hTv;Sui D hv;T.Su/i:
Thus .ST /u D T .Su/, as desired.
SECTION 7.A Self-Adjoint and Normal Operators 207
The next result shows the relationship between the null space and the range of a linear map and its adjoint. The symbol () used in the proof means “if and only if”; this symbol could also be read to mean “is equivalent to”.
7.7 Null space and range of T SupposeT 2L.V;W/.Then
(a) nullT D.rangeT/?;
(b) rangeT D.nullT/?;
(c) nullT D.rangeT/?;
(d) rangeT D.nullT/?.
Proof We begin by proving (a). Let w 2 W. Then
w2nullT ()TwD0
() hv; T wi D 0 for all v 2 V
() hT v; wi D 0 for all v 2 V () w 2 .range T /?:
Thus null T D .range T /?, proving (a).
If we take the orthogonal complement of both sides of (a), we get (d),
where we have used 6.51. Replacing T with T in (a) gives (c), where we have used 7.6(c). Finally, replacing T with T in (d) gives (b).
7.8 Definition conjugate transpose
The conjugate transpose of an m-by-n matrix is the n-by-m matrix ob- tained by interchanging the rows and columns and then taking the complex conjugate of each entry.
7.9 Example
The conjugate transpose of the matrix
If F D R, then the conjugate trans- pose of a matrix is the same as its transpose, which is the matrix ob- tained by interchanging the rows and columns.
23C4i 7
6 5 8i is the matrix
0261 @34i 5 A:
7 8i
208
CHAPTER 7
Operators on Inner Product Spaces
The next result shows how to com- pute the matrix of T from the matrix of T.
Caution: Remember that the result below applies only when we are dealing with orthonormal bases. With respect to nonorthonormal bases, the matrix of T does not necessarily equal the conjugate transpose of the matrix of T.
The adjoint of a linear map does not depend on a choice of basis. This explains why this book em- phasizes adjoints of linear maps instead of conjugate transposes of matrices.
7.10 The matrix of T
LetT 2L.V;W/. Supposee1;:::;en isanorthonormalbasisofV and
f1; : : : ; fm is an orthonormal basis of W. Then MT;.f1;:::;fm/;.e1;:::;en/
is the conjugate transpose of
MT;.e1;:::;en/;.f1;:::;fm/:
In this proof, we will write M.T/ instead of the longer expres- sion MT;.e1;:::;en/;.f1;:::;fm/; we will also write M.T/ instead of MT;.f1;:::;fm/;.e1;:::;en/.
Recall that we obtain the kth column of M.T / by writing T ek as a linear combination of the fj ’s; the scalars used in this linear combination then become the kth column of M.T /. Because f1; : : : ; fm is an orthonormal basis of W, we know how to write T ek as a linear combination of the fj ’s (see 6.30):
Tek DhTek;f1if1 CChTek;fmifm:
Thus the entry in row j , column k, of M.T / is hT ek ; fj i.
Replacing T with T and interchanging the roles played by the e’s and f’s, we see that the entry in row j, column k, of M.T/ is hTfk;eji, which equals hfk ; T ej i, which equals hT ej ; fk i, which equals the complex conjugate of the entry in row k, column j , of M.T /. In other words, M.T /
is the conjugate transpose of M.T /.
Proof
SECTION 7.A Self-Adjoint and Normal Operators 209 Self-Adjoint Operators
Now we switch our attention to operators on inner product spaces. Thus instead of considering linear maps from V to W, we will be focusing on linear maps from V to V ; recall that such linear maps are called operators.
7.11 Definition self-adjoint
An operator T 2 L.V / is called self-adjoint if T D T . In other words,
T 2 L.V / is self-adjoint if and only if hTv;wi D hv;Twi
for all v; w 2 V.
7.12 Example Suppose T is the operator on F2 whose matrix (with re- spect to the standard basis) is
Solution The operator T is self-adjoint if and only if b D 3 (because M.T/ D M.T/ if and only if b D 3; recall that M.T/ is the conjugate transpose of M.T /—see 7.10).
You should verify that the sum of two self-adjoint operators is self-adjoint and that the product of a real scalar and a self-adjoint operator is self-adjoint.
A good analogy to keep in mind (es- pecially when F D C) is that the adjoint on L.V / plays a role similar to complex conjugation on C. A complex number z is real if and only if z D zN; thus a self- adjoint operator (T D T ) is analogous to a real number.
We will see that the analogy discussed above is reflected in some important properties of self-adjoint operators, beginning with eigenvalues in the next result.
If F D R, then by definition every eigenvalue is real, so the next result is interesting only when F D C.
: Find all numbers b such that T is self-adjoint.
2b 37
Some mathematicians use the term Hermitian instead of self-adjoint, honoring French mathematician Charles Hermite, who in 1873 pub- lished the first proof that e is not a zero of any polynomial with integer coefficients.
210 CHAPTER 7 Operators on Inner Product Spaces
7.13 Eigenvalues of self-adjoint operators are real
Every eigenvalue of a self-adjoint operator is real.
Proof Suppose T is a self-adjoint operator on V. Let be an eigenvalue of T, and let v be a nonzero vector in V such that Tv D v. Then
kvk2 D hv;vi D hTv;vi D hv;Tvi D hv;vi D Nkvk2: Thus D N , which means that is real, as desired.
The next result is false for real inner product spaces. As an example, consider the operator T 2 L.R2/ that is a counterclockwise rotation of 90ı around the origin; thus T .x; y/ D .y; x/. Obviously T v is orthogonal to v for every v 2 R2, even though T ¤ 0.
7.14 Over C, T v is orthogonal to v for all v only for the 0 operator Suppose V is a complex inner product space and T 2 L.V /. Suppose
forallv2V. ThenT D0.
hTv;vi D 0
Proof We have
hTu;wiD hT.uCw/;uCwihT.uw/;uwi
4
C hT .u C i w/; u C i wi hT .u i w/; u i wi i
4
for all u; w 2 V, as can be verified by computing the right side. Note that each term on the right side is of the form hT v; vi for appropriate v 2 V. Thus our hypothesis implies that hT u; wi D 0 for all u; w 2 V. This implies that T D0(takewDTu).
The next result is false for real inner product spaces, as shown by consider- ing any operator on a real inner product space that is not self-adjoint.
The next result provides another ex- ample of how self-adjoint opera- tors behave like real numbers.
SECTION 7.A Self-Adjoint and Normal Operators 211
7.15 Over C, hT v; vi is real for all v only for self-adjoint operators Suppose V is a complex inner product space and T 2 L.V /. Then T is
self-adjoint if and only if for every v 2 V.
hTv;vi 2 R
Proof Let v 2 V. Then
hTv;vihTv;vi D hTv;vihv;Tvi D hTv;vihTv;vi D h.T T/v;vi:
If hT v; vi 2 R for every v 2 V, then the left side of the equation above equals 0,soh.T T/v;viD0foreveryv2V. ThisimpliesthatT T D0(by 7.14). Hence T is self-adjoint.
Conversely, if T is self-adjoint, then the right side of the equation above equals 0, so hTv;vi D hTv;vi for every v 2 V. This implies that hTv;vi 2 R for every v 2 V, as desired.
On a real inner product space V, a nonzero operator T might satisfy hT v; vi D 0 for all v 2 V. However, the next result shows that this cannot happen for a self-adjoint operator.
7.16 IfT DT andhTv;viD0forallv,thenT D0 Suppose T is a self-adjoint operator on V such that
forallv2V. ThenT D0.
hTv;vi D 0
Proof We have already proved this (without the hypothesis that T is self- adjoint) when V is a complex inner product space (see 7.14). Thus we can assume that V is a real inner product space. If u; w 2 V, then
7.17 hTu;wiD hT.uCw/;uCwihT.uw/;uwiI 4
this is proved by computing the right side using the equation
hTw;ui D hw;Tui D hTu;wi;
where the first equality holds because T is self-adjoint and the second equality holds because we are working in a real inner product space.
Each term on the right side of 7.17 is of the form hT v; vi for appropriate v. ThushTu;wiD0forallu;w2V.ThisimpliesthatT D0(takewDTu).
212 CHAPTER 7 Operators on Inner Product Spaces Normal Operators
7.18 Definition normal
An operator on an inner product space is called normal if it com-
mutes with its adjoint.
In other words, T 2 L.V / is normal if
TT DTT:
Obviously every self-adjoint operator is normal, because if T is self-adjoint then T D T.
7.19 Example Let T be the operator on F2 whose matrix (with respect to
the standard basis) is
2 3 32:
Show that T is not self-adjoint and that T is normal.
Solution This operator is not self-adjoint because the entry in row 2, column 1 (which equals 3) does not equal the complex conjugate of the entry in row 1, column 2 (which equals 3).
The matrix of T T equals
2 3 2 3 13 0
32 32;whichequals013: Similarly, the matrix of T T equals
2 3 2 3 13 0 32 32;whichequals013:
Because TT and TT have the same matrix, we see that TT D TT. Thus T is normal.
In the next section we will see why normal operators are worthy of special attention.
The next result provides a simple characterization of normal operators.
The next result implies that null T D null T for every normal operator T.
SECTION 7.A Self-Adjoint and Normal Operators 213
7.20 T is normal if and only if kTvk D kTvk for all v An operator T 2 L.V / is normal if and only if
for all v 2 V.
kTvk D kTvk
Proof Let T 2 L.V /. We will prove both directions of this result at the same time. Note that
T is normal () T T T T D 0
()h.TT TT/v;viD0 forallv2V
()hTTv;viDhTTv;vi forallv2V () kTvk2 D kTvk2 for all v 2 V;
where we used 7.16 to establish the second equivalence (note that the operator T T T T is self-adjoint). The equivalence of the first and last conditions above gives the desired result.
Compare the next corollary to Exercise 2. That exercise states that the eigenvalues of the adjoint of each operator are equal (as a set) to the complex conjugates of the eigenvalues of the operator. The exercise says nothing about eigenvectors, because an operator and its adjoint may have different eigenvectors. However, the next corollary implies that a normal operator and its adjoint have the same eigenvectors.
Proof Because T is normal, so is T I, as you should verify. Using 7.20, we have
0Dk.T I/vkDk.T I/vkDk.T NI/vk: Hence v is an eigenvector of T with eigenvalue N , as desired.
Because every self-adjoint operator is normal, the next result applies in particular to self-adjoint operators.
7.21 For T normal, T and T have the same eigenvectors
Suppose T 2 L.V / is normal and v 2 V is an eigenvector of T with eigenvalue . Then v is also an eigenvector of T with eigenvalue N .
214 CHAPTER 7 Operators on Inner Product Spaces
7.22 Orthogonal eigenvectors for normal operators
Suppose T 2 L.V / is normal. Then eigenvectors of T corresponding to distinct eigenvalues are orthogonal.
Proof Suppose ̨; ˇ are distinct eigenvalues of T, with corresponding eigen- vectors u;v. Thus Tu D ̨u and Tv D ˇv. From 7.21 we have Tv D ˇNv. Thus
. ̨ ˇ/hu; vi D h ̨u; vi hu; ˇNvi D hTu;vihu;Tvi
D 0:
Because ̨ ¤ ˇ, the equation above implies that hu; vi D 0. Thus u and v are
orthogonal, as desired.
EXERCISES 7.A
1 Suppose n is a positive integer. Define T 2 L.Fn/ by T.z1;:::;zn/ D .0;z1;:::;zn1/:
Find a formula for T.z1;:::;zn/.
2 SupposeT 2L.V/and2F. ProvethatisaneigenvalueofT ifand
only if N is an eigenvalue of T .
3 Suppose T 2 L.V / and U is a subspace of V. Prove that U is invariant
under T if and only if U ? is invariant under T .
4 Suppose T 2 L.V; W /. Prove that
(a) T is injective if and only if T is surjective;
(b) T is surjective if and only if T is injective.
5 Prove that
dimnullT D dimnullT CdimW dimV
and
foreveryT 2L.V;W/.
dim range T D dim range T
SECTION 7.A Self-Adjoint and Normal Operators 215
6 Make P2.R/ into an inner product space by defining
Z1
p.x/q.x/ dx: DefineT 2LP2.R/byT.a0Ca1xCa2x2/Da1x.
(a) Show that T is not self-adjoint.
(b) The matrix of T with respect to the basis .1; x; x2/ is
00001 @010A:
000
This matrix equals its conjugate transpose, even though T is not
self-adjoint. Explain why this is not a contradiction.
7 Suppose S; T 2 L.V / are self-adjoint. Prove that ST is self-adjoint if
andonlyifST DTS.
8 Suppose V is a real inner product space. Show that the set of self-adjoint
operators on V is a subspace of L.V /.
9 Suppose V is a complex inner product space with V ¤ f0g. Show that
the set of self-adjoint operators on V is not a subspace of L.V /.
10 Suppose dim V 2. Show that the set of normal operators on V is not a
subspace of L.V /.
11 SupposeP 2L.V/issuchthatP2 DP. Provethatthereisasubspace
hp; qi D
0
U of V such that P D PU if and only if P is self-adjoint.
12 Suppose that T is a normal operator on V and that 3 and 4 are eigenvalues
p
of T. Prove that there exists a vector v 2 V such that kvk D 2 and
kTvk D 5.
13 Give an example of an operator T 2 L.C4/ such that T is normal but
not self-adjoint.
14 Suppose T is a normal operator on V. Suppose also that v; w 2 V satisfy the equations
kvkDkwkD2; TvD3v; TwD4w: Show that kT .v C w/k D 10.
216 CHAPTER 7 Operators on Inner Product Spaces 15 Fixu;x2V.DefineT 2L.V/by
Tv D hv;uix
(a) Suppose F D R. Prove that T is self-adjoint if and only if u; x is
for every v 2 V.
linearly dependent.
(b) Prove that T is normal if and only if u; x is linearly dependent.
16 Suppose T 2 L.V / is normal. Prove that rangeT DrangeT:
17 Suppose T 2 L.V / is normal. Prove that
nullTk D nullT and rangeTk D rangeT
for every positive integer k.
18 Prove or give a counterexample: If T 2 L.V / and there exists an ortho- normal basis e1;:::;en of V such that kTejk D kTejk for each j, then T is normal.
19 Suppose T 2 L.C3/ is normal and T .1; 1; 1/ D .2; 2; 2/. Suppose .z1;z2;z3/2nullT.Provethatz1Cz2Cz3 D0.
20 SupposeT 2L.V;W/andFDR.LetˆV betheisomorphismfromV onto the dual space V 0 given by Exercise 17 in Section 6.B, and let ˆW be the corresponding isomorphism from W onto W 0. Show that if ˆV and ˆW areusedtoidentifyVandW withV0 andW0,thenT isidentified withthedualmapT0. Moreprecisely,showthatˆV ıT DT0 ıˆW.
21 Fix a positive integer n. In the inner product space of continuous real- valued functions on Œ; with inner product
Z
let
V D span.1;cosx;cos2x;:::;cosnx;sinx;sin2x;:::;sinnx/:
(a) (b)
DefineD2L.V/byDf Df0.ShowthatD DD.Conclude that D is normal but not self-adjoint.
Define T 2 L.V / by Tf D f 00. Show that T is self-adjoint.
hf; gi D
f .x/g.x/ dx;
SECTION 7.B The Spectral Theorem 217 7.B The Spectral Theorem
Recall that a diagonal matrix is a square matrix that is 0 everywhere except possibly along the diagonal. Recall also that an operator on V has a diagonal matrix with respect to a basis if and only if the basis consists of eigenvectors of the operator (see 5.41).
The nicest operators on V are those for which there is an orthonormal basis of V with respect to which the operator has a diagonal matrix. These are precisely the operators T 2 L.V / such that there is an orthonormal basis of V consisting of eigenvectors of T. Our goal in this section is to prove the Spectral Theorem, which characterizes these operators as the normal operators when F D C and as the self-adjoint operators when F D R. The Spectral Theorem is probably the most useful tool in the study of operators on inner product spaces.
Because the conclusion of the Spectral Theorem depends on F, we will break the Spectral Theorem into two pieces, called the Complex Spectral Theorem and the Real Spectral Theorem. As is often the case in linear algebra, complex vector spaces are easier to deal with than real vector spaces. Thus we present the Complex Spectral Theorem first.
The Complex Spectral Theorem
The key part of the Complex Spectral Theorem (7.24) states that if F D C and T 2 L.V / is normal, then T has a diagonal matrix with respect to some orthonormal basis of V. The next example illustrates this conclusion.
7.23 Example Consider the normal operator T 2 L.C2/ from Example 7.19, whose matrix (with respect to the standard basis) is
2 3 32:
.i;1/ .i;1/ 2
As you can verify, p2 ; p2 is an orthonormal basis of C consisting of
eigenvectors of T, and with respect to this basis the matrix of T is the diagonal
matrix
2C3i 0 0 23i :
In the next result, the equivalence of (b) and (c) is easy (see 5.41). Thus we prove only that (c) implies (a) and that (a) implies (c).
218 CHAPTER 7 Operators on Inner Product Spaces
7.24 Complex Spectral Theorem
Suppose F D C and T 2 L.V /. Then the following are equivalent:
(a) T is normal.
(b) V has an orthonormal basis consisting of eigenvectors of T.
(c) T has a diagonal matrix with respect to some orthonormal basis of V.
Proof First suppose (c) holds, so T has a diagonal matrix with respect to some orthonormal basis of V. The matrix of T (with respect to the same basis) is obtained by taking the conjugate transpose of the matrix of T ; hence T also has a diagonal matrix. Any two diagonal matrices commute; thus T commutes with T , which means that T is normal. In other words, (a) holds.
Now suppose (a) holds, so T is normal. By Schur’s Theorem (6.38), there is an orthonormal basis e1; : : : ; en of V with respect to which T has an upper-triangular matrix. Thus we can write
01 a1;1 ::: a1;n
7.25 MT;.e1;:::;en/ DB@ ::: : CA: 0 an;n
We will show that this matrix is actually a diagonal matrix. We see from the matrix above that
and
kTe1k2 D ja1;1j2
kT e1k2 D ja1;1j2 C ja1;2j2 C C ja1;nj2:
BecauseT isnormal,kTe1kDkTe1k(see7.20).Thusthetwoequations above imply that all entries in the first row of the matrix in 7.25, except possibly the first entry a1;1, equal 0.
Now from 7.25 we see that
kTe2k2 D ja2;2j2
(because a1;2 D 0, as we showed in the paragraph above) and
kT e2k2 D ja2;2j2 C ja2;3j2 C C ja2;nj2:
Because T is normal, kT e2 k D kT e2 k. Thus the two equations above imply that all entries in the second row of the matrix in 7.25, except possibly the diagonal entry a2;2, equal 0.
Continuing in this fashion, we see that all the nondiagonal entries in the matrix 7.25 equal 0. Thus (c) holds.
The Real Spectral Theorem
SECTION 7.B The Spectral Theorem 219
We will need a few preliminary results, which apply to both real and complex inner product spaces, for our proof of the Real Spectral Theorem.
You could guess that the next result is true and even discover its proof by thinking about quadratic polynomials with real coefficients. Specifically, sup- poseb;c2Randb2 <4c.Letxbea real number. Then
2 b2 b2 xCbxCcDxC2 Cc4 >0:
In particular, x2 C bx C c is an invertible real number (a convoluted way of saying that it is not 0). Replacing the real number x with a self-adjoint operator (recall the analogy between real numbers and self-adjoint operators), we are led to the result below.
7.26 Invertible quadratic expressions
Suppose T 2 L.V / is self-adjoint and b; c 2 R are such that b2 < 4c. Then
is invertible.
T2 CbT CcI
Proof
Let v be a nonzero vector in V. Then
h.T2 CbT CcI/v;viDhT2v;viCbhTv;viCchv;vi D hTv;TviCbhTv;viCckvk2
kT vk2 jbjkT vkkvk C ckvk2
jbjkvk2 b2 2
DkTvk 2 Cc4kvk
> 0;
where the third line above holds by the Cauchy–Schwarz Inequality (6.15). Thelastinequalityimpliesthat.T2 CbT CcI/v¤0. ThusT2 CbT CcI is injective, which implies that it is invertible (see 3.69).
We know that every operator, self-adjoint or not, on a finite-dimensional nonzero complex vector space has an eigenvalue (see 5.21). Thus the next result tells us something new only for real inner product spaces.
This technique of completing the square can be used to derive the quadratic formula.
220 CHAPTER 7 Operators on Inner Product Spaces
7.27 Self-adjoint operators have eigenvalues
Suppose V ¤ f0g and T 2 L.V / is a self-adjoint operator. Then T has an eigenvalue.
Proof We can assume that V is a real inner product space, as we have already noted.LetnDdimV andchoosev2V withv¤0.Then
v;Tv;T2v;:::;Tnv
cannot be linearly independent, because V has dimension n and we have n C 1
vectors. Thus there exist real numbers a0; : : : ; an, not all 0, such that 0 D a0vCa1TvCCanTnv:
Make the a’s the coefficients of a polynomial, which can be written in factored form (see 4.17) as
a0 C a1x C C anxn
Dc.x2 Cb1xCc1/.x2 CbMxCcM/.x1/.xm/;
where c is a nonzero real number, each bj , cj , and j is real, each bj 2 is less than 4cj , m C M 1, and the equation holds for all real x. We then have
0 D a0vCa1TvCCanTnv
D .a0I C a1T C C anT n/v
Dc.T2 Cb1T Cc1I/.T2 CbMT CcMI/.T 1I/.T mI/v:
By 7.26, each T 2 C bj T C cj I is invertible. Recall also that c ¤ 0. Thus the equation above implies that m > 0 and
0D.T 1I/.T mI/v:
Hence T j I is not injective for at least one j . In other words, T has an
eigenvalue.
The next result shows that if U is a subspace of V that is invariant under a self-adjoint operator T, then U ? is also invariant under T. Later we will show that the hypothesis that T is self-adjoint can be replaced with the weaker hypothesis that T is normal (see 9.30).
SECTION 7.B The Spectral Theorem 221
7.28 Self-adjoint operators and invariant subspaces
Suppose T 2 L.V / is self-adjoint and U is a subspace of V that is invariant under T. Then
(a) U ? is invariant under T ;
(b) TjU 2L.U/isself-adjoint;
(c) TjU? 2L.U?/isself-adjoint.
Proof To prove (a), suppose v 2 U ?. Let u 2 U. Then hTv;ui D hv;Tui D 0;
where the first equality above holds because T is self-adjoint and the second equality above holds because U is invariant under T (and hence T u 2 U ) and because v 2 U ?. Because the equation above holds for each u 2 U, we conclude that T v 2 U ?. Thus U ? is invariant under T, completing the proof of (a).
To prove (b), note that if u; v 2 U, then
h.T jU /u; vi D hT u; vi D hu; T vi D hu; .T jU /vi:
ThusTjU isself-adjoint.
Now (c) follows from replacing U with U ? in (b), which makes sense
by (a).
We can now prove the next result, which is one of the major theorems in linear algebra.
7.29 Real Spectral Theorem
Suppose F D R and T 2 L.V /. Then the following are equivalent:
(a) T is self-adjoint.
(b) V has an orthonormal basis consisting of eigenvectors of T.
(c) T has a diagonal matrix with respect to some orthonormal basis of V.
222 CHAPTER 7 Operators on Inner Product Spaces
Proof First suppose (c) holds, so T has a diagonal matrix with respect to some orthonormal basis of V. A diagonal matrix equals its transpose. Hence T D T , and thus T is self-adjoint. In other words, (a) holds.
We will prove that (a) implies (b) by induction on dim V. To get started, note that if dim V D 1, then (a) implies (b). Now assume that dim V > 1 and that (a) implies (b) for all real inner product spaces of smaller dimension.
Suppose (a) holds, so T 2 L.V / is self-adjoint. Let u be an eigenvector of T with kuk D 1 (7.27 guarantees that T has an eigenvector, which can then be divided by its norm to produce an eigenvector with norm 1). Let U D span.u/. Then U is a 1-dimensional subspace of V that is invariant underT.By7.28(c),theoperatorTjU? 2L.U?/isself-adjoint.
By our induction hypothesis, there is an orthonormal basis of U ? consist- ing of eigenvectors of T jU ? . Adjoining u to this orthonormal basis of U ? gives an orthonormal basis of V consisting of eigenvectors of T, completing the proof that (a) implies (b).
We have proved that (c) implies (a) and that (a) implies (b). Clearly (b) implies (c), completing the proof.
7.30 Example Consider the self-adjoint operator T on R3 whose matrix (with respect to the standard basis) is
As you can verify,
014 13 81 @13 14 8A:
8 8 7
.1;1;0/ .1;1;1/ .1;1;2/ p;p;p
236
is an orthonormal basis of R3 consisting of eigenvectors of T, and with respect
to this basis, the matrix of T is the diagonal matrix
0 27 0 0 1 @090A:
0 0 15
If F D C, then the Complex Spectral Theorem gives a complete descrip- tion of the normal operators on V. A complete description of the self-adjoint operators on V then easily follows (they are the normal operators on V whose eigenvalues all are real; see Exercise 6).
If F D R, then the Real Spectral Theorem gives a complete description of the self-adjoint operators on V. In Chapter 9, we will give a complete description of the normal operators on V (see 9.34).
EXERCISES 7.B
SECTION 7.B The Spectral Theorem 223
1 True or false (and give a proof of your answer): There exists T 2 L.R3/ such that T is not self-adjoint (with respect to the usual inner product) and such that there is a basis of R3 consisting of eigenvectors of T.
2 Suppose that T is a self-adjoint operator on a finite-dimensional inner product space and that 2 and 3 are the only eigenvalues of T. Prove that T 2 5T C 6I D 0.
3 Give an example of an operator T 2 L.C3/ such that 2 and 3 are the onlyeigenvaluesofT andT2 5T C6I ¤0.
4 SupposeFDCandT2L.V/.ProvethatTisnormalifandonlyif all pairs of eigenvectors corresponding to distinct eigenvalues of T are orthogonal and
V DE.1;T/ ̊ ̊E.m;T/; where 1; : : : ; m denote the distinct eigenvalues of T.
5 Suppose F D R and T 2 L.V /. Prove that T is self-adjoint if and only if all pairs of eigenvectors corresponding to distinct eigenvalues of T are orthogonal and
V DE.1;T/ ̊ ̊E.m;T/; where 1; : : : ; m denote the distinct eigenvalues of T.
6 Prove that a normal operator on a complex inner product space is self- adjoint if and only if all its eigenvalues are real.
[The exercise above strengthens the analogy (for normal operators) between self-adjoint operators and real numbers.]
7 Suppose V is a complex inner product space and T 2 L.V / is a normal operator such that T 9 D T 8. Prove that T is self-adjoint and T 2 D T.
8 Give an example of an operator T on a complex vector space such that T9 DT8 butT2 ¤T.
9 Suppose V is a complex inner product space. Prove that every normal operator on V has a square root. (An operator S 2 L.V / is called a squarerootofT 2L.V/ifS2 DT.)
224 CHAPTER 7 Operators on Inner Product Spaces
10 Give an example of a real inner product space V and T 2 L.V / and real numbers b; c with b2 < 4c such that T 2 C bT C cI is not invertible. [The exercise above shows that the hypothesis that T is self-adjoint is needed in 7.26, even for real vector spaces.]
11 Prove or give a counterexample: every self-adjoint operator on V has a cuberoot. (AnoperatorS 2L.V/iscalledacuberootofT 2L.V/if S3 D T.)
12 Suppose T 2 L.V / is self-adjoint, 2 F, and > 0. Suppose there existsv2V suchthatkvkD1and
kT v vk < :
Prove that T has an eigenvalue 0 such that j 0j < .
13 Give an alternative proof of the Complex Spectral Theorem that avoids Schur’s Theorem and instead follows the pattern of the proof of the Real Spectral Theorem.
14 Suppose U is a finite-dimensional real vector space and T 2 L.U /. Prove that U has a basis consisting of eigenvectors of T if and only if there is an inner product on U that makes T into a self-adjoint operator.
15 Find the matrix entry below that is covered up.
SECTION 7.C Positive Operators and Isometries 225 7.C Positive Operators and Isometries
Positive Operators
7.31 Definition positive operator
An operator T 2 L.V / is called positive if T is self-adjoint and
for all v 2 V.
hTv;vi 0
If V is a complex vector space, then the requirement that T is self-adjoint can be dropped from the definition above (by 7.15).
7.32 Example positive operators
(a) If U is a subspace of V, then the orthogonal projection PU is a positive
operator, as you should verify.
(b) If T 2 L.V/ is self-adjoint and b;c 2 R are such that b2 < 4c, then
T 2 C bT C cI is a positive operator, as shown by the proof of 7.26.
7.34 Example If T 2 L.F3/ is defined by T .z1; z2; z3/ D .z3; 0; 0/, then the operator R 2 L.F3/ defined by R.z1;z2;z3/ D .z2;z3;0/ is a square root of T.
The characterizations of the positive
operators in the next result correspond
to characterizations of the nonnegative
numbers among C. Specifically, a com-
plex number z is nonnegative if and
only if it has a nonnegative square root,
corresponding to condition (c). Also,
z is nonnegative if and only if it has a real square root, corresponding to condition (d). Finally, z is nonnegative if and only if there exists a complex numberwsuchthatzDwNw,correspondingtocondition(e).
7.33 Definition square root
An operator R is called a square root of an operator T if R2 D T.
The positive operators correspond to the numbers Œ0;1/, so better terminology would use the term nonnegative instead of positive. However, operator theorists consis- tently call these the positive opera- tors, so we will follow that custom.
226 CHAPTER 7 Operators on Inner Product Spaces
7.35 Characterization of positive operators
Let T 2 L.V /. Then the following are equivalent:
(a) T is positive;
(b) T is self-adjoint and all the eigenvalues of T are nonnegative;
(c) T has a positive square root;
(d) T has a self-adjoint square root;
(e) there exists an operator R 2 L.V / such that T D RR.
Proof Wewillprovethat(a))(b))(c))(d))(e))(a).
First suppose (a) holds, so that T is positive. Obviously T is self-adjoint (by the definition of a positive operator). To prove the other condition in (b), suppose is an eigenvalue of T. Let v be an eigenvector of T corresponding
to . Then
0 hTv;vi D hv;vi D hv;vi:
Thus is a nonnegative number. Hence (b) holds.
Now suppose (b) holds, so that T is self-adjoint and all the eigenvalues
of T are nonnegative. By the Spectral Theorem (7.24 and 7.29), there is
an orthonormal basis e1; : : : ; en of V consisting of eigenvectors of T. Let
1; : : : ; n be the eigenvalues of T corresponding to e1; : : : ; en,; thus each
j is a nonnegative number. Let R be the linear map from V to V such that q
RejD jej
for j D 1; : : : ; n (see 3.5). Then R is a positive operator, as you should verify. Furthermore,R2ej Djej DTej foreachj,whichimpliesthatR2 DT. Thus R is a positive square root of T. Hence (c) holds.
Clearly (c) implies (d) (because, by definition, every positive operator is self-adjoint).
Now suppose (d) holds, meaning that there exists a self-adjoint operator RonV suchthatT DR2. ThenT DRR(becauseR DR). Hence(e) holds.
Finally, suppose (e) holds. Let R 2 L.V / be such that T D RR. Then T D .RR/ D R.R/ D RR D T. Hence T is self-adjoint. To complete the proof that (a) holds, note that
hTv;vi D hRRv;vi D hRv;Rvi 0 for every v 2 V. Thus T is positive.
Each nonnegative number has a unique nonnegative square root. The next result shows that positive operators enjoy a similar property.
Proof Suppose T 2 L.V / is positive. Suppose v 2 V is an eigenvector of T. Thus there exists 0 such that Tv D v.
Let R be a positive square root of T.
SECTION 7.C Positive Operators and Isometries 227
7.36 Each positive operator has only one positive square root
Every positive operator on V has a unique positive square root.
A positive operator can have in- finitely many square roots (al- though only one of them can be positive). For example, the identity operator on V has infinitely many square roots if dim V > 1.
We will prove that Rv D
p
v. This
will imply that the behavior of R on the eigenvectors of T is uniquely deter-
mined. Because there is a basis of V consisting of eigenvectors of T (by the
Spectral Theorem), this will imply that R is uniquely determined.
To prove that Rv D
p
v, note that the Spectral Theorem asserts that
there is an orthonormal basis e1; : : : ; en of V consisting of eigenvectors of R.
Because R is a positive operator, all its eigenvalues are nonnegative. Thus
there exist nonnegative numbers 1; : : : ; n such that Rej D j D1;:::;n.
Because e1;:::;en is a basis of V, we can write v D a1e1 C C anen
for some numbers a1;:::;an 2 F. Thus pp
RvDa1 1e1CCan nen
and hence
R2vDa11e1 CCannen:
But R2 D T, and T v D v. Thus the equation above implies
p
j ej for
a1e1 CCanen Da11e1 CCannen: Theequationaboveimpliesthataj.j/D0forj D1;:::;n. Hence
and thus
X
fjWjDg
Xpp
aj ejD v;
fjWjDg
v D
aj ej ;
as desired.
RvD
Some mathematicians also use the term positive semidefinite opera- tor, which means the same as posi- tive operator.
228 CHAPTER 7 Operators on Inner Product Spaces Isometries
Operators that preserve norms are sufficiently important to deserve a name:
7.37 Definition isometry
An operator S 2 L.V / is called an isometry if
kSvk D kvk
for all v 2 V.
In other words, an operator is an isometry if it preserves norms.
For example, I is an isometry whenever 2 F satisfies jj D 1. We will see soon that if F D C, then the next example includes all isometries.
7.38 Example Suppose 1; : : : ; n are scalars with absolute value 1 and S 2L.V/satisfiesSej Djej forsomeorthonormalbasise1;:::;en ofV. Show that S is an isometry.
The Greek word isos means equal; the Greek word metron means measure. Thus isometry literally means equal measure.
Suppose v 2 V. Then
vDhv;e1ie1 CChv;enien
kvk2 D jhv; e1ij2 C C jhv; enij2;
where we have used 6.30. Applying S to both sides of 7.39 gives
SvDhv;e1iSe1 CChv;eniSen
D 1hv; e1ie1 C C nhv; enien:
The last equation, along with the equation jj j D 1, shows that 7.41 kSvk2 Djhv;e1ij2 CCjhv;enij2:
Comparing 7.40 and 7.41 shows that kvk D kSvk. In other words, S is an isometry.
Solution
7.39
and
7.40
SECTION 7.C Positive Operators and Isometries 229
The next result provides several con-
ditions that are equivalent to being an
isometry. The equivalence of (a) and (b)
shows that an operator is an isometry if
and only if it preserves inner products.
The equivalence of (a) and (c) [or (d)]
shows that an operator is an isometry
if and only if the list of columns of its
matrix with respect to every [or some] basis is orthonormal. Exercise 10 implies that in the previous sentence we can replace “columns” with “rows”.
An isometry on a real inner product space is often called an orthogonal operator. An isometry on a com- plex inner product space is often called a unitary operator. We use the term isometry so that our re- sults can apply to both real and complex inner product spaces.
7.42 Characterization of isometries
Suppose S 2 L.V /. Then the following are equivalent:
(a) S is an isometry;
(b) hSu;Svi D hu;vi for all u;v 2 V ;
(c) S e1 ; : : : ; S en is orthonormal for every orthonormal list of vectors e1;:::;en inV;
(d) there exists an orthonormal Se1;:::;Sen isorthonormal;
(e) SSDI;
(f) SSDI;
(g) S is an isometry;
(h) S is invertible and S1 D S.
basis
e1; : : : ; en
of V such that
Proof First suppose (a) holds, so S is an isometry. Exercises 19 and 20 in Section 6.A show that inner products can be computed from norms. Because S preserves norms, this implies that S preserves inner products, and hence (b) holds. More precisely, if V is a real inner product space, then for every u;v2V wehave
hSu;SviD.kSuCSvk2 kSuSvk2/=4 D .kS.u C v/k2 kS.u v/k2/=4
D.kuCvk2 kuvk2/=4 D hu; vi;
230 CHAPTER 7 Operators on Inner Product Spaces
where the first equality comes from Exercise 19 in Section 6.A, the second equality comes from the linearity of S , the third equality holds because S is an isometry, and the last equality again comes from Exercise 19 in Section 6.A. If V is a complex inner product space, then use Exercise 20 in Section 6.A instead of Exercise 19 to obtain the same conclusion. In either case, we see that (b) holds.
Now suppose (b) holds, so S preserves inner products. Suppose that e1; : : : ; en is an orthonormal list of vectors in V. Then we see that the list Se1;:::;Sen isorthonormalbecausehSej;SekiDhej;eki.Thus(c)holds.
Clearly (c) implies (d).
Now suppose (d) holds. Let e1; : : : ; en be an orthonormal basis of V such that Se1; : : : ; Sen is orthonormal. Thus
hSSej;eki D hej;eki
for j;k D 1;:::;n [because the term on the left equals hSej;Seki and .Se1; : : : ; Sen/ is orthonormal]. All vectors u; v 2 V can be written as linear combinations of e1; : : : ; en, and thus the equation above implies that hSSu;vi D hu;vi. Hence SS D I; in other words, (e) holds.
Now suppose (e) holds, so that SS D I. In general, an operator S need not commute with S. However, SS D I if and only if SS D I; this is a special case of Exercise 10 in Section 3.D. Thus SS D I, showing that (f) holds.
Now suppose (f) holds, so SS D I. If v 2 V, then
kSvk2 D hSv; Svi D hSSv; vi D hv; vi D kvk2:
Thus S is an isometry, showing that (g) holds.
Now suppose (g) holds, so S is an isometry. We know that (a) ) (e) and
(a))(f)becausewehaveshown(a))(b))(c))(d))(e))(f). Using the implications (a) ) (e) and (a) ) (f) but with S replaced with S [and using the equation .S/ D S], we conclude that SS D I and SS D I. Thus S is invertible and S1 D S; in other words, (h) holds.
Now suppose (h) holds, so S is invertible and S1 D S. Thus SS D I. If v 2 V, then
kSvk2 D hSv; Svi D hSSv; vi D hv; vi D kvk2:
Thus S is an isometry, showing that (a) holds.
We have shown (a) ) (b) ) (c) ) (d) ) (e) ) (f) ) (g) ) (h) ) (a),
completing the proof.
SECTION 7.C Positive Operators and Isometries 231
The previous result shows that every isometry is normal [see (a), (e), and (f) of 7.42]. Thus characterizations of normal operators can be used to give descriptions of isometries. We do this in the next result in the complex case and in Chapter 9 in the real case (see 9.36).
7.43 Description of isometries when F D C
Suppose V is a complex inner product space and S 2 L.V /. Then the
following are equivalent:
(a) S is an isometry.
(b) There is an orthonormal basis of V consisting of eigenvectors of S whose corresponding eigenvalues all have absolute value 1.
Proof We have already shown (see Example 7.38) that (b) implies (a).
To prove the other direction, suppose (a) holds, so S is an isometry. By the Complex Spectral Theorem (7.24), there is an orthonormal basis e1; : : : ; en of V consisting of eigenvectors of S. For j 2 f1;:::;ng, let j be the
eigenvalue corresponding to ej . Then
jjj D kjejk D kSejk D kejk D 1:
Thus each eigenvalue of S has absolute value 1, completing the proof.
EXERCISES 7.C
1 Prove or give a counterexample: If T 2 L.V / is self-adjoint and there existsanorthonormalbasise1;:::;en ofV suchthathTej;eji0for each j , then T is a positive operator.
2 Suppose T is a positive operator on V. Suppose v; w 2 V are such that TvDw and TwDv:
Prove that v D w.
3 Suppose T is a positive operator on V and U is a subspace of V invariant
underT.ProvethatTjU 2L.U/isapositiveoperatoronU.
4 Suppose T 2 L.V; W /. Prove that T T is a positive operator on V and T T is a positive operator on W.
232 CHAPTER 7 Operators on Inner Product Spaces
5 Prove that the sum of two positive operators on V is positive.
6 Suppose T 2 L.V / is positive. Prove that T k is positive for every positive integer k.
7 Suppose T is a positive operator on V. Prove that T is invertible if and only if
hTv;vi > 0
foreveryv2V withv¤0.
8 SupposeT 2L.V/. Foru;v2V,definehu;viT by
hu;viT DhTu;vi:
Prove that h; iT is an inner product on V if and only if T is an invertible
positive operator (with respect to the original inner product h; i).
9 Prove or disprove: the identity operator on F2 has infinitely many self-
adjoint square roots.
10 Suppose S 2 L.V /. Prove that the following are equivalent:
(a) S is an isometry;
(b) hSu;Svi D hu;vi for all u;v 2 V ;
(c) S e1 ; : : : ; S em is an orthonormal list for every orthonormal list of vectors e1;:::;em in V ;
(d) Se1; : : : ; Sen is an orthonormal basis for some orthonormal basis e1;:::;en of V.
11 Suppose T1; T2 are normal operators on L.F3/ and both operators have 2; 5; 7 as eigenvalues. Prove that there exists an isometry S 2 L.F3/ such that T1 D ST2S.
12 Give an example of two self-adjoint operators T1; T2 2 L.F4/ such that the eigenvalues of both operators are 2; 5; 7 but there does not exist an isometry S 2 L.F4/ such that T1 D ST2S. Be sure to explain why there is no isometry with the required property.
13 Prove or give a counterexample: if S 2 L.V / and there exists an ortho- normalbasise1;:::;en ofV suchthatkSejkD1foreachej,thenS is an isometry.
14 Let T be the second derivative operator in Exercise 21 in Section 7.A. Show that T is a positive operator.
SECTION 7.D Polar Decomposition and Singular Value Decomposition 233 7.D Polar Decomposition and Singular
Value Decomposition
Polar Decomposition
Recall our analogy between C and L.V /. Under this analogy, a complex number z corresponds to an operator T, and zN corresponds to T . The real numbers (z D zN) correspond to the self-adjoint operators (T D T ), and the nonnegative numbers correspond to the (badly named) positive operators.
Another distinguished subset of C is the unit circle, which consists of the complex numbers z such that jzj D 1. The condition jzj D 1 is equivalent to the condition zNz D 1. Under our analogy, this would correspond to the condition T T D I, which is equivalent to T being an isometry (see 7.42). In other words, the unit circle in C corresponds to the isometries.
Continuing with our analogy, note that each complex number z except 0 can be written in the form
z z p zD jzj jzjD jzj zNz;
where the first factor, namely, z=jzj, is an element of the unit circle. Our analogy leads us to guess that each operator T 2 L.V / can be written as an isometry times pT T . That guess is indeed correct, as we now prove after defining the obvious notation, which is justified by 7.36.
Now we can state and prove the Polar Decomposition, which gives a beautiful description of an arbitrary operator on V. Note that TT is a positive operator for every T 2 L.V /, and thus pT T is well defined.
7.44 Notation
p
p
T
If T is a positive operator, then root of T.
T denotes the unique positive square
7.45 Polar Decomposition
Suppose T 2 L.V /. Then there exists an isometry S 2 L.V / such that
p TDS TT:
234
Proof
CHAPTER 7 Operators on Inner Product Spaces If v 2 V, then
kTvk2 D hTv;Tvi D hTTv;vi pp
Dh TT TTv;vi pp
Dh TTv; TTvi D kpT T vk2:
p kTvkDk TTvk
Thus
7.46
forallv2V.
DefinealinearmapS1W range T T !rangeT by
7.47 S1.
p
p
TTv/ D Tv:
The idea of the proof is to extend S1 to an isometry S 2 L.V / such that T D SpT T . Now for the details.
First we must check that S1 is well defined. To do this, suppose v1; v2 2 V are such that pT T v1 D pT T v2. For the definition given by 7.47 to make sense, we must show that T v1 D T v2. Note that
kTv1 Tv2k D kT.v1 v2/k p
Dk TT.v1v2/k pp
Dk TTv1 TTv2k D 0;
where the second equality holds by 7.46. The equation above shows that T v1 D T v2, so S1 is indeed well defined. You should verify that S1 is a linear map. p
We see from 7.47 that S1 maps range T T onto range T. Clearly 7.46 and 7.47 imply that
for all u 2 rangepTT.
kS1uk D kuk
In particular, S1 is injective. Thus from the Fundamental Theorem of Lin- ear Maps (3.22), applied to S1, we have
p
The rest of the proof extends S1 to an isometry S on all of V.
dimrange
TT DdimrangeT:
SECTION 7.D Polar Decomposition and Singular Value Decomposition 235
This implies that dim.range pT T /? D dim.range T /? (see 6.50). Thus orthonormal bases e1;:::;em of .rangepTT/? and f1;:::;fm of .range T /? can be chosen; the key point here is that these two ortho- normal bases have the same length (denoted m). Now define a linear map S2W .rangepTT/? ! .rangeT/? by
S2.a1e1 CCamem/Da1f1 CCamfm:
For all w 2 .range pT T /?, we have kS2wk D kwk (from 6.25).
Now let S be the operator on V that equals S1 on range pT T and equals S2 on .range pT T /?. More precisely, recall that each v 2 V can be written
uniquely in the form
7.48 v D u C w;
where u 2 rangepTT and w 2 .rangepTT/? (see 6.47). For v 2 V
with decomposition as above, define Sv by
Sv D S1u C S2w:
Foreachv2V wehave pp
S. TTv/ D S1. TTv/ D Tv;
so T D SpT T , as desired. All that remains is to show that S is an isometry. However, this follows easily from two uses of the Pythagorean Theorem: if v 2 V has decomposition as in 7.48, then
kSvk2 D kS1u C S2wk2 D kS1uk2 C kS2wk2 D kuk2 C kwk2 D kvk2I the second equality holds because S1u 2 range T and S2w 2 .range T /?.
The Polar Decomposition (7.45) states that each operator on V is the product of an isometry and a positive operator. Thus we can write each operator on V as the product of two operators, each of which comes from a class that we can completely describe and that we understand reasonably well. The isometries are described by 7.43 and 9.36; the positive operators are described by the Spectral Theorem (7.24 and 7.29). p
Specifically, consider the case F D C, and suppose T D S T T is a Polar Decomposition of an operator T 2 L.V /, where S is an isometry. Then there is an orthonormal basis of V with respect to which S has a diagonal matrix, and there is an orthonormal basis of V with respect to which pT T has a diagonal matrix. Warning: there may not exist an orthonormal basis that simultaneously puts the matrices of both S and pT T into these nice diagonal forms. In other words, S may require one orthonormal basis and pT T may require a different orthonormal basis.
236 CHAPTER 7 Operators on Inner Product Spaces Singular Value Decomposition
The eigenvalues of an operator tell us something about the behavior of the operator. Another collection of numbers, called the singular values, is also useful. Recall that eigenspaces and the notation E are defined in 5.36.
The singular values of T are all nonnegative, because they are the eigen- values of the positive operator pT T .
7.50 Example Define T 2 L.F4/ by
T .z1; z2; z3; z4/ D .0; 3z1; 2z2; 3z4/:
Find the singular values of T.
Solution A calculation shows T T .z1; z2; z3; z4/ D .9z1; 4z2; 0; 9z4/, as you should verify. Thus
p
T T .z1; z2; z3; z4/ D .3z1; 2z2; 0; 3z4/; and we see that the eigenvalues of pT T are 3; 2; 0 and
ppp
dimE.3; TT/ D 2; dimE.2; TT/ D 1; dimE.0; TT/ D 1:
Hence the singular values of T are 3; 3; 2; 0.
Note that 3 and 0 are the only eigenvalues of T. Thus in this case, the
collection of eigenvalues did not pick up the number 2 that appears in the definition (and hence the behavior) of T, but the collection of singular values does include 2.
Each T 2 L.V / has dim V singular values, as can be seen by applying the Spectral Theorem and 5.41 [see especially part (e)] to the positive (hence self-adjoint) operator pT T . For example, the operator T defined in Exam- ple 7.50 on the four-dimensional vector space F4 has four singular values (they are 3; 3; 2; 0), as we saw above.
The next result shows that every operator on V has a clean description in terms of its singular values and two orthonormal bases of V.
7.49 Definition singular values
Suppose T 2 L.V /. The singular values of T are the eigenvalues of pT T , with each eigenvalue repeated dim E.; pT T / times.
SECTION 7.D Polar Decomposition and Singular Value Decomposition 237
7.51 Singular Value Decomposition
Suppose T 2 L.V / has singular values s1; : : : ; sn. Then there exist orthonormal bases e1;:::;en and f1;:::;fn of V such that
for every v 2 V.
Tv D s1hv;e1if1 CCsnhv;enifn
Proof By the Spectral Theorem applied to pT T , there is an orthonormal basise1;:::;en ofV suchthatpTTej Dsjej forj D1;:::;n.
getting
TTv D s1hv;e1ie1 CCsnhv;enien
for every v 2 V. By the Polar Decomposition (see 7.45), there is an isometry S 2 L.V / such that T D SpT T . Apply S to both sides of the equation above, getting
Tv D s1hv;e1iSe1 CCsnhv;eniSen
for every v 2 V. For each j, let fj D Sej. Because S is an isometry, f1; : : : ; fn is an orthonormal basis of V (see 7.42). The equation above now becomes
Tv D s1hv;e1if1 CCsnhv;enifn for every v 2 V, completing the proof.
When we worked with linear maps from one vector space to a second vector space, we considered the matrix of a linear map with respect to a basis of the first vector space and a basis of the second vector space. When dealing with operators, which are linear maps from a vector space to itself, we almost always use only one basis, making it play both roles.
The Singular Value Decomposition allows us a rare opportunity to make
good use of two different bases for the matrix of an operator. To do this,
suppose T 2 L.V /. Let s1; : : : ; sn denote the singular values of T, and let
e1;:::;en and f1;:::;fn be orthonormal bases of V such that the Singular
ValueDecomposition7.51holds.BecauseTej Dsjfj foreachj,wehave 01
Bs10C M T;.e1;:::;en/;.f1;:::;fn/ D @ ::: A:
We have
for every v 2 V (see 6.30). Apply pT T to both sides of this equation,
vDhv;e1ie1 CChv;enien p
0 sn
238 CHAPTER 7 Operators on Inner Product Spaces
In other words, every operator on V has a diagonal matrix with respect to some orthonormal bases of V, provided that we are permitted to use two different bases rather than a single basis as customary when working with operators.
Singular values and the Singular Value Decomposition have many applica- tions (some are given in the exercises), including applications in computational linear algebra. To compute numeric approximations to the singular values of an operator T, first compute T T and then compute approximations to the eigenvalues of T T (good techniques exist for approximating eigenvalues of positive operators). The nonnegative square roots of these (approximate) eigenvalues of T T will be the (approximate) singular values of T. In other words, the singular values of T can be approximated without computing the square root of T T. The next result helps justify working with T T instead of pT T .
7.52 Singular values without taking square root of an operator
Suppose T 2 L.V /. Then the singular values of T are the nonnegative square roots of the eigenvalues of T T, with each eigenvalue repeated dim E.; T T / times.
Proof The Spectral Theorem implies that there are an orthonormal basis e1;:::;en and nonnegative numbers 1;:::;n such that TTej D jej forj D1;:::;n. ItiseasytoseethatpTTej Dpjej forj D1;:::;n, which implies the desired result.
EXERCISES 7.D
1 Fixu;x2V withu¤0.DefineT 2L.V/by Tv D hv;uix
for every v 2 V. Prove that
pTTvD kxkhv;uiu
kuk
for every v 2 V.
2 Give an example of T 2 L.C2/ such that 0 is the only eigenvalue of T and the singular values of T are 5; 0.
SECTION 7.D Polar Decomposition and Singular Value Decomposition 239 3 Suppose T 2 L.V /. Prove that there exists an isometry S 2 L.V / such
that
p
TD TTS:
4 Suppose T 2 L.V / and s is a singular value of T. Prove that there exists avectorv2V suchthatkvkD1andkTvkDs.
5 Suppose T 2 L.C2/ is defined by T .x; y/ D .4y; x/. Find the singu- lar values of T.
6 Find the singular values of the differentiation operator D 2 P .R2 / defined by Dp D p0, where the inner product on P.R2/ is as in Example 6.33.
7 Define T 2 L.F3 / by
T .z1; z2; z3/ D .z3; 2z1; 3z2/:
Find (explicitly) an isometry S 2 L.F3/ such that T D SpT T .
8 SupposeT2L.V/,S2L.V/isanisometry,andR2L.V/isa positive operator such that T D SR. Prove that R D pT T .
[The exercise above shows that if we write T as the product of an isometry and a positive operator (as in the Polar Decomposition 7.45), then the positive operator equals pT T .]
9 Suppose T 2 L.V /. Prove that T is invertible if and only if there exists auniqueisometryS 2L.V/suchthatT DSpTT.
10 Suppose T 2 L.V / is self-adjoint. Prove that the singular values of T equal the absolute values of the eigenvalues of T, repeated appropriately.
11 Suppose T 2 L.V /. Prove that T and T have the same singular values.
12 Prove or give a counterexample: if T 2 L.V /, then the singular values
of T 2 equal the squares of the singular values of T.
13 Suppose T 2 L.V /. Prove that T is invertible if and only if 0 is not a
singular value of T.
14 Suppose T 2 L.V /. Prove that dim range T equals the number of
nonzero singular values of T.
15 Suppose S 2 L.V /. Prove that S is an isometry if and only if all the singular values of S equal 1.
240 CHAPTER 7 Operators on Inner Product Spaces
16 Suppose T1; T2 2 L.V /. Prove that T1 and T2 have the same singular values if and only if there exist isometries S1; S2 2 L.V / such that T1 D S1T2S2.
17 Suppose T 2 L.V / has singular value decomposition given by Tv D s1hv;e1if1 CCsnhv;enifn
for every v 2 V, where s1;:::;sn are the singular values of T and e1;:::;en and f1;:::;fn are orthonormal bases of V.
(a) Prove that if v 2 V, then
Tv D s1hv;f1ie1 CCsnhv;fnien:
(b) Prove that if v 2 V, then
TTv D s12hv;e1ie1 CCsn2hv;enien:
(c) Prove that if v 2 V, then
p
TTv D s1hv;e1ie1 CCsnhv;enien:
(d) Suppose T is invertible. Prove that if v 2 V, then
T1vDhv;f1ie1 CChv;fnien
s1 sn
for every v 2 V.
18 Suppose T 2 L.V /. Let sO denote the smallest singular value of T, and
let s denote the largest singular value of T.
(a) Prove that sOkvk kT vk skvk for every v 2 V.
(b) Suppose is an eigenvalue of T. Prove that sO jj s.
19 Suppose T 2 L.V /. Show that T is uniformly continuous with respect
tothemetricd onV definedbyd.u;v/Dkuvk.
20 Suppose S;T 2 L.V/. Let s denote the largest singular value of S, let t denote the largest singular value of T, and let r denote the largest singular value of S C T. Prove that r s C t.
CHAPTER
8
Hypatia, the 5th century Egyptian mathematician and philosopher, as envisioned around 1900 by Alfred Seifert.
Operators on Complex Vector Spaces
In this chapter we delve deeper into the structure of operators, with most of the attention on complex vector spaces. An inner product does not help with this material, so we return to the general setting of a finite-dimensional vector space. To avoid some trivialities, we will assume that V ¤ f0g. Thus our assumptions for this chapter are as follows:
8.1 Notation F, V
F denotes R or C.
V denotes a finite-dimensional nonzero vector space over F.
LEARNING OBJECTIVES FOR THIS CHAPTER generalized eigenvectors and generalized eigenspaces characteristic polynomial and the Cayley–Hamilton Theorem decomposition of an operator
minimal polynomial
Jordan Form
© Springer International Publishing 2015 241 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__8
242 CHAPTER 8 Operators on Complex Vector Spaces
8.A Generalized Eigenvectors and Nilpotent
Operators
Null Spaces of Powers of an Operator
We begin this chapter with a study of null spaces of powers of an operator.
8.2 Sequence of increasing null spaces
Suppose T 2 L.V /. Then
f0g D nullT0 nullT1 nullTk nullTkC1 :
Proof Suppose k is a nonnegative integer and v 2 null T k. Then T kv D 0, and hence TkC1v D T.Tkv/ D T.0/ D 0. Thus v 2 nullTkC1. Hence null T k null T kC1, as desired.
The next result says that if two consecutive terms in this sequence of subspaces are equal, then all later terms in the sequence are equal.
8.3 Equality in the sequence of null spaces
Suppose T 2 L.V /. Suppose m is a nonnegative integer such that null T m D null T mC1. Then
nullTm D nullTmC1 D nullTmC2 D nullTmC3 D :
Proof Let k be a positive integer. We want to prove that nullTmCk DnullTmCkC1:
We already know from 8.2 that null T mCk null T mCkC1.
To prove the inclusion in the other direction, suppose v 2 null T mCkC1.
Then Hence
TmC1.Tkv/ D TmCkC1v D 0: Tkv 2 nullTmC1 D nullTm:
Thus TmCkv D Tm.Tkv/ D 0, which means that v 2 nullTmCk. This implies that null T mCkC1 null T mCk , completing the proof.
SECTION 8.A Generalized Eigenvectors and Nilpotent Operators 243
The proposition above raises the question of whether there exists a non- negative integer m such that null T m D null T mC1. The proposition below shows that this equality holds at least when m equals the dimension of the vector space on which T operates.
Proof We need only prove that null T n D null T nC1 (by 8.3). Suppose this is not true. Then, by 8.2 and 8.3, we have
f0g D nullT0 ̈ nullT1 ̈ ̈ nullTn ̈ nullTnC1;
where the symbol ̈ means “contained in but not equal to”. At each of the strict inclusions in the chain above, the dimension increases by at least 1. Thus dim null T nC1 n C 1, a contradiction because a subspace of V cannot have a larger dimension than n.
Unfortunately, it is not true that V D null T ̊ range T for each T 2 L.V /. However, the following result is a useful substitute.
Proof First we show that
8.6 .null T n/ \ .range T n/ D f0g:
Supposev2.nullTn/\.rangeTn/. ThenTnvD0,andthereexistsu2V such that v D T nu. Applying T n to both sides of the last equation shows that Tnv D T2nu. Hence T2nu D 0, which implies that Tnu D 0 (by 8.4). Thus v D T nu D 0, completing the proof of 8.6.
Now 8.6 implies that null T n C range T n is a direct sum (by 1.45). Also, dim.nullTn ̊rangeTn/DdimnullTn CdimrangeTn DdimV;
where the first equality above comes from 3.78 and the second equality comes from the Fundamental Theorem of Linear Maps (3.22). The equation above implies that null T n ̊ range T n D V, as desired.
8.4 Null spaces stop growing
SupposeT 2L.V/.LetnDdimV.Then
nullTn D nullTnC1 D nullTnC2 D :
8.5 V is the direct sum of nullTdimV and rangeTdimV SupposeT 2L.V/.LetnDdimV.Then
V DnullTn ̊rangeTn:
244 CHAPTER 8 Operators on Complex Vector Spaces
8.7 Example Suppose T 2 L.F3/ is defined by T .z1; z2; z3/ D .4z2; 0; 5z3/:
For this operator, null T C range T is not a direct sum of subspaces, because nullT D f.z1;0;0/ W z1 2 Fg and rangeT D f.z1;0;z3/ W z1;z3 2 Fg. ThusnullT \rangeT ¤f0gandhencenullT CrangeT isnotadirectsum. Also note that null T C range T ¤ F3.
However, we have T 3.z1; z2; z3/ D .0; 0; 125z3/. Thus we see that nullT3 D f.z1;z2;0/ W z1;z2 2 Fg and rangeT3 D f.0;0;z3/ W z3 2 Fg. HenceF3 DnullT3 ̊rangeT3.
Generalized Eigenvectors
Unfortunately, some operators do not have enough eigenvectors to lead to a good description. Thus in this subsection we introduce the concept of generalized eigenvectors, which will play a major role in our description of the structure of an operator.
To understand why we need more than eigenvectors, let’s examine the question of describing an operator by decomposing its domain into invariant subspaces. Fix T 2 L.V /. We seek to describe T by finding a “nice” direct sum decomposition
V DU1 ̊ ̊Um;
where each Uj is a subspace of V invariant under T. The simplest possible nonzero invariant subspaces are 1-dimensional. A decomposition as above where each Uj is a 1-dimensional subspace of V invariant under T is possible if and only if V has a basis consisting of eigenvectors of T (see 5.41). This happens if and only if V has an eigenspace decomposition
8.8 V DE.1;T/ ̊ ̊E.m;T/;
where 1; : : : ; m are the distinct eigenvalues of T (see 5.41).
The Spectral Theorem in the previous chapter shows that if V is an inner product space, then a decomposition of the form 8.8 holds for every normal operator if F D C and for every self-adjoint operator if F D R because operators of those types have enough eigenvectors to form a basis of V (see
7.24 and 7.29).
SECTION 8.A Generalized Eigenvectors and Nilpotent Operators 245
Sadly, a decomposition of the form 8.8 may not hold for more general oper- ators, even on a complex vector space. An example was given by the operator in 5.43, which does not have enough eigenvectors for 8.8 to hold. General- ized eigenvectors and generalized eigenspaces, which we now introduce, will remedy this situation.
8.9 Definition generalized eigenvector
SupposeT 2L.V/andisaneigenvalueofT. Avectorv2V iscalled
a generalized eigenvector of T corresponding to if v ¤ 0 and .T I /j v D 0
for some positive integer j .
Although j is allowed to be an arbi- trary integer in the equation
.T I /j v D 0
in the definition of a generalized eigen- vector, we will soon prove that every generalized eigenvector satisfies this equation with j D dim V.
Note that we do not define the con- cept of a generalized eigenvalue, because this would not lead to any- thing new. Reason: if .T I /j is not injective for some positive inte- ger j , then T I is not injective, and hence is an eigenvalue of T.
8.10 Definition generalized eigenspace, G.; T /
Suppose T 2 L.V / and 2 F. The generalized eigenspace of T corre- sponding to , denoted G.; T /, is defined to be the set of all generalized eigenvectors of T corresponding to , along with the 0 vector.
Because every eigenvector of T is a generalized eigenvector of T (take j D 1 in the definition of generalized eigenvector), each eigenspace is contained in the corresponding generalized eigenspace. In other words, if T 2L.V/and2F,then
E.;T/ G.;T/:
The next result implies that if T 2 L.V/ and 2 F, then G.;T/ is a subspace of V (because the null space of each linear map on V is a subspace of V ).
246 CHAPTER 8 Operators on Complex Vector Spaces
8.11 Description of generalized eigenspaces
Suppose T 2 L.V / and 2 F. Then G.; T / D null.T I /dim V .
Proof Suppose v 2 null.T I /dim V . The definitions imply v 2 G.; T /. ThusG.;T/null.T I/dimV.
Conversely, suppose v 2 G.; T /. Thus there is a positive integer j such that
v 2 null.T I /j :
From8.2and8.4(withT I replacingT),wegetv2null.T I/dimV.
Thus G.; T / null.T I /dim V , completing the proof.
8.12 Example Define T 2 L.C3/ by
T .z1; z2; z3/ D .4z2; 0; 5z3/:
(a) Find all eigenvalues of T, the corresponding eigenspaces, and the corresponding generalized eigenspaces.
(b) Show that C3 is the direct sum of generalized eigenspaces correspond- ing to the distinct eigenvalues of T.
Solution
(a) A routine use of the definition of eigenvalue shows that the eigenvalues of T are 0 and 5. The corresponding eigenspaces are easily seen to be E.0;T/ D f.z1;0;0/ W z1 2 Cg and E.5;T/ D f.0;0;z3/ W z3 2 Cg.
Note that this operator T does not have enough eigenvectors to span its domain C3.
We have T3.z1;z2;z3/ D .0;0;125z3/ for all z1;z2;z3 2 C. Thus 8.11 implies that G.0;T/ D f.z1;z2;0/ W z1;z2 2 Cg.
We have .T 5I /3.z1; z2; z3/ D .125z1 C 300z2; 125z2; 0/. Thus 8.11 implies that G.5;T/ D f.0;0;z3/ W z3 2 Cg.
(b) Theresultsinpart(a)showthatC3 DG.0;T/ ̊G.5;T/.
SECTION 8.A Generalized Eigenvectors and Nilpotent Operators 247
One of our major goals in this chapter is to show that the result in part (b) of the example above holds in general for operators on finite-dimensional complex vector spaces; we will do this in 8.21.
We saw earlier (5.10) that eigenvectors corresponding to distinct eigenval- ues are linearly independent. Now we prove a similar result for generalized eigenvectors.
8.13 Linearly independent generalized eigenvectors
Let T 2 L.V /. Suppose 1; : : : ; m are distinct eigenvalues of T and v1; : : : ; vm are corresponding generalized eigenvectors. Then v1; : : : ; vm is linearly independent.
Proof Suppose a1; : : : ; am are complex numbers such that
8.14 0 D a1v1 C C amvm:
Let k be the largest nonnegative integer such that .T 1 I /k v1 ¤ 0. Let
Thus
wD.T 1I/kv1:
.T 1I/wD.T 1I/kC1wD0;
and hence T w D 1w. Thus .T I /w D .1 /w for every 2 F and hence
8.15 .T I/nwD.1 /nw for every 2 F, where n D dim V.
Apply the operator
.T 1I/k.T 2I/n.T mI/n to both sides of 8.14, getting
0Da1.T 1I/k.T 2I/n.T mI/nv1 Da1.T 2I/n.T mI/nw
Da1.1 2/n.1 m/nw;
where we have used 8.11 to get the first equation above and 8.15 to get the last equation above.
The equation above implies that a1 D 0. In a similar fashion, aj D 0 for each j , which implies that v1; : : : ; vm is linearly independent.
248 CHAPTER 8 Operators on Complex Vector Spaces Nilpotent Operators
8.17 Example nilpotent operators
(a) The operator N 2 L.F4/ defined by
N.z1; z2; z3; z4/ D .z3; z4; 0; 0/
is nilpotent because N 2 D 0.
(b) The operator of differentiation on Pm.R/ is nilpotent because the .m C 1/st derivative of every polynomial of degree at most m equals 0. Note that on this space of dimension m C 1, we need to raise the nilpotent operator to the power m C 1 to get the 0 operator.
The next result shows that we never need to use a power higher than the di- mension of the space.
Proof Because N is nilpotent, G.0;N/ D V. Thus 8.11 implies that nullNdimV DV,asdesired.
Given an operator T on V, we want to find a basis of V such that the matrix of T with respect to this basis is as simple as possible, meaning that the matrix contains many 0’s.
The next result shows that if N is nilpotent, then we can choose a basis of V such that the matrix of N with respect to this basis has more than half of its entries equal to 0. Later in this chapter we will do even better.
8.16 Definition nilpotent
An operator is called nilpotent if some power of it equals 0.
The Latin word nil means noth- ing or zero; the Latin word potent means power. Thus nilpotent liter- ally means zero power.
8.18 Nilpotent operator raised to dimension of domain is 0 SupposeN2L.V/isnilpotent.ThenNdimV D0.
If V is a complex vector space, a proof of the next result follows eas- ily from Exercise 7, 5.27, and 5.32. But the proof given here uses sim- pler ideas than needed to prove 5.27, and it works for both real and complex vector spaces.
SECTION 8.A Generalized Eigenvectors and Nilpotent Operators 249
8.19 Matrix of a nilpotent operator
Suppose N is a nilpotent operator on V. Then there is a basis of V with respect to which the matrix of N has the form
00 1 B@ : : : CA I
00
here all entries on and below the diagonal are 0’s.
Proof First choose a basis of null N. Then extend this to a basis of null N 2 . Then extend to a basis of null N 3 . Continue in this fashion, eventually getting a basis of V (because 8.18 states that null N dim V D V ).
Now let’s think about the matrix of N with respect to this basis. The first column, and perhaps additional columns at the beginning, consists of all 0’s, because the corresponding basis vectors are in null N. The next set of columns comes from basis vectors in null N 2. Applying N to any such vector, we get a vector in null N ; in other words, we get a vector that is a linear combination of the previous basis vectors. Thus all nonzero entries in these columns lie above the diagonal. The next set of columns comes from basis vectors in null N 3. Applying N to any such vector, we get a vector in nullN2; in other words, we get a vector that is a linear combination of the previous basis vectors. Thus once again, all nonzero entries in these columns lie above the diagonal. Continue in this fashion to complete the proof.
EXERCISES 8.A
1 Define T 2 L.C2 / by
Find all generalized eigenvectors of T.
2 Define T 2 L.C2 / by
T.w;z/ D .z;w/:
Find the generalized eigenspaces corresponding to the distinct eigenval- ues of T.
T.w;z/ D .z;0/:
250 CHAPTER 8 Operators on Complex Vector Spaces
3 SupposeT 2L.V/isinvertible. ProvethatG.;T/DG.1;T1/for
every 2 F with ¤ 0.
4 SupposeT 2L.V/and ̨;ˇ2Fwith ̨¤ˇ.Provethat
G. ̨; T / \ G.ˇ; T / D f0g:
5 Suppose T 2 L.V /, m is a positive integer, and v 2 V is such that
Tm1v ¤ 0 but Tmv D 0. Prove that v;Tv;T2v;:::;Tm1v
is linearly independent.
6 Suppose T 2 L.C3/ is defined by T .z1; z2; z3/ D .z2; z3; 0/. Prove that T has no square root. More precisely, prove that there does not exist S 2 L.C3/ such that S2 D T.
7 Suppose N 2 L.V / is nilpotent. Prove that 0 is the only eigenvalue of N.
8 Prove or give a counterexample: The set of nilpotent operators on V is a subspace of L.V /.
9 Suppose S ; T 2 L.V / and S T is nilpotent. Prove that T S is nilpotent.
10 Suppose that T 2 L.V / is not nilpotent. Let n D dim V. Show that
V DnullTn1 ̊rangeTn1.
11 Prove or give a counterexample: If V is a complex vector space and
dim V D n and T 2 L.V /, then T n is diagonalizable.
12 Suppose N 2 L.V / and there exists a basis of V with respect to which N has an upper-triangular matrix with only 0’s on the diagonal. Prove that N is nilpotent.
13 Suppose V is an inner product space and N 2 L.V / is normal and nilpotent. Prove that N D 0.
14 Suppose V is an inner product space and N 2 L.V / is nilpotent. Prove that there exists an orthonormal basis of V with respect to which N has an upper-triangular matrix.
[If F D C, then the result above follows from Schur’s Theorem (6.38) without the hypothesis that N is nilpotent. Thus the exercise above needs to be proved only when F D R.]
SECTION 8.A Generalized Eigenvectors and Nilpotent Operators 251
15 Suppose N 2 L.V/ is such that nullNdimV1 ¤ nullNdimV . Prove
that N is nilpotent and that
dimnullNj Dj
for every integer j with 0 j dimV.
16 Suppose T 2 L.V /. Show that
V D rangeT0 rangeT1 rangeTk rangeTkC1 :
17 Suppose T 2 L.V / and m is a nonnegative integer such that rangeTm DrangeTmC1:
Prove that rangeTk D rangeTm for all k > m.
18 SupposeT2L.V/.LetnDdimV.Provethat
rangeTn D rangeTnC1 D rangeTnC2 D :
19 Suppose T 2 L.V / and m is a nonnegative integer. Prove that
nullTm D nullTmC1 if and only if rangeTm D rangeTmC1:
20 Suppose T 2 L.C5/ is such that range T 4 ¤ range T 5. Prove that T is
nilpotent.
21 Find a vector space W and T 2 L.W/ such that nullTk ̈ nullTkC1 and range T k © range T kC1 for every positive integer k.
252 CHAPTER 8 Operators on Complex Vector Spaces 8.B Decomposition of an Operator
Description of Operators on Complex Vector Spaces
We saw earlier that the domain of an operator might not decompose into eigenspaces, even on a finite-dimensional complex vector space. In this section we will see that every operator on a finite-dimensional complex vector space has enough generalized eigenvectors to provide a decomposition.
We observed earlier that if T 2 L.V /, then null T and range T are invari- ant under T [see 5.3, parts (c) and (d)]. Now we show that the null space and the range of each polynomial of T is also invariant under T.
Proof Suppose v 2 null p.T /. Then p.T /v D 0. Thus .p.T/.Tv/ D Tp.T/v D T.0/ D 0:
Hence T v 2 null p.T /. Thus null p.T / is invariant under T, as desired. Suppose v 2 range p.T /. Then there exists u 2 V such that v D p.T /u.
Thus
The following major result shows that every operator on a complex vector space can be thought of as composed of pieces, each of which is a nilpotent operator plus a scalar multiple of the identity. Actually we have already done the hard work in our discussion of the generalized eigenspaces G.; T /, so at this point the proof is easy.
8.20 The null space and range of p.T / are invariant under T
Suppose T 2 L.V/ and p 2 P.F/. Then nullp.T/ and rangep.T/ are invariant under T.
Tv D Tp.T/u D p.T/.Tu/:
Hence T v 2 range p.T /. Thus range p.T / is invariant under T, as desired.
8.21 Description of operators on complex vector spaces
SupposeV isacomplexvectorspaceandT 2L.V/. Let1;:::;m be the distinct eigenvalues of T. Then
(a) V D G.1;T/ ̊ ̊G.m;T/;
(b) each G.j ; T / is invariant under T ;
(c) each .T j I /jG.j ;T / is nilpotent.
SECTION 8.B Decomposition of an Operator 253
Proof LetnDdimV. RecallthatG.j;T/Dnull.T jI/n foreachj (by 8.11). From 8.20 [with p.z/ D .z j /n], we get (b). Obviously (c) follows from the definitions.
We will prove (a) by induction on n. To get started, note that the desired result holds if n D 1. Thus we can assume that n > 1 and that the desired result holds on all vector spaces of smaller dimension.
Because V is a complex vector space, T has an eigenvalue (see 5.21); thus m 1. Applying 8.5 to T 1I shows that
8.22 V DG.1;T/ ̊U;
where U D range.T 1I /n. Using 8.20 [with p.z/ D .z 1/n], we see that U is invariant under T. Because G.1; T / ¤ f0g, we have dim U < n. Thus we can apply our induction hypothesis to T jU.
NoneofthegeneralizedeigenvectorsofTjU correspondtotheeigenvalue 1, because all generalized eigenvectors of T corresponding to 1 are in G.1;T/. Thus each eigenvalue of TjU is in f2;:::;mg.
By our induction hypothesis, U D G.2;TjU/ ̊ ̊ G.m;TjU/. Combining this information with 8.22 will complete the proof if we can show that G.k ; T jU / D G.k ; T / for k D 2; : : : ; m.
Thus fix k 2 f2; : : : ; mg. The inclusion G.k ; T jU / G.k ; T / is clear.
To prove the inclusion in the other direction, suppose v 2 G.k ; T /. By 8.22,wecanwritevDv1Cu,wherev1 2G.1;T/andu2U. Our induction hypothesis implies that
u D v2 C C vm;
whereeachvj isinG.j;TjU/,whichisasubsetofG.j;T/. Thus
v D v1 C v2 C C vm;
Because generalized eigenvectors corresponding to distinct eigenvalues are linearlyindependent(see8.13),theequationaboveimpliesthateachvj equals 0exceptpossiblywhenj Dk. Inparticular,v1 D0andthusvDu2U. Because v 2 U, we can conclude that v 2 G.k;TjU/, completing the proof.
As we know, an operator on a complex vector space may not have enough eigenvectors to form a basis of the domain. The next result shows that on a complex vector space there are enough generalized eigenvectors to do this.
254 CHAPTER 8 Operators on Complex Vector Spaces
8.23 A basis of generalized eigenvectors
Suppose V is a complex vector space and T 2 L.V /. Then there is a basis of V consisting of generalized eigenvectors of T.
Proof Choose a basis of each G.j ; T / in 8.21. Put all these bases together to form a basis of V consisting of generalized eigenvectors of T.
Multiplicity of an Eigenvalue
If V is a complex vector space and T 2 L.V /, then the decomposition of V provided by 8.21 can be a powerful tool. The dimensions of the subspaces involved in this decomposition are sufficiently important to get a name.
8.24 Definition multiplicity
Suppose T 2 L.V /. The multiplicity of an eigenvalue of T is defined to be the dimension of the corresponding generalized eigenspace G.; T /.
In other words, the multiplicity of an eigenvalue of T equals dim null.T I /dim V .
The second bullet point above is justified by 8.11.
8.25 Example Suppose T 2 L.C3/ is defined by T.z1;z2;z3/D.6z1 C3z2 C4z3;6z2 C2z3;7z3/:
The matrix of T (with respect to the standard basis) is
06341 @062A:
007
The eigenvalues of T are 6 and 7, as follows from 5.32. You can verify that
the generalized eigenspaces of T are as follows:
G.6; T / D span.1; 0; 0/; .0; 1; 0/ and G.7; T / D span.10; 2; 1/:
Thus the eigenvalue 6 has multiplicity 2 and the eigenvalue 7 has multiplicity 1. The direct sum C3 D G.6; T / ̊G.7; T / is the decomposition promised by 8.21. A basis of C3 consisting of generalized eigenvectors of T , as promised
by 8.23, is
.1; 0; 0/; .0; 1; 0/; .10; 2; 1/:
SECTION 8.B Decomposition of an Operator 255
In Example 8.25, the sum of the multiplicities of the eigenvalues of T equals 3, which is the dimension of the domain of T. The next result shows that this always happens on a complex vector space.
Proof The desired result follows from 8.21 and the obvious formula for the dimension of a direct sum (see 3.78 or Exercise 16 in Section 2.C).
The terms algebraic multiplicity and geometric multiplicity are used in some books. In case you encounter this terminology, be aware that the algebraic multiplicity is the same as the multiplicity defined here and the geometric multiplicity is the dimension of the corresponding eigenspace. In other words, if T 2 L.V / and is an eigenvalue of T, then
algebraicmultiplicityofDdimnull.TI/dimV DdimG.;T/; geometric multiplicity of D dim null.T I / D dim E.; T /:
Note that as defined above, the algebraic multiplicity also has a geometric meaning as the dimension of a certain null space. The definition of multiplicity given here is cleaner than the traditional definition that involves determinants; 10.25 implies that these definitions are equivalent.
Block Diagonal Matrices
To interpret our results in matrix form, we make the following definition, gener- alizing the notion of a diagonal matrix.
If each matrix Aj in the definition
below is a 1-by-1 matrix, then we actually have a diagonal matrix.
8.26 Sum of the multiplicities equals dim V
Suppose V is a complex vector space and T 2 L.V /. Then the sum of the multiplicities of all the eigenvalues of T equals dim V.
Often we can understand a matrix better by thinking of it as composed of smaller matrices.
8.27 Definition block diagonal matrix
A block diagonal matrix is a square matrix of the form
01
A1 0
B@ : : : CA ;
0 Am
where A1; : : : ; Am are square matrices lying along the diagonal and all the other entries of the matrix equal 0.
256
CHAPTER 8 Operators on Complex Vector Spaces
8.28
Example The 5-by-5 matrix 0400001
B0 23!00C ADB0 02 00!C B@ 0 0 0 1 7 CA
00001
is a block diagonal matrix with
0 A1 0 1 ADB@ A2 CA;
0 A3 !!
where
Here the inner matrices in the 5-by-5 matrix above are blocked off to show how we can think of it as a block diagonal matrix.
Note that in the next result we get many more zeros in the matrix of T than are needed to make it upper triangular.
A1D4; A2D 2 3 ; A3D 1 7 : 02 01
8.29 Block diagonal matrix with upper-triangular blocks
SupposeV isacomplexvectorspaceandT 2L.V/. Let1;:::;m be the distinct eigenvalues of T, with multiplicities d1; : : : ; dm. Then there is a basis of V with respect to which T has a block diagonal matrix of the form
01
A1 0
B@ : : : CA ;
0 Am
where each Aj is a dj -by-dj upper-triangular matrix of the form
0 j 1
A j D B@ : : :
0 j
CA :
SECTION 8.B Decomposition of an Operator 257
Proof Each .T j I /jG.j ;T / is nilpotent [see 8.21(c)]. For each j , choose a basis of G.j ; T /, which is a vector space with dimension dj , such that the matrix of .T j I /jG.j ;T / with respect to this basis is as in 8.19. Thus the matrixofTjG.j;T/,whichequals.T jI/jG.j;T/ CjIjG.j;T/,with respect to this basis will look like the desired form shown above for Aj .
Putting the bases of the G.j ; T /’s together gives a basis of V [by 8.21(a)]. The matrix of T with respect to this basis has the desired form.
The 5-by-5 matrix in 8.28 is of the form promised by 8.29, with each of the blocks itself an upper-triangular matrix that is constant along the diagonal of the block. If T is an operator on a 5-dimensional vector space whose matrix is as in 8.28, then the eigenvalues of T are 4; 2; 1 (as follows from 5.32), with multiplicities 1, 2, 2.
8.30 Example Suppose T 2 L.C3/ is defined by T.z1;z2;z3/D.6z1 C3z2 C4z3;6z2 C2z3;7z3/:
The matrix of T (with respect to the standard basis) is
06341 @062A;
007
which is an upper-triangular matrix but is not of the form promised by 8.29. As we saw in Example 8.25, the eigenvalues of T are 6 and 7 and the
corresponding generalized eigenspaces are
G.6; T / D span.1; 0; 0/; .0; 1; 0/ and G.7; T / D span.10; 2; 1/:
We also saw that a basis of C3 consisting of generalized eigenvectors of T is .1; 0; 0/; .0; 1; 0/; .10; 2; 1/:
The matrix of T with respect to this basis is 06301
@060A; 007
which is a matrix of the block diagonal form promised by 8.29.
When we discuss the Jordan Form in Section 8.D, we will see that we can find a basis with respect to which an operator T has a matrix with even more 0’s than promised by 8.29. However, 8.29 and its equivalent companion 8.21 are already quite powerful. For example, in the next subsection we will use 8.21 to show that every invertible operator on a complex vector space has a square root.
258 CHAPTER 8 Operators on Complex Vector Spaces Square Roots
Recall that a square root of an operator T 2 L.V / is an operator R 2 L.V / such that R2 D T (see 7.33). Every complex number has a square root, but not every operator on a complex vector space has a square root. For example, the operator on C3 in Exercise 6 in Section 8.A has no square root. The noninvertibility of that operator is no accident, as we will soon see. We begin by showing that the identity plus any nilpotent operator has a square root.
8.31 Identity plus nilpotent has a square root
Suppose N 2 L.V / is nilpotent. Then I C N has a square root.
Proof
8.32
p
Consider the Taylor series for the function 1 C x: p1 C x D 1 C a1x C a2x2 C :
We will not find an explicit formula for the coefficients or worry about whether the infinite sum converges be- cause we will use this equation only as
Because a1 D 1=2, the formula
above shows that 1 C x=2 is a
good estimate for
is small.
p
1 C x when x
motivation.
Because N is nilpotent, N m D 0 for some positive integer m. In 8.32,
suppose we replace x with N and 1 with I. Then the infinite sum on the right side becomes a finite sum (because N j D 0 for all j m). In other words, we guess that there is a square root of I C N of the form
I Ca1N Ca2N2 CCam1Nm1:
Having made this guess, we can try to choose a1; a2; : : : ; am1 such that the
operator above has its square equal to I C N. Now .ICa1N Ca2N2 Ca3N3 CCam1Nm1/2
DI C2a1N C.2a2 Ca12/N2 C.2a3 C2a1a2/N3 C C .2am1 C terms involving a1; : : : ; am2/N m1:
We want the right side of the equation above to equal I C N. Hence choose a1 such that 2a1 D 1 (thus a1 D 1=2). Next, choose a2 such that 2a2 Ca12 D 0 (thus a2 D 1=8). Then choose a3 such that the coefficient of N 3 on the right side of the equation above equals 0 (thus a3 D 1=16). Continue in this fashion for j D 4;:::;m 1, at each step solving for aj so that the coefficient of N j on the right side of the equation above equals 0. Actually we do not care about the explicit formula for the aj ’s. We need only know that some choice of the aj ’s gives a square root of I C N.
SECTION 8.B Decomposition of an Operator 259
The previous lemma is valid on real and complex vector spaces. However, the next result holds only on complex vector spaces. For example, the operator of multiplication by 1 on the 1-dimensional real vector space R has no square root.
Proof Let 1; : : : ; m be the distinct eigenvalues of T. For each j , there ex- ists a nilpotent operator Nj 2 LG.j ; T / such that T jG.j ;T / D j I CNj [see 8.21(c)]. Because T is invertible, none of the j ’s equals 0, so we can write
8.33 Over C, invertible operators have square roots
Suppose V is a complex vector space and T 2 L.V / is invertible. Then T has a square root.
Nj TjG.j;T/Dj ICj
foreachj.ClearlyNj=j isnilpotent,andsoICNj=j hasasquareroot (by 8.31). Multiplying a square root of the complex number j by a square rootofI CNj=j,weobtainasquarerootRj ofTjG.j;T/.
A typical vector v 2 V can be written uniquely in the form v D u1 C C um;
whereeachuj isinG.j;T/(see8.21).Usingthisdecomposition,definean operator R 2 L.V / by
Rv D R1u1 C C Rmum:
You should verify that this operator R is a square root of T, completing the
proof.
By imitating the techniques in this section, you should be able to prove that if V is a complex vector space and T 2 L.V / is invertible, then T has a kth root for every positive integer k.
EXERCISES 8.B
1 Suppose V is a complex vector space, N 2 L.V /, and 0 is the only eigenvalue of N. Prove that N is nilpotent.
2 Give an example of an operator T on a finite-dimensional real vector space such that 0 is the only eigenvalue of T but T is not nilpotent.
260 CHAPTER 8 Operators on Complex Vector Spaces
3 Suppose T 2 L.V /. Suppose S 2 L.V / is invertible. Prove that T and
S1TS have the same eigenvalues with the same multiplicities.
4 Suppose V is an n-dimensional complex vector space and T is an oper- ator on V such that null T n2 ¤ null T n1. Prove that T has at most two distinct eigenvalues.
5 Suppose V is a complex vector space and T 2 L.V /. Prove that V has a basis consisting of eigenvectors of T if and only if every generalized eigenvector of T is an eigenvector of T.
[For F D C, the exercise above adds an equivalence to the list in 5.41.]
6 Define N 2 L.F5 / by
N.x1; x2; x3; x4; x5/ D .2x2; 3x3; x4; 4x5; 0/:
Find a square root of I C N.
7 Suppose V is a complex vector space. Prove that every invertible operator
on V has a cube root.
8 Suppose T 2 L.V/ and 3 and 8 are eigenvalues of T. Let n D dimV.
ProvethatV D.nullTn2/ ̊.rangeTn2/.
9 Suppose A and B are block diagonal matrices of the form 0101
A1 0 B1 0
A D B@ : : : CA ; B D B@ : : : CA ;
0Am 0Bm
where Aj has the same size as Bj for j D 1;:::;m. Show that AB is a
block diagonal matrix of the form
01
A1B1 AB D B@
0
:::
0 AmBm
CA:
10 Suppose F D C and T 2 L.V/. Prove that there exist D;N 2 L.V/ such that T D D C N, the operator D is diagonalizable, N is nilpotent, and DN D ND.
11 Suppose T 2 L.V / and 2 F. Prove that for every basis of V with respect to which T has an upper-triangular matrix, the number of times that appears on the diagonal of the matrix of T equals the multiplicity of as an eigenvalue of T.
SECTION 8.C Characteristic and Minimal Polynomials 261 8.C Characteristic and Minimal Polynomials
The Cayley–Hamilton Theorem
The next definition associates a polynomial with each operator on V if F D C. For F D R, the corresponding definition will be given in the next chapter.
8.34 Definition characteristic polynomial
Suppose V is a complex vector space and T 2 L.V/. Let 1;:::;m denote the distinct eigenvalues of T, with multiplicities d1; : : : ; dm. The polynomial
.z1/d1 .zm/dm is called the characteristic polynomial of T.
8.35 Example Suppose T 2 L.C3/ is defined as in Example 8.25. Be- cause the eigenvalues of T are 6, with multiplicity 2, and 7, with multiplicity 1, we see that the characteristic polynomial of T is .z 6/2.z 7/.
8.36 Degree and zeros of characteristic polynomial
Suppose V is a complex vector space and T 2 L.V /. Then
(a) the characteristic polynomial of T has degree dim V ;
(b) the zeros of the characteristic polynomial of T are the eigenvalues of T.
Proof Clearly part (a) follows from 8.26 and part (b) follows from the defini- tion of the characteristic polynomial.
Most texts define the characteristic polynomial using determinants (the two definitions are equivalent by 10.25). The approach taken here, which is considerably simpler, leads to the following easy proof of the Cayley– Hamilton Theorem. In the next chapter, we will see that this result also holds on real vector spaces (see 9.24).
8.37 Cayley–Hamilton Theorem
Suppose V is a complex vector space and T 2 L.V /. Let q denote the characteristic polynomial of T. Then q.T / D 0.
262 CHAPTER 8
Operators on Complex Vector Spaces
Proof Let 1;:::;m be the distinct eigenvalues of the operator T, and let d1;:::;dm be the dimensions of the corresponding generalized eigenspaces G.1;T/;:::;G.m;T/. For each j 2 f1;:::;mg, we know that .T j I /jG.j ;T / is nilpotent. Thus we have .T j I /dj jG.j ;T / D 0 (by 8.18).
EveryvectorinV isasumofvectorsinG.1;T/;:::;G.m;T/(by8.21). Thustoprovethatq.T/D0,weneedonlyshowthatq.T/jG.j;T/ D0for each j .
Thus fix j 2 f1;:::;mg. We have
q.T/D.T 1I/d1 .T mI/dm:
The operators on the right side of the equation above all commute, so we can move the factor .T j I /dj to be the last term in the expression on the right. Because.T jI/djjG.j;T/ D0,weconcludethatq.T/jG.j;T/ D0,as desired.
The Minimal Polynomial
In this subsection we introduce another important polynomial associated with each operator. We begin with the following definition.
8.39 Example The polynomial 2 C 9z2 C z7 is a monic polynomial of degree 7.
English mathematician Arthur Cayley (1821–1895) published three math papers before complet- ing his undergraduate degree in 1842. Irish mathematician William Rowan Hamilton (1805–1865) was made a professor in 1827 when he was 22 years old and still an undergraduate!
8.38 Definition monic polynomial
A monic polynomial is a polynomial whose highest-degree coefficient equals 1.
8.40 Minimal polynomial
Suppose T 2 L.V /. Then there is a unique monic polynomial p of smallest degree such that p.T / D 0.
SECTION 8.C Characteristic and Minimal Polynomials 263
Proof Let n D dim V. Then the list I;T;T2;:::;Tn2
is not linearly independent in L.V /, because the vector space L.V / has dimension n2 (see 3.61) and we have a list of length n2 C 1. Let m be the smallest positive integer such that the list
8.41 I;T;T2;:::;Tm
is linearly dependent. The Linear Dependence Lemma (2.21) implies that one of the operators in the list above is a linear combination of the previous ones. Because m was chosen to be the smallest positive integer such that the list above is linearly dependent, we conclude that T m is a linear combination of I;T;T2;:::;Tm1. Thus there exist scalars a0;a1;a2;:::;am1 2 F such that
8.42 a0I Ca1T Ca2T2 CCam1Tm1 CTm D0: Define a monic polynomial p 2 P.F/ by
p.z/Da0 Ca1zCa2z2 CCam1zm1 Czm:
Then 8.42 implies that p.T / D 0.
To prove the uniqueness part of the result, note that the choice of m implies
that no monic polynomial q 2 P.F/ with degree smaller than m can satisfy q.T/ D 0. Suppose q 2 P.F/ is a monic polynomial with degree m and q.T/D0. Then.pq/.T/D0anddeg.pq/
Because N is nilpotent, N is not injective. Thus N is not surjective (by 3.69) and hence range N is a subspace of V that has a smaller dimension than V. Thus we can apply our induction hypothesis to the restriction operator N jrange N 2 L.range N /. [We can ignore the trivial case range N D f0g, because in that case N is the 0 operator and we can choose v1;:::;vn to be anybasisofV andm1 DDmn D0togetthedesiredresult.]
By our induction hypothesis applied to NjrangeN, there exist vectors v1;:::;vn 2 rangeN and nonnegative integers m1;:::;mn such that
8.56 Nm1v1;:::;Nv1;v1;:::;Nmnvn;:::;Nvn;vn is a basis of range N and
Nm1C1v1 D D NmnC1vn D 0:
Because each vj is in range N, for each j there exists uj 2 V such that vj D Nuj . Thus NkC1uj D Nkvj for each j and each nonnegative integer k. We now claim that
8.57 Nm1C1u1;:::;Nu1;u1;:::;NmnC1un;:::;Nun;un
100e CHAPTER 8 Operators on Complex Vector Spaces
is a linearly independent list of vectors in V. To verify this claim, suppose that some linear combination of 8.57 equals 0. Applying N to that linear combination, we get a linear combination of 8.56 equal to 0. However, the list 8.56 is linearly independent, and hence all the coefficients in our original linear combination of 8.57 equal 0 except possibly the coefficients of the vectors
which equal the vectors
Nm1C1u1;:::;NmnC1un; Nm1v1;:::;Nmnvn:
Again using the linear independence of the list 8.56, we conclude that those coefficients also equal 0, completing our proof that the list 8.57 is linearly independent.
Now extend 8.57 to a basis
8.58 Nm1C1u1;:::;Nu1;u1;:::;NmnC1un;:::;Nun;un;w1;:::;wp
of V (which is possible by 2.33). Each N wj is in range N and hence is in the span of 8.56. Each vector in the list 8.56 equals N applied to some vector in thelist8.57.Thusthereexistsxj inthespanof8.57suchthatNwj DNxj. Now let
unCj Dwj xj: ThenNunCj D0.Furthermore,
Nm1C1u1;:::;Nu1;u1;:::;NmnC1un;:::;Nun;un;unC1;:::;unCp
spans V because its span contains each xj and each unCj and hence each wj (and because 8.58 spans V ).
Thus the spanning list above is a basis of V because it has the same length as the basis 8.58 (where we have used 2.42). This basis has the required form, completing the proof.
In the next definition, the diagonal of each Aj is filled with some eigenvalue j of T, the line directly above the di- agonal of Aj is filled with 1’s, and all
other entries in Aj are 0 (to understand why each j is an eigenvalue of T, see 5.32). The j ’s need not be distinct. Also, Aj may be a 1-by-1 matrix .j / containing just an eigenvalue of T.
French mathematician Camille Jor- dan (1838–1922) first published a proof of 8.60 in 1870.
010
B ::: ::: C
B : C: @ ::1A
00
Thus the desired result holds for nilpotent operators.
Now suppose T 2 L.V /. Let 1; : : : ; m be the distinct eigenvalues of T.
We have the generalized eigenspace decomposition
V D G.1;T/ ̊ ̊G.m;T/;
where each .T j I /jG.j ;T / is nilpotent (see 8.21). Thus some basis of each G.j ; T / is a Jordan basis for .T j I /jG.m ;T / (see previous paragraph). Put these bases together to get a basis of V that is a Jordan basis for T.
SECTION 8.D Jordan Form 273
8.59 Definition Jordan basis
Suppose T 2 L.V /. A basis of V is called a Jordan basis for T if with
respect to this basis T has a block diagonal matrix 01
A1 0
B@ : : : CA ;
0 Ap
where each Aj is an upper-triangular matrix of the form
01
j 1 0
B ::: ::: C Aj DB : C:
@ ::1A 0 j
8.60 Jordan Form
Suppose V is a complex vector space. If T 2 L.V /, then there is a basis of V that is a Jordan basis for T.
Proof First consider a nilpotent operator N 2 L.V / and the vectors v1;:::;vn 2 V given by 8.55. For each j, note that N sends the first vector inthelistNmjvj;:::;Nvj;vj to0andthatN sendseachvectorinthislist other than the first vector to the previous vector. In other words, 8.55 gives a basis of V with respect to which N has a block diagonal matrix, where each matrix on the diagonal has the form
01
274 CHAPTER 8 Operators on Complex Vector Spaces EXERCISES 8.D
1 Find the characteristic polynomial and the minimal polynomial of the operator N in Example 8.53.
2 Find the characteristic polynomial and the minimal polynomial of the operator N in Example 8.54.
3 Suppose N 2 L.V / is nilpotent. Prove that the minimal polynomial of N is zmC1, where m is the length of the longest consecutive string of 1’s that appears on the line directly above the diagonal in the matrix of N with respect to any Jordan basis for N.
4 SupposeT 2L.V/andv1;:::;vn isabasisofV thatisaJordanbasis for T. Describe the matrix of T with respect to the basis vn;:::;v1 obtained by reversing the order of the v’s.
5 SupposeT 2L.V/andv1;:::;vn isabasisofV thatisaJordanbasis for T. Describe the matrix of T 2 with respect to this basis.
6 SupposeN 2L.V/isnilpotentandv1;:::;vn andm1;:::;mn areas in 8.55. Prove that Nm1v1;:::;Nmnvn is a basis of nullN.
[The exercise above implies that n, which equals dim null N, depends only on N and not on the specific Jordan basis chosen for N.]
7 Suppose p; q 2 P.C/ are monic polynomials with the same zeros and q is a polynomial multiple of p. Prove that there exists T 2 L.C deg q / such that the characteristic polynomial of T is q and the minimal polynomial ofT isp.
8 Suppose V is a complex vector space and T 2 L.V /. Prove that there does not exist a direct sum decomposition of V into two proper subspaces invariant under T if and only if the minimal polynomial of T is of the form.z/dimV forsome2C.
CHAPTER
9
Euclid explaining geometry (from The School of Athens, painted by Raphael around 1510).
Operators on Real Vector Spaces
In the last chapter we learned about the structure of an operator on a finite- dimensional complex vector space. In this chapter, we will use our results about operators on complex vector spaces to learn about operators on real vector spaces.
Our assumptions for this chapter are as follows:
9.1 Notation F, V
F denotes R or C.
V denotes a finite-dimensional nonzero vector space over F.
LEARNING OBJECTIVES FOR THIS CHAPTER
complexification of a real vector space
complexification of an operator on a real vector space
operators on finite-dimensional real vector spaces have an eigenvalue or a 2-dimensional invariant subspace
characteristic polynomial and the Cayley–Hamilton Theorem description of normal operators on a real inner product space description of isometries on a real inner product space
© Springer International Publishing 2015 275 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__9
276 CHAPTER 9 Operators on Real Vector Spaces 9.A Complexification
Complexification of a Vector Space
As we will soon see, a real vector space V can be embedded, in a natural way, in a complex vector space called the complexification of V. Each operator on V can be extended to an operator on the complexification of V. Our results about operators on complex vector spaces can then be translated to information about operators on real vector spaces.
We begin by defining the complexification of a real vector space.
9.2 Definition complexification of V, VC Suppose V is a real vector space.
The complexification of V, denoted VC , equals V V. An element of VC is an ordered pair .u; v/, where u; v 2 V, but we will write this as u C iv.
Addition on VC is defined by
.u1 Civ1/C.u2 Civ2/D.u1 Cu2/Ci.v1 Cv2/
foru1;v1;u2;v2 2V.
Complex scalar multiplication on VC is defined by
.a C bi/.u C iv/ D .au bv/ C i.av C bu/
for a; b 2 R and u; v 2 V.
Motivation for the definition above of complex scalar multiplication comes from usual algebraic properties and the identity i2 D 1. If you remember the motivation, then you do not need to memorize the definition above.
WethinkofV asasubsetofVC byidentifyingu2V withuCi0. The construction of VC from V can then be thought of as generalizing the construction of Cn from Rn.
The proof of the result above is left as an exercise for the reader. Note that the additive identity of VC is 0 C i 0, which we write as just 0.
9.3 VC is a complex vector space.
Suppose V is a real vector space. Then with the definitions of addition and scalar multiplication as above, VC is a complex vector space.
SECTION 9.A Complexification 277
Probably everything that you think should work concerning complexifica- tion does work, usually with a straightforward verification, as illustrated by the next result.
9.4 Basis of V is basis of VC Suppose V is a real vector space.
(a) If v1;:::;vn is a basis of V (as a real vector space), then v1;:::;vn is a basis of VC (as a complex vector space).
(b) The dimension of VC (as a complex vector space) equals the dimen- sion of V (as a real vector space).
Proof To prove (a), suppose v1; : : : ; vn is a basis of the real vector space V. Then span.v1; : : : ; vn/ in the complex vector space VC contains all the vectors v1;:::;vn;iv1;:::;ivn. Thus v1;:::;vn spans the complex vector space VC.
To show that v1;:::;vn is linearly independent in the complex vector space VC, suppose 1;:::;n 2 C and
1v1 C C nvn D 0: Then the equation above and our definitions imply that
.Re1/v1 CC.Ren/vn D 0 and .Im1/v1 CC.Imn/vn D 0:
Because v1; : : : ; vn is linearly independent in V, the equations above imply Re1 DDRen D0andIm1 DDImn D0. Thuswehave 1 D D n D 0. Hence v1;:::;vn is linearly independent in VC, completing the proof of (a).
Clearly (b) follows immediately from (a).
Complexification of an Operator
Now we can define the complexification of an operator.
9.5 Definition complexification of T, TC
Suppose V is a real vector space and T 2 L.V /. The complexification of
T, denoted TC , is the operator TC 2 L.VC / defined by TC .u C i v/ D T u C i T v
for u; v 2 V.
278 CHAPTER 9 Operators on Real Vector Spaces
You should verify that if V is a real vector space and T 2 L.V /, then TC
is indeed in L.VC /. The key point here is that our definition of complex scalar
multiplicationcanbeusedtoshowthatT .uCiv/DT .uCiv/for CC
all u; v 2 V and all complex numbers .
The next example gives a good way to think about the complexification of
a typical operator.
9.6 Example Suppose A is an n-by-n matrix of real numbers. Define T 2 L.Rn/ by T x D Ax, where elements of Rn are thought of as n-by-1 column vectors. Identifying the complexification of Rn with Cn, we then have TCz D Az for each z 2 Cn, where again elements of Cn are thought of as n-by-1 column vectors.
In other words, if T is the operator of matrix multiplication by A on Rn, then the complexification TC is also matrix multiplication by A but now acting on the larger domain Cn.
The next result makes sense because 9.4 tells us that a basis of a real vector space is also a basis of its complexification. The proof of the next result follows immediately from the definitions.
9.7 Matrix of TC equals matrix of T
Suppose V is a real vector space with basis v1;:::;vn and T 2 L.V/. Then M.T / D M.TC /, where both matrices are with respect to the basis v1;:::;vn.
The result above and Example 9.6 provide complete insight into complexi- fication, because once a basis is chosen, every operator essentially looks like Example 9.6. Complexification of an operator could have been defined using matrices, but the approach taken here is more natural because it does not depend on the choice of a basis.
We know that every operator on a nonzero finite-dimensional complex vector space has an eigenvalue (see 5.21) and thus has a 1-dimensional in- variant subspace. We have seen an example [5.8(a)] of an operator on a nonzero finite-dimensional real vector space with no eigenvalues and thus no 1-dimensional invariant subspaces. However, we now show that an invariant subspace of dimension 1 or 2 always exists. Notice how complexification leads to a simple proof of this result.
Thus
T u C iT v D .au bv/ C .av C bu/i: Tu D aubv and Tv D avCbu:
SECTION 9.A Complexification 279
9.8 Every operator has an invariant subspace of dimension 1 or 2
Every operator on a nonzero finite-dimensional vector space has an invariant subspace of dimension 1 or 2.
Proof Every operator on a nonzero finite-dimensional complex vector space has an eigenvalue (5.21) and thus has a 1-dimensional invariant subspace.
Hence assume V is a real vector space and T 2 L.V /. The complexifica- tion TC has an eigenvalue a C bi (by 5.21), where a; b 2 R. Thus there exist u;v2V,notboth0,suchthatTC.uCiv/D.aCbi/.uCiv/. Usingthe definition of TC , the last equation can be rewritten as
LetU equalthespaninV ofthelistu;v. ThenU isasubspaceofV with dimension 1 or 2. The equations above show that U is invariant under T, completing the proof.
The Minimal Polynomial of the Complexification
Suppose V is a real vector space and T 2 L.V /. Repeated application of the definition of TC shows that
9.9 .TC /n .u C i v/ D T n u C i T n v
for every positive integer n and all u; v 2 V.
Notice that the next result implies that the minimal polynomial of TC has
real coefficients.
9.10 Minimal polynomial of TC equals minimal polynomial of T
Suppose V is a real vector space and T 2 L.V /. Then the minimal polynomial of TC equals the minimal polynomial of T.
Proof Let p 2 P.R/ denote the minimal polynomial of T. From 9.9 it is easytoseethatp.TC/Dp.T/ ,andthusp.TC/D0.
C
Suppose q 2 P.C/ is a monic polynomial such that q.TC/ D 0. Then
q.TC /.u/ D 0 for every u 2 V. Letting r denote the polynomial whose j th coefficient is the real part of the j th coefficient of q, we see that r is a monic polynomial and r.T/ D 0. Thus degq D degr degp.
The conclusions of the two previous paragraphs imply that p is the minimal polynomial of TC , as desired.
280 CHAPTER 9 Operators on Real Vector Spaces Eigenvalues of the Complexification
Now we turn to questions about the eigenvalues of the complexification of an operator. Again, everything that we expect to work indeed works easily.
We begin with a result showing that the real eigenvalues of TC are precisely the eigenvalues of T. We give two different proofs of this result. The first proof is more elementary, but the second proof is shorter and gives some useful insight.
Proof 1 First suppose is an eigenvalue of T. Then there exists v 2 V withv¤0suchthatTvDv. ThusTCvDv,whichshowsthatisan eigenvalue of TC , completing one direction of the proof.
To prove the other direction, suppose now that is an eigenvalue of TC . Thenthereexistu;v2V withuCiv¤0suchthat
TC .u C i v/ D .u C i v/:
The equation above implies that Tu D u and Tv D v. Because u ¤ 0 or
v ¤ 0, this implies that is an eigenvalue of T, completing the proof.
Proof 2 The (real) eigenvalues of T are the (real) zeros of the minimal polynomial of T (by 8.49). The real eigenvalues of TC are the real zeros of the minimal polynomial of TC (again by 8.49). These two minimal polynomials are the same (by 9.10). Thus the eigenvalues of T are precisely the real eigenvalues of TC , as desired.
Our next result shows that TC behaves symmetrically with respect to an eigenvalue and its complex conjugate N .
9.11 Real eigenvalues of TC
SupposeV isarealvectorspace,T 2L.V/,and2R. Thenisan eigenvalue of TC if and only if is an eigenvalue of T.
9.12 TC I andTC NI
Suppose V is a real vector space, T 2 L.V /, 2 C, j is a nonnegative
integer, and u; v 2 V. Then
.TC I/j.uCiv/ D 0 if and only if .TC NI/j.uiv/ D 0:
SECTION 9.A Complexification 281
Proof We will prove this result by induction on j . To get started, note that if j D 0 then (because an operator raised to the power 0 equals the identity operator) the result claims that uCiv D 0 if and only if uiv D 0, which is clearly true.
Thus assume by induction that j 1 and the desired result holds for j 1. Suppose .TC I/j .u C iv/ D 0. Then
9.13 .TC I/j1.TC I/.uCiv/D0:
Writing D a C bi, where a; b 2 R, we have
9.14 .TC I/.uCiv/D.TuauCbv/Ci.Tvavbu/ and
9.15 .TC NI/.uiv/D.TuauCbv/i.Tvavbu/: Our induction hypothesis, 9.13, and 9.14 imply that
.TC NI/j1.TuauCbv/i.Tvavbu/D0:
Now the equation above and 9.15 imply that .TC NI/j .u iv/ D 0, completing the proof in one direction.
The other direction is proved by replacing with N , replacing v with v, and then using the first direction.
An important consequence of the result above is the next result, which states that if a number is an eigenvalue of TC , then its complex conjugate is also an eigenvalue of TC .
Proof Take j D 1 in 9.12.
By definition, the eigenvalues of an operator on a real vector space are real numbers. Thus when mathematicians sometimes informally mention the complex eigenvalues of an operator on a real vector space, what they have in mind is the eigenvalues of the complexification of the operator.
Recall that the multiplicity of an eigenvalue is defined to be the dimension of the generalized eigenspace corresponding to that eigenvalue (see 8.24). The next result states that the multiplicity of an eigenvalue of a complexification equals the multiplicity of its complex conjugate.
9.16 Nonreal eigenvalues of TC come in pairs
SupposeV isarealvectorspace,T 2L.V/,and2C. Thenisan eigenvalue of TC if and only if N is an eigenvalue of TC .
282 CHAPTER 9 Operators on Real Vector Spaces
9.17 Multiplicity of equals multiplicity of N
Suppose V is a real vector space, T 2 L.V /, and 2 C is an eigenvalue of TC. Then the multiplicity of as an eigenvalue of TC equals the multiplicity of N as an eigenvalue of TC .
Proof Suppose u1 C iv1; : : : ; um C ivm is a basis of the generalized eigenspace G.;TC/, where u1;:::;um;v1;:::;vm 2 V. Then using 9.12, routine arguments show that u1 iv1; : : : ; um ivm is a basis of the gen- eralized eigenspace G.N;TC/. Thus both and N have multiplicity m as eigenvalues of TC .
9.18 Example Suppose T 2 L.R3/ is defined by T.x1;x2;x3/D.2×1;x2 x3;x2 Cx3/:0 2 0 0 1
The matrix of T with respect to the standard basis of R3 is @ 0 1 1 A: 011
As you can verify, 2 is an eigenvalue of T with multiplicity 1 and T has no other eigenvalues.
If we identify the complexification of R3 with C3, then the matrix of TC with respect to the standard basis of C3 is the matrix above. As you can verify, the eigenvalues of TC are 2, 1 C i , and 1 i , each with multiplicity 1. Thus the nonreal eigenvalues of TC come as a pair, with each the complex conjugate of the other and with the same multiplicity, as expected by 9.17.
We have seen an example [5.8(a)] of an operator on R2 with no eigenvalues. The next result shows that no such example exists on R3.
Proof Suppose V is a real vector space with odd dimension and T 2 L.V /. Because the nonreal eigenvalues of TC come in pairs with equal multiplicity (by 9.17), the sum of the multiplicities of all the nonreal eigenvalues of TC is an even number.
Because the sum of the multiplicities of all the eigenvalues of TC equals the (complex) dimension of VC (by Theorem 8.26), the conclusion of the paragraph above implies that TC has a real eigenvalue. Every real eigenvalue of TC is also an eigenvalue of T (by 9.11), giving the desired result.
9.19 Operator on odd-dimensional vector space has eigenvalue
Every operator on an odd-dimensional real vector space has an eigenvalue.
SECTION 9.A Complexification 283 Characteristic Polynomial of the Complexification
In the previous chapter we defined the characteristic polynomial of an operator on a finite-dimensional complex vector space (see 8.34). The next result is a key step toward defining the characteristic polynomial for operators on finite-dimensional real vector spaces.
Proof Suppose is a nonreal eigenvalue of TC with multiplicity m. Then N is also an eigenvalue of TC with multiplicity m (by 9.17). Thus the characteristic polynomial of TC includes factors of .z /m and .z N /m. Multiplying together these two factors, we have
.z/m.zN/m Dz2 2.Re/zCjj2m:
The polynomial above on the right has real coefficients.
The characteristic polynomial of TC is the product of terms of the form
above and terms of the form .z t /d , where t is a real eigenvalue of TC with multiplicity d . Thus the coefficients of the characteristic polynomial of TC are all real.
Now we can define the characteristic polynomial of an operator on a finite-dimensional real vector space to be the characteristic polynomial of its complexification.
9.22 Example Suppose T 2 L.R3/ is defined by T.x1;x2;x3/D.2×1;x2 x3;x2 Cx3/:
As we noted in 9.18, the eigenvalues of TC are 2, 1 C i, and 1 i, each with multiplicity 1. Thus the characteristic polynomial of the complexification TC is .z 2/z .1 C i/z .1 i/, which equals z3 4z2 C 6z 4. Hence the characteristic polynomial of T is also z3 4z2 C 6z 4.
9.20 Characteristic polynomial of TC
Suppose V is a real vector space and T 2 L.V /. Then the coefficients of the characteristic polynomial of TC are all real.
9.21 Definition Characteristic polynomial
Suppose V is a real vector space and T 2 L.V /. Then the characteristic polynomial of T is defined to be the characteristic polynomial of TC .
284 CHAPTER 9 Operators on Real Vector Spaces
In the next result, the eigenvalues of T are all real (because T is an operator
on a real vector space).
9.23 Degree and zeros of characteristic polynomial
Suppose V is a real vector space and T 2 L.V /. Then
(a) the coefficients of the characteristic polynomial of T are all real;
(b) the characteristic polynomial of T has degree dim V ;
(c) the eigenvalues of T are precisely the real zeros of the characteristic polynomial of T.
Proof Part (a) holds because of 9.20.
Part (b) follows from 8.36(a).
Part (c) holds because the real zeros of the characteristic polynomial of T
are the real eigenvalues of TC [by 8.36(a)], which are the eigenvalues of T (by 9.11).
In the previous chapter, we proved the Cayley–Hamilton Theorem (8.37) for complex vector spaces. Now we can also prove it for real vector spaces.
Proof We have already proved this result when V is a complex vector space. Thus assume that V is a real vector space.
The complex case of the Cayley–Hamilton Theorem (8.37) implies that q.TC/ D 0. Thus we also have q.T/ D 0, as desired.
9.25 Example Suppose T 2 L.R3/ is defined by T.x1;x2;x3/D.2×1;x2 x3;x2 Cx3/:
As we saw in 9.22, the characteristic polynomial of T is z3 4z2 C 6z 4. Thus the Cayley–Hamilton Theorem implies that T 3 4T 2 C 6T 4I D 0, which can also be verified by direct calculation.
We can now prove another result that we previously knew only in the complex case.
9.24 Cayley–Hamilton Theorem
Suppose T 2 L.V /. Let q denote the characteristic polynomial of T. Then q.T / D 0.
SECTION 9.A Complexification 285
9.26 Characteristic polynomial is a multiple of minimal polynomial
Suppose T 2 L.V /. Then
(a) the degree of the minimal polynomial of T is at most dim V ;
(b) the characteristic polynomial of T is a polynomial multiple of the minimal polynomial of T.
Proof Part (a) follows immediately from the Cayley–Hamilton Theorem. Part (b) follows from the Cayley–Hamilton Theorem and 8.46.
EXERCISES 9.A
1 Prove 9.3.
2 Verify that if V is a real vector space and T 2 L.V /, then TC 2 L.VC/.
3 Suppose V is a real vector space and v1;:::;vm 2 V. Prove that v1;:::;vm is linearly independent in VC if and only if v1;:::;vm is linearly independent in V.
4 Suppose V is a real vector space and v1;:::;vm 2 V. Prove that v1;:::;vm spans VC if and only if v1;:::;vm spans V.
5 Suppose that V is a real vector space and S; T 2 L.V /. Show that .SCT/C DSC CTC andthat.T/C DTC forevery2R.
6 Suppose V is a real vector space and T 2 L.V /. Prove that TC is invertible if and only if T is invertible.
7 Suppose V is a real vector space and N 2 L.V /. Prove that NC is nilpotent if and only if N is nilpotent.
8 Suppose T 2 L.R3/ and 5; 7 are eigenvalues of T. Prove that TC has no nonreal eigenvalues.
9 Prove there does not exist an operator T 2 L.R7/ such that T 2 C T C I is nilpotent.
10 Give an example of an operator T 2 L.C7/ such that T2 C T C I is nilpotent.
286 CHAPTER 9 Operators on Real Vector Spaces
11 Suppose V is a real vector space and T 2 L.V /. Suppose there exist b; c 2 R such that T 2 C bT C cI D 0. Prove that T has an eigenvalue if and only if b2 4c.
12 Suppose V is a real vector space and T 2 L.V /. Suppose there exist b; c 2 R such that b2 < 4c and T 2 C bT C cI is nilpotent. Prove that T has no eigenvalues.
13 SupposeV isarealvectorspace,T 2L.V/,andb;c2Raresuchthat b2 < 4c. Prove that null.T 2 C bT C cI /j has even dimension for every positive integer j .
14 Suppose V is a real vector space with dim V D 8. Suppose T 2 L.V / issuchthatT2 CT CI isnilpotent. Provethat.T2 CT CI/4 D0.
15 Suppose V is a real vector space and T 2 L.V / has no eigenvalues. Prove that every subspace of V invariant under T has even dimension.
16 Suppose V is a real vector space. Prove that there exists T 2 L.V / such that T 2 D I if and only if V has even dimension.
17 Suppose V is a real vector space and T 2 L.V / satisfies T 2 D I. Define complex scalar multiplication on V as follows: if a; b 2 R, then
.a C bi/v D av C bT v:
(a) Show that the complex scalar multiplication on V defined above
and the addition on V makes V into a complex vector space.
(b) Show that the dimension of V as a complex vector space is half
the dimension of V as a real vector space.
18 Suppose V is a real vector space and T 2 L.V /. Prove that the following
are equivalent:
(a) All the eigenvalues of TC are real.
(b) There exists a basis of V with respect to which T has an upper- triangular matrix.
(c) There exists a basis of V consisting of generalized eigenvectors of T.
19 Suppose V is a real vector space with dimV D n and T 2 L.V/ is such that null T n2 ¤ null T n1. Prove that T has at most two distinct eigenvalues and that TC has no nonreal eigenvalues.
SECTION 9.B Operators on Real Inner Product Spaces 287 9.B Operators on Real Inner Product Spaces
We now switch our focus to the context of inner product spaces. We will give a complete description of normal operators on real inner product spaces; a key step in the proof of this result (9.34) requires the result from the previous section that an operator on a finite-dimensional real vector space has an invariant subspace of dimension 1 or 2 (9.8).
After describing the normal operators on real inner product spaces, we will use that result to give a complete description of isometries on such spaces.
Normal Operators on Real Inner Product Spaces
The Complex Spectral Theorem (7.24) gives a complete description of normal operators on complex inner product spaces. In this subsection we will give a complete description of normal operators on real inner product spaces.
We begin with a description of the operators on 2-dimensional real inner product spaces that are normal but not self-adjoint.
9.27 Normal but not self-adjoint operators
Suppose V is a 2-dimensional real inner product space and T 2 L.V /. Then the following are equivalent:
(a) T is normal but not self-adjoint.
(b) The matrix of T with respect to every orthonormal basis of V has
the form
a b ba;
with b ¤ 0.
(c) The matrix of T with respect to some orthonormal basis of V has
the form
with b > 0.
a b ba;
Proof First suppose (a) holds, so that T is normal but not self-adjoint. Let e1; e2 be an orthonormal basis of V. Suppose
ac 9.28 MT;.e1;e2/D bd :
288 CHAPTER 9 Operators on Real Vector Spaces
ThenkTe1k2 Da2Cb2 andkTe1k2 Da2Cc2. BecauseT isnormal, kT e1k D kT e1k (see 7.20); thus these equations imply that b2 D c2. Thus c D b or c D b. But c ¤ b, because otherwise T would be self-adjoint, as can be seen from the matrix in 9.28. Hence c D b, so
ab 9.29 MT;.e1;e2/D b d :
The matrix of T is the transpose of the matrix above. Use matrix multipli- cation to compute the matrices of T T and T T (do it now). Because T is normal, these two matrices are equal. Equating the entries in the upper-right corner of the two matrices you computed, you will discover that bd D ab. Now b ¤ 0, because otherwise T would be self-adjoint, as can be seen from the matrix in 9.29. Thus d D a, completing the proof that (a) implies (b).
Now suppose (b) holds. We want to prove that (c) holds. Choose an
orthonormal basis e1; e2 of V. We know that the matrix of T with respect to
this basis has the form given by (b), with b ¤ 0. If b > 0, then (c) holds
and we have proved that (b) implies (c). If b < 0, then, as you should verify,
the matrix of T with respect to the orthonormal basis e1; e2 equals a b ,
b a
where b > 0; thus in this case we also see that (b) implies (c).
Now suppose (c) holds, so that the matrix of T with respect to some orthonormal basis has the form given in (c) with b > 0. Clearly the matrix
of T is not equal to its transpose (because b ¤ 0). Hence T is not self-adjoint. Now use matrix multiplication to verify that the matrices of T T and T T are equal. We conclude that T T D T T. Hence T is normal. Thus (c) implies (a), completing the proof.
The next result tells us that a normal operator restricted to an invariant subspace is normal. This will allow us to use induction on dim V when we prove our description of normal operators (9.34).
9.30 Normal operators and invariant subspaces
Suppose V is an inner product space, T 2 L.V / is normal, and U is a subspace of V that is invariant under T. Then
(a) U ? is invariant under T ;
(b) U is invariant under T ;
(c) .TjU/ D.T/jU;
(d) TjU 2 L.U/ and TjU? 2 L.U?/ are normal operators.
9.32
SECTION 9.B Operators on Real Inner Product Spaces 289
Proof First we will prove (a). Let e1; : : : ; em be an orthonormal basis of U. Extend to an orthonormal basis e1;:::;em;f1;:::;fn of V (this is possible by 6.35). Because U is invariant under T, each Tej is a linear combination of e1; : : : ; em. Thus the matrix of T with respect to the basis e1;:::;em;f1;:::;fn is of the form
e1 ::: em f1 ::: fn 01
e1
:BA BC
e B C M.T/D fm B CI
:1 B C :@0 CA
fn
here A denotes an m-by-m matrix, 0 denotes the n-by-m matrix of all 0’s, B denotes an m-by-n matrix, C denotes an n-by-n matrix, and for convenience the basis has been listed along the top and left sides of the matrix.
For each j 2 f1;:::;mg, kTejk2 equals the sum of the squares of the absolute values of the entries in the j th column of A (see 6.25). Hence
jD1
absolute values of the entries in the j th rows of A and B . Hence
Xm the sum of the squares of the absolute
Xm the sum of the squares of the absolute
kT ej k2 D values of the entries of A.
For each j 2 f1;:::;mg, kTejk2 equals the sum of the squares of the
9.31
kT ej k2 D values of the entries of A and B. BecauseT isnormal,kTejkDkTejkforeachj (see7.20);thus
jD1
Xm
jD1 jD1
kTejk2 D
Xm
kTejk2:
This equation, along with 9.31 and 9.32, implies that the sum of the squares of the absolute values of the entries of B equals 0. In other words, B is the matrix of all 0’s. Thus
290
CHAPTER 9
Operators on Real Vector Spaces
9.33
e1 ::: em f1 ::: fn 01
e1
:BA 0C
e B C M.T/D fm B C:
:1 B C :@0 CA
fn
This representation shows that Tfk is in the span of f1; : : : ; fn for each k. Because f1;:::;fn is a basis of U?, this implies that Tv 2 U? whenever v 2 U ?. In other words, U ? is invariant under T, completing the proof of (a).
To prove (b), note that M.T /, which is the conjugate transpose of M.T /, has a block of 0’s in the lower left corner (because M.T /, as given above, has a block of 0’s in the upper right corner). In other words, each T ej can be written as a linear combination of e1; : : : ; em. Thus U is invariant under T , completing the proof of (b).
Toprove(c),letS DTjU 2L.U/. Fixv2U. Then hSu;vi D hTu;vi
D hu; T vi
for all u 2 U. Because T v 2 U [by (b)], the equation above shows that Sv D T v. In other words, .T jU / D .T /jU, completing the proof of (c). To prove (d), note that T commutes with T (because T is normal) and that .T jU / D .T /jU [by (c)]. Thus T jU commutes with its adjoint and hence is normal. Interchanging the roles of U and U ?, which is justified by
(a),showsthatTjU? isalsonormal,completingtheproofof(d).
Our next result shows that normal operators on real inner product spaces come close to having diagonal matrices. Specifically, we get block diagonal ma- trices, with each block having size at most 2-by-2.
We cannot expect to do better than the next result, because on a real inner product space there exist normal operators that do not have a diagonal matrix with respect to any basis. For example, the operator T 2 L.R2/ defined by T .x; y/ D .y; x/ is normal (as you should verify) but has no eigenvalues; thus this particular T does not have even an upper-triangular matrix with respect to any basis of R2.
Note that if an operator T has a block diagonal matrix with respect to some basis, then the entry in each 1-by-1 block on the diagonal of this matrix is an eigenvalue of T.
SECTION 9.B Operators on Real Inner Product Spaces 291
9.34 Characterization of normal operators when F D R
Suppose V is a real inner product space and T 2 L.V /. Then the follow-
ing are equivalent:
(a) T is normal.
(b) There is an orthonormal basis of V with respect to which T has a block diagonal matrix such that each block is a 1-by-1 matrix or a 2-by-2 matrix of the form a b
with b > 0.
ba;
Proof First suppose (b) holds. With respect to the basis given by (b), the matrix of T commutes with the matrix of T (which is the transpose of the matrix of T ), as you should verify (use Exercise 9 in Section 8.B for the product of two block diagonal matrices). Thus T commutes with T , which means that T is normal, completing the proof that (b) implies (a).
Now suppose (a) holds, so T is normal. We will prove that (b) holds by induction on dim V. To get started, note that our desired result holds if dim V D 1 (trivially) or if dim V D 2 [if T is self-adjoint, use the Real Spectral Theorem (7.29); if T is not self-adjoint, use 9.27].
Now assume that dim V > 2 and that the desired result holds on vector spaces of smaller dimension. Let U be a subspace of V of dimension 1 that is invariant under T if such a subspace exists (in other words, if T has an eigenvector, let U be the span of this eigenvector). If no such subspace exists, let U be a subspace of V of dimension 2 that is invariant under T (an invariant subspace of dimension 1 or 2 always exists by 9.8).
If dim U D 1, choose a vector in U with norm 1; this vector will be an orthonormal basis of U, and of course the matrix of T jU 2 L.U / is a 1-by-1 matrix. If dim U D 2, then T jU 2 L.U / is normal (by 9.30) but not self-adjoint (otherwise T jU, and hence T, would have an eigenvector by 7.27). Thus we can choose an orthonormal basis of U with respect to which the matrixofTjU 2L.U/hastherequiredform(see9.27).
Now U? is invariant under T and TjU? is a normal operator on U? (by 9.30). Thus by our induction hypothesis, there is an orthonormal basis ofU?withrespecttowhichthematrixofTjU? hasthedesiredform.Adjoin- ing this basis to the basis of U gives an orthonormal basis of V with respect to which the matrix of T has the desired form. Thus (b) holds.
292 CHAPTER 9 Operators on Real Vector Spaces Isometries on Real Inner Product Spaces
As we will see, the next example is a key building block for isometries on real inner product spaces. Also, note that the next example shows that an isometry on R2 may have no eigenvalues.
9.35 Example Let 2 R. Then the operator on R2 of counterclockwise rotation (centered at the origin) by an angle of is an isometry, as is geomet- rically obvious. The matrix of this operator with respect to the standard basis is
cos sin sin cos :
If is not an integer multiple of , then no nonzero vector of R2 gets mapped to a scalar multiple of itself, and hence the operator has no eigenvalues.
The next result shows that every isometry on a real inner product space is composed of pieces that are rotations on 2-dimensional subspaces, pieces that equal the identity operator, and pieces that equal multiplication by 1.
9.36 Description of isometries when F D R
Suppose V is a real inner product space and S 2 L.V /. Then the following
are equivalent:
(a) S is an isometry.
(b) There is an orthonormal basis of V with respect to which S has a block diagonal matrix such that each block on the diagonal is a 1-by-1 matrix containing 1 or 1 or is a 2-by-2 matrix of the form
with 2.0;/.
cos sin sin cos ;
Proof First suppose (a) holds, so S is an isometry. Because S is normal, there is an orthonormal basis of V with respect to which S has a block diagonal matrix such that each block is a 1-by-1 matrix or a 2-by-2 matrix of the form
a b 9.37 ba;
with b > 0 (by 9.34).
Thus
Sej Daej CbejC1:
1 D kej k2 D kSej k2 D a2 C b2:
norms gives
SECTION 9.B Operators on Real Inner Product Spaces 293
If is an entry in a 1-by-1 matrix along the diagonal of the matrix of S (with respect to the basis mentioned above), then there is a basis vector ej such that Sej D ej . Because S is an isometry, this implies that jj D 1. Thus D 1 or D 1, because these are the only real numbers with absolute value 1.
Now consider a 2-by-2 matrix of the form 9.37 along the diagonal of the matrix of S . There are basis vectors ej ; ej C1 such that
The equation above, along with the condition b > 0, implies that there exists a number 2 .0;/ such that a D cos and b D sin. Thus the matrix 9.37 has the required form, completing the proof in this direction.
Conversely, now suppose (b) holds, so there is an orthonormal basis of V with respect to which the matrix of S has the form required by the theorem. Thus there is a direct sum decomposition
V DU1 ̊ ̊Um;
where each Uj is a subspace of V of dimension 1 or 2. Furthermore, any two vectors belonging to distinct U ’s are orthogonal, and each S jUj is an isometry mappingUj intoUj. Ifv2V,wecanwrite
v D u1 C C um;
where each uj is in Uj . Applying S to the equation above and then taking
kSvk2 DkSu1 CCSumk2 DkSu1k2 CCkSumk2
D ku1k2 C C kumk2 D kvk2:
Thus S is an isometry, and hence (a) holds.
294 CHAPTER 9 Operators on Real Vector Spaces EXERCISES 9.B
1 Suppose S 2 L.R3/ is an isometry. Prove that there exists a nonzero vector x 2 R3 such that S2x D x.
2 Prove that every isometry on an odd-dimensional real inner product space has 1 or 1 as an eigenvalue.
3 Suppose V is a real inner product space. Show that
hu C iv; x C iyi D hu; xi C hv; yi C hv; xi hu; yii
for u; v; x; y 2 V defines a complex inner product on VC .
4 Suppose V is a real inner product space and T 2 L.V / is self-adjoint. Show that TC is a self-adjoint operator on the inner product space VC defined by the previous exercise.
5 Use the previous exercise to give a proof of the Real Spectral Theorem (7.29) via complexification and the Complex Spectral Theorem (7.24).
6 Give an example of an operator T on an inner product space such that T has an invariant subspace whose orthogonal complement is not invariant under T.
[The exercise above shows that 9.30 can fail without the hypothesis that T is normal.]
7 Suppose T 2 L.V / and T has a block diagonal matrix 01
A1 0
B@ : : : CA
0 Am
withrespecttosomebasisofV.Forj D1;:::;m,letTj betheoperator on V whose matrix with respect to the same basis is a block diagonal matrix with blocks the same size as in the matrix above, with Aj in the j th block, and with all the other blocks on the diagonal equal to identity matrices (of the appropriate size). Prove that T D T1 Tm.
8 Suppose D is the differentiation operator on the vector space V in Exercise 21 in Section 7.A. Find an orthonormal basis of V such that the matrix of the normal operator D has the form promised by 9.34.
CHAPTER
10
British mathematician and pioneer computer scientist Ada Lovelace (1815–1852), as painted by Alfred Chalon in this 1840 portrait.
Trace and Determinant
Throughout this book our emphasis has been on linear maps and operators rather than on matrices. In this chapter we pay more attention to matrices as we define the trace and determinant of an operator and then connect these notions to the corresponding notions for matrices. The book concludes with an explanation of the important role played by determinants in the theory of volume and integration.
Our assumptions for this chapter are as follows:
10.1 Notation F, V
F denotes R or C.
V denotes a finite-dimensional nonzero vector space over F.
LEARNING OBJECTIVES FOR THIS CHAPTER change of basis and its effect upon the matrix of an operator trace of an operator and of a matrix
determinant of an operator and of a matrix
determinants and volume
© Springer International Publishing 2015 295 S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics,
DOI 10.1007/978-3-319-11080-6__10
296 CHAPTER 10 Trace and Determinant 10.A Trace
For our study of the trace and determinant, we will need to know how the matrix of an operator changes with a change of basis. Thus we begin this chapter by developing the necessary material about change of basis.
Change of Basis
With respect to every basis of V, the matrix of the identity operator I 2 L.V / is the diagonal matrix with 1’s on the diagonal and 0’s elsewhere. We also use the symbol I for the name of this matrix, as shown in the next definition.
10.2 Definition identity matrix, I
Suppose n is a positive integer. The n-by-n diagonal matrix
01
10
B@ : : : CA
01
is called the identity matrix and is denoted I.
Note that we use the symbol I to denote the identity operator (on all vector spaces) and the identity matrix (of all possible sizes). You should always be able to tell from the context which particular meaning of I is intended. For example, consider the equation M.I/ D II on the left side I denotes the identity operator, and on the right side I denotes the identity matrix.
If A is a square matrix (with entries in F, as usual) with the same size as I, then AI D IA D A, as you should verify.
10.3 Definition invertible, inverse, A1
A square matrix A is called invertible if there is a square matrix B of thesamesizesuchthatAB DBADI;wecallB theinverseofAand denote it by A1.
The same proof as used in 3.54 shows that if A is an invertible square matrix, then there is a unique matrix B suchthatAB DBADI (andthusthe notation B D A1 is justified).
Some mathematicians use the terms nonsingular, which means the same as invertible, and singular, which means the same as noninvertible.
In Section 3.C we defined the matrix of a linear map from one vector space to another with respect to two bases—one basis of the first vector space and another basis of the second vector space. When we study operators, which are linear maps from a vector space to itself, we almost always use the same basis for both vector spaces (after all, the two vector spaces in question are equal). Thus we usually refer to the matrix of an operator with respect to a basis and display at most one basis because we are using one basis in two capacities.
The next result is one of the unusual cases in which we use two different bases even though we have operators from a vector space to itself. It is just a convenient restatement of 3.43 (with U and W both equal to V ), but now we are being more careful to include the various bases explicitly in the notation. The result below holds because we defined matrix multiplication to make it true—see 3.43 and the material preceding it.
The next result deals with the matrix of the identity operator I with respect to two different bases. Note that the kth column of the matrix MI; .u1; : : : ; un/; .v1; : : : ; vn/ consists of the scalars needed to write uk as a linear combination of v1; : : : ; vn.
Proof In 10.4, replace wj with uj , and replace S and T with I, getting
I D MI;.v1;:::;vn/;.u1;:::;un/MI;.u1;:::;un/;.v1;:::;vn/:
Now interchange the roles of the u’s and v’s, getting
I D MI;.u1;:::;un/;.v1;:::;vn/MI;.v1;:::;vn/;.u1;:::;un/:
SECTION 10.A Trace 297
10.4 The matrix of the product of linear maps
Suppose u1;:::;un and v1;:::;vn and w1;:::;wn are all bases of V. SupposeS;T 2L.V/.Then
MST;.u1;:::;un/;.w1;:::;wn/ D MS;.v1;:::;vn/;.w1;:::;wn/MT;.u1;:::;un/;.v1;:::;vn/:
10.5 Matrix of the identity with respect to two bases
Suppose u1;:::;un and v1;:::;vn are bases of V. Then the matrices MI;.u1;:::;un/;.v1;:::;vn/ and MI;.v1;:::;vn/;.u1;:::;un/ are invertible, and each is the inverse of the other.
These two equations give the desired result.
298 CHAPTER 10 Trace and Determinant
10.6 Example Consider the bases .4; 2/; .5; 3/ and .1; 0/; .0; 1/ of F2. Obviously
4 5 M I; .4;2/;.5;3/ ; .1;0/;.0;1/ D 2 3 ;
because I.4; 2/ D 4.1; 0/ C 2.0; 1/ and I.5; 3/ D 5.1; 0/ C 3.0; 1/. The inverse of the matrix above is !
3 5 22;
1 2
as you should verify. Thus 10.5 implies that
35! M I; .1;0/;.0;1/ ; .4;2/;.5;3/ D 2 2 :
1 2
Now we can see how the matrix of T changes when we change bases. In the result below, we have two different bases of V. Recall that the notation MT;.u1;:::;un/ is shorthand for MT;.u1;:::;un/;.u1;:::;un/
10.7 Change of basis formula
Suppose T 2 L.V/. Let u1;:::;un and v1;:::;vn be bases of V. Let A D MI;.u1;:::;un/;.v1;:::;vn/. Then
MT;.u1;:::;un/ D A1MT;.v1;:::;vn/A:
Proof In 10.4, replace wj with uj and replace S with I, getting
10.8 MT;.u1;:::;un/ D A1MT;.u1;:::;un/;.v1;:::;vn/;
where we have used 10.5.
Again use 10.4, this time replacing wj with vj . Also replace T with I and
replace S with T, getting
MT;.u1;:::;un/;.v1;:::;vn/ D MT;.v1;:::;vn/A:
Substituting the equation above into 10.8 gives the desired result.
SECTION 10.A Trace 299 Trace: A Connection Between Operators and Matrices
Suppose T 2 L.V/ and is an eigenvalue of T. Let n D dimV. Re- call that we defined the multiplicity of to be the dimension of the gen- eralized eigenspace G.;T/ (see 8.24) and that this multiplicity equals dim null.T I /n (see 8.11). Recall also that if V is a complex vector space, then the sum of the multiplicities of all the eigenvalues of T equals n (see 8.26).
In the definition below, the sum of the eigenvalues “with each eigenvalue repeated according to its multiplicity” means that if 1; : : : ; m are the distinct eigenvalues of T (or of TC if V is a real vector space) with multiplicities d1;:::;dm, then the sum is
d11 CCdmm:
Or if you prefer to list the eigenvalues with each repeated according to its multiplicity, then the eigenvalues could be denoted 1; : : : ; n (where the index n equals dim V ) and the sum is
1 CCn:
10.9 Definition trace of an operator Suppose T 2 L.V /.
IfFDC,thenthetraceofT isthesumoftheeigenvaluesofT, with each eigenvalue repeated according to its multiplicity.
IfFDR,thenthetraceofT isthesumoftheeigenvaluesofTC, with each eigenvalue repeated according to its multiplicity.
The trace of T is denoted by trace T.
10.10 Example Suppose T 2 L.C3/ is the operator whose matrix is
03 1 21 @3 2 3A:
120
Then the eigenvalues of T are 1, 2 C 3i , and 2 3i , each with multiplicity 1, as you can verify. Computing the sum of the eigenvalues, we find that traceT D1C.2C3i/C.23i/;inotherwords,traceT D5.
300 CHAPTER 10 Trace and Determinant
The trace has a close connection with the characteristic polynomial. Sup- pose 1;:::;n are the eigenvalues of T (or of TC if V is a real vector space) with each eigenvalue repeated according to its multiplicity. Then by definition (see 8.34 and 9.21), the characteristic polynomial of T equals
.z 1/ .z n/:
Expanding the polynomial above, we can write the characteristic polynomial
of T in the form
10.11 zn .1 CCn/zn1 CC.1/n.1 n/:
The expression above immediately leads to the following result.
Most of the rest of this section is devoted to discovering how to compute trace T from the matrix of T (with respect to an arbitrary basis).
Let’s start with the easiest situation. Suppose V is a complex vector space, T 2 L.V /, and we choose a basis of V as in 8.29. With respect to that basis, T has an upper-triangular matrix with the diagonal of the matrix containing precisely the eigenvalues of T, each repeated according to its multiplicity. Thus trace T equals the sum of the diagonal entries of M.T / with respect to that basis.
The same formula works for the operator T 2 L.C3/ in Example 10.10 whose trace equals 5. In that example, the matrix is not in upper-triangular form. However, the sum of the diagonal entries of the matrix in that example equals 5, which is the trace of the operator T.
At this point you should suspect that trace T equals the sum of the diagonal entries of the matrix of T with respect to an arbitrary basis. Remarkably, this suspicion turns out to be true. To prove it, we start by making the following definition.
10.12 Trace and characteristic polynomial
Suppose T 2 L.V /. Let n D dim V. Then trace T equals the negative of the coefficient of zn1 in the characteristic polynomial of T.
10.13 Definition trace of a matrix
The trace of a square matrix A, denoted trace A, is defined to be the sum of the diagonal entries of A.
Now we have defined the trace of an operator and the trace of a square matrix, using the same word “trace” in two different contexts. This would be bad terminology unless the two concepts turn out to be essentially the same. As we will see, it is indeed true that traceT D traceMT;.v1;:::;vn/, where v1; : : : ; vn is an arbitrary basis of V. We will need the following result for the proof.
Proof Suppose 0101
Thus
An;1 ::: An;n Xn
Aj;k Bk;j : kD1
Xn Xn jD1kD1
Xn Xn kD1jD1
Xn kD1
A1;1 ::: A1;n A D B@ : : : : : :
B1;1 ::: B1;n
: : : : : : CA :
Bn;1 ::: Bn;n
CA ;
The j th term on the diagonal of AB equals
trace.AB/ D
D
Aj;kBk;j Bk;j Aj;k
kth term on the diagonal of BA
Now we can prove that the sum of the diagonal entries of the matrix of an operator is independent of the basis with respect to which the matrix is computed.
as desired.
D
D trace.BA/;
B D B@
SECTION 10.A Trace 301
10.14 Trace of AB equals trace of BA
If A and B are square matrices of the same size, then
trace.AB/ D trace.BA/:
302
CHAPTER 10 Trace and Determinant
10.15 Trace of matrix of operator does not depend on basis
LetT 2L.V/. Supposeu1;:::;un andv1;:::;vn arebasesofV. Then traceMT;.u1;:::;un/ D traceMT;.v1;:::;vn/:
Proof
Let A D MI;.u1;:::;un/;.v1;:::;vn/. Then
1
traceM T;.u1;:::;un/ D trace A M T;.v1;:::;vn/ A 1
D trace M T;.v1;:::;vn/ A A D traceMT;.v1;:::;vn/;
where the first equality comes from 10.7 and the second equality follows from 10.14. The third equality completes the proof.
The result below, which is the most important result in this section, states that the trace of an operator equals the sum of the diagonal entries of the matrix of the operator. This theorem does not specify a basis because, by the result above, the sum of the diagonal entries of the matrix of an operator is the same for every choice of basis.
Proof As noted above, trace M.T / is independent of which basis of V we choose (by 10.15). Thus to show that
trace T D trace M.T /
for every basis of V, we need only show that the equation above holds for some basis of V.
As we have already discussed, if V is a complex vector space, then choos- ing the basis as in 8.29 gives the desired result. If V is a real vector space, then applying the complex case to the complexification TC (which is used to define trace T ) gives the desired result.
If we know the matrix of an operator on a complex vector space, the result above allows us to find the sum of all the eigenvalues without finding any of the eigenvalues, as shown by the next example.
10.16 Trace of an operator equals trace of its matrix
Suppose T 2 L.V /. Then trace T D trace M.T /.
10.17 Example
Consider the operator on C5 whose matrix is
0 0 0 0 0 3 1 B 1 0 0 0 6 C B0 1 0 0 0 C: @00100A
00010
No one can find an exact formula for any of the eigenvalues of this operator. However, we do know that the sum of the eigenvalues equals 0, because the sum of the diagonal entries of the matrix above equals 0.
We can use 10.16 to give easy proofs of some useful properties about traces of operators by shifting to the language of traces of matrices, where certain properties have already been proved or are obvious. The proof of the next result is an example of this technique. The eigenvalues of S C T are not, in general, formed from adding together eigenvalues of S and eigenvalues of T. Thus the next result would be difficult to prove without using 10.16.
Proof Choose a basis of V. Then
trace.S CT/ D traceM.S CT/
D traceM.S / C M.T /
D trace M.S / C trace M.T / D trace S C trace T;
where again the first and last equalities come from 10.16; the third equality is obvious from the definition of the trace of a matrix.
The techniques we have developed have the following curious consequence. A generalization of this result to infinite- dimensional vector spaces has impor- tant consequences in modern physics, particularly in quantum theory.
SECTION 10.A Trace 303
10.18 Trace is additive
SupposeS;T 2L.V/.Thentrace.SCT/DtraceSCtraceT.
The statement of the next result does not involve traces, although the short proof uses traces. When- ever something like this happens in mathematics, we can be sure that a good definition lurks in the back- ground.
304
CHAPTER 10 Trace and Determinant
10.19 The identity is not the difference of ST and TS TheredonotexistoperatorsS;T 2L.V/suchthatST TS DI.
Suppose S; T 2 L.V /. Choose a basis of V . Then
Proof
where the first equality comes from 10.18, the second equality comes from 10.16, the third equality comes from 3.43, and the fourth equality comes from 10.14. Clearly the trace of I equals dim V, which is not 0. Because ST TS and I have different traces, they cannot be equal.
EXERCISES 10.A
1 SupposeT 2L.V/andv1;:::;vn isabasisofV. Provethatthematrix MT; .v1; : : : ; vn/ is invertible if and only if T is invertible.
2 Suppose A and B are square matrices of the same size and AB D I. Prove that BA D I.
3 Suppose T 2 L.V / has the same matrix with respect to every basis of V. Prove that T is a scalar multiple of the identity operator.
4 Supposeu1;:::;un andv1;:::;vn arebasesofV. LetT 2L.V/bethe operator such that Tvk D uk for k D 1;:::;n. Prove that
MT;.v1;:::;vn/ D MI;.u1;:::;un/;.v1;:::;vn/:
5 Suppose B is a square matrix with complex entries. Prove that there exists an invertible square matrix A with complex entries such that A1BA is an upper-triangular matrix.
6 Give an example of a real vector space V and T 2 L.V / such that trace.T 2/ < 0.
7 Suppose V is a real vector space, T 2 L.V /, and V has a basis consisting of eigenvectors of T. Prove that trace.T 2/ 0.
trace.ST TS/ D trace.ST / trace.TS/
D trace M.ST / trace M.TS/
D traceM.S /M.T / traceM.T /M.S / D 0;
SECTION 10.A Trace 305
8 SupposeV isaninnerproductspaceandv;w2V. DefineT 2L.V/by
T u D hu; viw. Find a formula for trace T.
9 SupposeP 2L.V/satisfiesP2 DP. Provethat
trace P D dim range P:
10 Suppose V is an inner product space and T 2 L.V /. Prove that
traceT D traceT:
11 Suppose V is an inner product space. Suppose T 2 L.V / is a positive
operator and trace T D 0. Prove that T D 0.
12 Suppose V is an inner product space and P; Q 2 L.V / are orthogonal
projections. Prove that trace.PQ/ 0.
13 Suppose T 2 L.C3/ is the operator whose matrix is
051 12 211 @60 40 28A:
57 68 1
Someone tells you (accurately) that 48 and 24 are eigenvalues of T. Without using a computer or writing anything down, find the third eigen- value of T.
14 SupposeT 2L.V/andc2F.Provethattrace.cT/DctraceT.
15 SupposeS;T 2L.V/.Provethattrace.ST/Dtrace.TS/.
16 Prove or give a counterexample: if S; T 2 L.V /, then trace.ST / D .trace S /.trace T /.
17 SupposeT 2L.V/issuchthattrace.ST/D0forallS 2L.V/. Prove that T D 0.
18 Suppose V is an inner product space with orthonormal basis e1; : : : ; en and T 2 L.V /. Prove that
trace.TT/ D kTe1k2 CCkTenk2:
Conclude that the right side of the equation above is independent of which orthonormal basis e1; : : : ; en is chosen for V.
306 CHAPTER 10 Trace and Determinant
19 Suppose V is an inner product space. Prove that hS; T i D trace.ST /
defines an inner product on L.V /.
20 Suppose V is a complex inner product space and T 2 L.V /. Let 1; : : : ; n be the eigenvalues of T, repeated according to multiplicity.
Suppose
01
A1;1 ::: A1;n
B@ : : CA
An;1 ::: An;n
is the matrix of T with respect to some orthonormal basis of V. Prove
that
Xn Xn kD1jD1
j1j2 CCjnj2
jAj;kj2:
21 Suppose V is an inner product space. Suppose T 2 L.V / and kTvk kTvk
for every v 2 V. Prove that T is normal.
[The exercise above fails on infinite-dimensional inner product spaces, leading to what are called hyponormal operators, which have a well- developed theory.]
10.B Determinant Determinant of an Operator
Now we are ready to define the determinant of an operator. Notice that the definition below mimics the approach we took when defining the trace, with the product of the eigenvalues replacing the sum of the eigenvalues.
SECTION 10.B Determinant 307
10.20 Definition determinant of an operator, det T Suppose T 2 L.V /.
If F D C, then the determinant of T is the product of the eigenvalues of T, with each eigenvalue repeated according to its multiplicity.
If F D R, then the determinant of T is the product of the eigenvalues of TC , with each eigenvalue repeated according to its multiplicity.
The determinant of T is denoted by det T.
If 1;:::;m are the distinct eigenvalues of T (or of TC if V is a real vector space) with multiplicities d1; : : : ; dm, then the definition above implies
detT D d1 dm: 1m
Or if you prefer to list the eigenvalues with each repeated according to its multiplicity, then the eigenvalues could be denoted 1; : : : ; n (where the index n equals dim V ) and the definition above implies
detT D 1 n:
10.21 Example Suppose T 2 L.C3/ is the operator whose matrix is
03 1 21 @3 2 3A:
120
Then the eigenvalues of T are 1, 2 C 3i , and 2 3i , each with multiplicity 1, as you can verify. Computing the product of the eigenvalues, we find that detT D1.2C3i/.23i/;inotherwords,detT D13.
308 CHAPTER 10 Trace and Determinant
The determinant has a close connection with the characteristic polynomial. Suppose 1;:::;n are the eigenvalues of T (or of TC if V is a real vector space) with each eigenvalue repeated according to its multiplicity. Then the expression for the characteristic polynomial of T given by 10.11 gives the following result.
Combining the result above and 10.12, we have the following result.
10.22 Determinant and characteristic polynomial
Suppose T 2 L.V /. Let n D dim V. Then det T equals .1/n times the constant term of the characteristic polynomial of T.
10.23 Characteristic polynomial, trace, and determinant
Suppose T 2 L.V /. Then the characteristic polynomial of T can be written as
zn .traceT/zn1 CC.1/n.detT/:
We turn now to some simple but important properties of determinants. Later we will discover how to calculate det T from the matrix of T (with respect to an arbitrary basis).
The crucial result below has an easy proof due to our definition.
Proof First suppose V is a complex vector space and T 2 L.V /. The operator T is invertible if and only if 0 is not an eigenvalue of T. Clearly this happens if and only if the product of the eigenvalues of T is not 0. Thus T is invertible if and only if det T ¤ 0, as desired.
Now consider the case where V is a real vector space and T 2 L.V /. Again, T is invertible if and only if 0 is not an eigenvalue of T, which happens if and only if 0 is not an eigenvalue of TC (because TC and T have the same real eigenvalues by 9.11). Thus again we see that T is invertible if and only if detT ¤0.
Some textbooks take the result below as the definition of the characteristic polynomial and then have our definition of the characteristic polynomial as a consequence.
10.24 Invertible is equivalent to nonzero determinant
An operator on V is invertible if and only if its determinant is nonzero.
Proof First suppose V is a complex vector space. If ; z 2 C, then is an eigenvalueofT ifandonlyifzisaneigenvalueofzIT,ascanbeseen from the equation
.T I/D.zI T/.z/I:
Raising both sides of this equation to the dim V power and then taking null spaces of both sides shows that the multiplicity of as an eigenvalue of T equals the multiplicity of z as an eigenvalue of zI T.
Let 1; : : : ; n denote the eigenvalues of T, repeated according to mul- tiplicity. Thus for z 2 C, the paragraph above shows that the eigenvalues of zI T are z 1; : : : ; z n, repeated according to multiplicity. The determinant of zI T is the product of these eigenvalues. In other words,
det.zI T/D.z1/.zn/:
The right side of the equation above is, by definition, the characteristic poly- nomial of T, completing the proof when V is a complex vector space.
Now suppose V is a real vector space. Applying the complex case to TC gives the desired result.
Determinant of a Matrix
Our next task is to discover how to compute det T from the matrix of T (with respect to an arbitrary basis). Let’s start with the easiest situation. Suppose V is a complex vector space, T 2 L.V/, and we choose a basis of V as in 8.29. With respect to that basis, T has an upper-triangular matrix with the diagonal of the matrix containing precisely the eigenvalues of T, each repeated according to its multiplicity. Thus det T equals the product of the diagonal entries of M.T / with respect to that basis.
When dealing with the trace in the previous section, we discovered that the formula (trace = sum of diagonal entries) that worked for the upper-triangular matrix given by 8.29 also worked with respect to an arbitrary basis. Could that also work for determinants? In other words, is the determinant of an operator equal to the product of the diagonal entries of the matrix of the operator with respect to an arbitrary basis?
SECTION 10.B Determinant 309
10.25 Characteristic polynomial of T equals det.zI T /
Suppose T 2 L.V /. Then the characteristic polynomial of T equals det.zI T /.
310 CHAPTER 10 Trace and Determinant
Unfortunately, the determinant is more complicated than the trace. In par- ticular, det T need not equal the product of the diagonal entries of M.T / with respect to an arbitrary basis. For example, the operator in Example 10.21 has determinant 13 but the product of the diagonal entries of its matrix equals 0.
For each square matrix A, we want to define the determinant of A, denoted det A, so that det T D det M.T / regardless of which basis is used to com- pute M.T /. We begin our search for the correct definition of the determinant of a matrix by calculating the determinants of some special operators.
10.26 Example
Suppose a1; : : : ; an 2 F. Let 01
0 an
B a1 0 C
A D B a2 0 C I @ :::::: A
an1 0
here all entries of the matrix are 0 except for the upper-right corner and along the line just below the diagonal. Suppose v1; : : : ; vn is a basis of V and T 2L.V/issuchthatMT;.v1;:::;vn/DA. FindthedeterminantofT.
Solution Firstassumeaj ¤0foreachj D1;:::;n1. Notethatthelist v1;Tv1;T2v1;:::;Tn1v1 equals v1;a1v2;a1a2v3;:::;a1 an1vn.
Thus v1;Tv1;:::;Tn1v1 is lin- early independent (because the a’s are all nonzero). Hence if p is a monic poly- nomial with degree at most n 1, then p.T /v1 ¤ 0. Thus the minimal poly- nomial of T cannot have degree less than n.
As you should verify, Tnvj D a1 anvj for each j. Thus we have Tn Da1anI.Hencezn a1an istheminimalpolynomialofT. Be- cause n D dim V and the characteristic polynomial is a polynomial multiple of the minimal polynomial (9.26), this implies that zn a1 an is also the characteristic polynomial of T.
Thus 10.22 implies that
detT D.1/n1a1an:
If some aj equals 0, then Tvj D 0 for some j, which implies that 0 is an eigenvalue of T and hence det T D 0. In other words, the formula above also holds if some aj equals 0.
Computing the minimal polynomial is often an efficient method of find- ing the characteristic polynomial, as is done in this example.
Thus in order to have det T D det M.T /, we will have to make the deter- minant of the matrix in Example 10.26 equal to .1/n1a1 an. However, we do not yet have enough evidence to make a reasonable guess about the proper definition of the determinant of an arbitrary square matrix.
To compute the determinants of a more complicated class of operators, we introduce the notion of permutation.
For example, .2; 3; 4; 5; 1/ 2 perm 5. You should think of an element of perm n as a rearrangement of the first n integers.
10.28 Example Suppose a1;:::;an 2 F and v1;:::;vn is a basis of V. Consider a permutation .p1; : : : ; pn/ 2 perm n that can be obtained as fol- lows: break .1; : : : ; n/ into lists of consecutive integers and in each list move the first term to the end of that list. For example, taking n D 9, the permutation
.2;3;1;5;6;7;4;9;8/
is obtained from .1; 2; 3/; .4; 5; 6; 7/; .8; 9/ by moving the first term of each of these lists to the end, producing .2; 3; 1/; .5; 6; 7; 4/; .9; 8/, and then putting these together to form the permutation displayed above.
Let T 2 L.V / be the operator such that Tvk Dakvpk
for k D 1;:::;n. Find detT.
Solution This generalizes Example 10.26, because if .p1; : : : ; pn/ is the permutation .2; 3; : : : ; n; 1/, then our operator T is the same as the operator T in Example 10.26.
With respect to the basis v1; : : : ; vn, the matrix of the operator T is a block
SECTION 10.B Determinant 311
10.27 Definition permutation, perm n
A permutation of .1;:::;n/ is a list .m1;:::;mn/ that contains
each of the numbers 1; : : : ; n exactly once.
Thesetofallpermutationsof.1;:::;n/isdenotedpermn.
diagonal matrix
01
A1 0
A D B@ : : : CA ;
0 AM
where each block is a square matrix of the form of the matrix in 10.26.
312 CHAPTER 10 Trace and Determinant
Correspondingly,wecanwriteV DV1 ̊ ̊VM,whereeachVj is invariant under T and each T jVj is of the form of the operator in 10.26. Because det T D .det T jV1 / .det T jVM / (because the dimensions of the generalizedeigenspacesintheVj adduptodimV),wehave
detT D.1/n11.1/nM1a1an;
whereVj hasdimensionnj (andcorrespondinglyeachAj hassizenj-by-nj)
and we have used the result from 10.26.
The number .1/n1 1 .1/nM 1 that appears above is called the sign of the corresponding permutation .p1;:::;pn/, denoted sign.p1;:::;pn/ [this is a temporary definition that we will change to an equivalent definition later, when we define the sign of an arbitrary permutation].
To put this into a form that does not depend on the particular permutation .p1;:::;pn/, let Aj;k denote the entry in row j, column k, of the matrix A from Example 10.28. Thus
0 ifj ¤pk; Aj;kD ak ifjDpk.
Example 10.28 shows that we want
10.29 detA D X sign.m1;:::;mn/Am1;1 Amn;nI
.m1 ;:::;mn /2perm n
note that each summand is 0 except the one corresponding to the permutation .p1;:::;pn/ [which is why it does not matter that the sign of the other permutations is not yet defined].
We can now guess that det A should be defined by 10.29 for an arbitrary square matrix A. This will turn out to be correct. We will now dispense with the motivation and begin the more formal approach. First we will need to define the sign of an arbitrary permutation.
10.30 Definition sign of a permutation
The sign of a permutation .m1;:::;mn/ is defined to be 1 if the number of pairs of integers .j;k/ with 1 j < k n such that j appears after k in the list .m1;:::;mn/ is even and 1 if the number of such pairs is odd.
In other words, the sign of a permutation equals 1 if the natural order has been changed an even number of times and equals 1 if the natural order has been changed an odd number of times.
10.31
Example sign of permutation
The only pair of integers .j; k/ with j < k such that j appears after k in the list .2; 1; 3; 4/ is .1; 2/. Thus the permutation .2; 1; 3; 4/ has sign 1.
In the permutation .2;3;:::;n;1/, the only pairs .j;k/ with j < k that appear with changed order are .1; 2/; .1; 3/; : : : ; .1; n/; because we have n 1 such pairs, the sign of this permutation equals .1/n1 (note that the same quantity appeared in Example 10.26).
SECTION 10.B Determinant 313
The next result shows that interchanging two entries of a permutation changes the sign of the permutation.
Proof Suppose we have two permutations, where the second permutation is obtained from the first by interchanging two entries. If the two interchanged entries were in their natural order in the first permutation, then they no longer are in the second permutation, and vice versa, for a net change (so far) of 1 or 1 (both odd numbers) in the number of pairs not in their natural order.
Consider each entry between the two
interchanged entries. If an intermediate
entry was originally in the natural order
with respect to both interchanged entries, then it is now in the natural order with respect to neither interchanged entry. Similarly, if an intermediate entry was originally in the natural order with respect to neither of the interchanged entries, then it is now in the natural order with respect to both interchanged entries. If an intermediate entry was originally in the natural order with respect to exactly one of the interchanged entries, then that is still true. Thus the net change for each intermediate entry in the number of pairs not in their natural order is 2, 2, or 0 (all even numbers).
For all the other entries, there is no change in the number of pairs not in their natural order. Thus the total net change in the number of pairs not in their natural order is an odd number. Thus the sign of the second permutation equals 1 times the sign of the first permutation.
10.32 Interchanging two entries in a permutation
Interchanging two entries in a permutation multiplies the sign of the permutation by 1.
Our motivation for the next definition comes from 10.29.
Some texts use the term signum, which means the same as sign.
100 CHAPTER 10 Trace and Determinant
10.33 Definition determinant of a matrix, det A
Suppose A is an n-by-n matrix 01
A1;1 ::: A1;n
A D B@ : : : : : : CA :
An;1 ::: An;n The determinant of A, denoted det A, is defined by
detA D X sign.m1;:::;mn/Am1;1 Amn;n: .m1 ;:::;mn /2perm n
10.34 Example determinants
If A is the 1-by-1 matrix ŒA1;1, then det A D A1;1, because perm 1
has only one element, namely .1/, which has sign 1.
Clearly perm 2 has only two elements, namely .1; 2/, which has sign 1,
and .2; 1/, which has sign 1. Thus
det
A1;1 A1;2 A2;1 A2;2
D A1;1A2;2 A2;1A1;2:
To make sure you understand this process, you should now find the for- mula for the determinant of an arbitrary 3-by-3 matrix using just the definition given above.
The set perm3 contains six ele- ments. In general, perm n contains nŠ elements. Note that nŠ rapidly grows large as n increases.
10.35
Example
Compute the determinant of an upper-triangular matrix
0A1;1 1 A D B@ : : : CA :
0 An;n
The permutation .1; 2; : : : ; n/ has sign 1 and thus contributes a term of A1;1 An;n to the sum defining det A in 10.33. Any other permutation .m1;:::;mn/ 2 permn contains at least one entry mj with mj > j, which means that Amj ;j D 0 (because A is upper triangular). Thus all the other terms in the sum in 10.33 make no contribution.
Hence det A D A1;1 An;n. In other words, the determinant of an upper- triangular matrix equals the product of the diagonal entries.
Solution
Suppose V is a complex vector space, T 2 L.V /, and we choose a basis of V as in 8.29. With respect to that basis, T has an upper-triangular matrix with the diagonal of the matrix containing precisely the eigenvalues of T, each repeated according to its multiplicity. Thus Example 10.35 tells us that det T D det M.T /, where the matrix is with respect to that basis.
Our goal is to prove that det T D det M.T / for every basis of V, not just the basis from 8.29. To do this, we will need to develop some properties of determinants of matrices. The result below is the first of the properties we will need.
SECTION 10.B Determinant 315
10.36 Interchanging two columns in a matrix
Suppose A is a square matrix and B is the matrix obtained from A by interchanging two columns. Then
detA D detB:
Proof Think of the sum defining det A in 10.33 and the corresponding sum defining detB. The same products of Aj;k’s appear in both sums, although they correspond to different permutations. The permutation corresponding to a given product of Aj;k ’s when computing det B is obtained by interchanging two entries in the corresponding permutation when computing det A, thus multiplying the sign of the permutation by 1 (see 10.32). Hence we see that detA D detB.
If T 2 L.V / and the matrix of T (with respect to some basis) has two equal columns, then T is not injective and hence det T D 0. Although this comment makes the next result plausible, it cannot be used in the proof, because we do not yet know that det T D det M.T / for every choice of basis.
Proof Suppose A is a square matrix that has two equal columns. Interchang- ing the two equal columns of A gives the original matrix A. Thus from 10.36 (with B D A), we have
10.37 Matrices with two equal columns
If A is a square matrix that has two equal columns, then det A D 0.
which implies that det A D 0.
detA D detA;
316 CHAPTER 10 Trace and Determinant Recall from 3.44 that if A is an n-by-n matrix
01
A1;1 ::: A1;n
A D B@ : : : : : : CA ;
An;1 ::: An;n
then we can think of the kth column of A as an n-by-1 matrix denoted A;k:
01
A1;k A;kDB@ : CA:
An;k
Note that Aj;k, with two subscripts, de- notes an entry of A, whereas A;k , with a dot as a placeholder and one subscript, denotes a column of A. This notation allows us to write A in the form
.A;1 ::: A;n/;
which will be useful.
The next result shows that a permutation of the columns of a matrix
changes the determinant by a factor of the sign of the permutation.
Some books define the determinant to be the function defined on the square matrices that is linear as a function of each column sepa- rately and that satisfies 10.38 and detI D 1. To prove that such a function exists and that it is unique takes a nontrivial amount of work.
10.38 Permuting the columns of a matrix
Suppose A D . A;1 ::: A;n / is an n-by-n matrix and .m1;:::;mn/ is a permutation. Then
det. A;m1 ::: A;mn / D sign.m1;:::;mn/detA:
Proof We can transform the matrix . A;m1 : : : A;mn / into A through a series of steps. In each step, we interchange two columns and hence multiply the determinant by 1 (see 10.36). The number of steps needed equals the number of steps needed to transform the permutation .m1; : : : ; mn/ into the permutation .1; : : : ; n/ by interchanging two entries in each step. The proof is completed by noting that the number of such steps is even if .m1; : : : ; mn/ has sign 1, odd if .m1; : : : ; mn/ has sign 1 (this follows from 10.32, along with the observation that the permutation .1; : : : ; n/ has sign 1).
SECTION 10.B Determinant 317 The next result about determinants will also be useful.
10.39 Determinant is a linear function of each column
Suppose k; n are positive integers with 1 k n. Fix n-by-1 matrices A;1;:::;A;n except A;k. Then the function that takes an n-by-1 column vector A;k to
det. A;1 ::: A;k ::: A;n /
is a linear map from the vector space of n-by-1 matrices with entries in F to F.
Proof The linearity follows easily from 10.33, where each term in the sum contains precisely one entry from the kth column of A.
Now we are ready to prove one of
the key properties about determinants
of square matrices. This property will
enable us to connect the determinant of
an operator with the determinant of its
matrix. Note that this proof is considerably more complicated than the proof of the corresponding result about the trace (see 10.14).
The result below was first proved in 1812 by French mathematicians Jacques Binet and Augustin-Louis Cauchy.
10.40 Determinant is multiplicative
Suppose A and B are square matrices of the same size. Then det.AB / D det.BA/ D .det A/.det B /:
Proof Write A D . A;1 : : : A;n /, where each A;k is an n-by-1 column of A. Also write
01
B1;1 ::: B1;n
BDB@ : : CAD.B;1 ::: B;n /;
Bn;1 ::: Bn;n
where each B;k is an n-by-1 column of B. Let ek denote the n-by-1 matrix that equals 1 in the kth row and 0 elsewhere. Note that Aek D A;k and Bek D B;k. Furthermore, B;k D PnmD1 Bm;kem.
First we will prove det.AB / D .det A/.det B /. As we observed ear- lier (see 3.49), the definition of matrix multiplication easily implies that ABD.AB;1 ::: AB;n /.Thus
318 CHAPTER 10 Trace and Determinant
det.AB/ D det. AB;1 ::: AB;n /
D det. A.Pnm1D1 Bm1;1em1 / : : : A.PnmnD1 Bmn;nemn / / D det. Pnm1D1 Bm1;1Aem1 : : : PnmnD1 Bmn;nAemn /
Xn
D m1 D1
Xn
Bm1;1 Bmn;n det. Aem1 ::: Aemn /;
mn D1
where the last equality comes from repeated applications of the linearity of det as a function of one column at a time (10.39). In the last sum above, all terms in which mj D mk for some j ¤ k can be ignored, because the determinant of a matrix with two equal columns is 0 (by 10.37). Thus instead of summing over all m1;:::;mn with each mj taking on values 1;:::;n, we can sum just over the permutations, where the mj ’s have distinct values. In other words,
det.AB/ D D
X
Bm1;1 Bmn;n det. Aem1 ::: Aemn / X
Bm1;1 Bmn;n sign.m1;:::;mn/ detA X
sign.m1;:::;mn/ Bm1;1 Bmn;n
.m1 ;:::;mn /2perm n D .det A/.det B /;
where the second equality comes from 10.38.
In the paragraph above, we proved that det.AB/ D .detA/.detB/. In-
terchanging the roles of A and B, we have det.BA/ D .detB/.detA/. The last equation can be rewritten as det.BA/ D .det A/.det B /, completing the proof.
Now we can prove that the determi- nant of the matrix of an operator is in- dependent of the basis with respect to which the matrix is computed.
.m1 ;:::;mn /2perm n
.m1 ;:::;mn /2perm n D .detA/
Note the similarity of the proof of the next result to the proof of the analogous result about the trace (see 10.15).
10.41 Determinant of matrix of operator does not depend on basis
LetT 2L.V/. Supposeu1;:::;un andv1;:::;vn arebasesofV. Then detMT;.u1;:::;un/ D detMT;.v1;:::;vn/:
Proof
SECTION 10.B Determinant 319 Let A D MI;.u1;:::;un/;.v1;:::;vn/. Then
1 detM T;.u1;:::;un/ D det A M T;.v1;:::;vn/ A
1 D det M T;.v1;:::;vn/ A A
D detMT;.v1;:::;vn/;
where the first equality follows from 10.7 and the second equality follows
from 10.40. The third equality completes the proof.
The result below states that the determinant of an operator equals the determinant of the matrix of the operator. This theorem does not specify a basis because, by the result above, the determinant of the matrix of an operator is the same for every choice of basis.
Proof As noted above, 10.41 implies that det M.T / is independent of which basis of V we choose. Thus to show that det T D det M.T / for every basis of V, we need only show that the result holds for some basis of V.
As we have already discussed, if V is a complex vector space, then choos- ing a basis of V as in 8.29 gives the desired result. If V is a real vector space, then applying the complex case to the complexification TC (which is used to define det T ) gives the desired result.
If we know the matrix of an operator on a complex vector space, the result above allows us to find the product of all the eigenvalues without finding any of the eigenvalues.
10.42 Determinant of an operator equals determinant of its matrix
SupposeT 2L.V/.ThendetT DdetM.T/.
10.43 Example
Suppose T is the operator on C5 whose matrix is
0 0 0 0 0 3 1 B 1 0 0 0 6 C B0 1 0 0 0 C: @00100A
00010
No one knows an exact formula for any of the eigenvalues of this operator. However, we do know that the product of the eigenvalues equals 3, because the determinant of the matrix above equals 3.
320 CHAPTER 10 Trace and Determinant
We can use 10.42 to give easy proofs of some useful properties about determinants of operators by shifting to the language of determinants of matrices, where certain properties have already been proved or are obvious. We carry out this procedure in the next result.
10.44 Determinant is multiplicative
SupposeS;T 2L.V/.Then
det.ST / D det.TS/ D .det S/.det T /:
Proof Choose a basis of V. Then
det.ST / D det M.ST /
D detM.S /M.T /
D det M.S /det M.T / D .det S /.det T /;
where the first and last equalities come from 10.42 and the third equality comes from 10.40.
In the paragraph above, we proved that det.S T / D .det S /.det T /. Inter- changing the roles of S and T, we have det.T S / D .det T /.det S /. Because multiplication of elements of F is commutative, the last equation can be rewritten as det.T S / D .det S /.det T /, completing the proof.
The Sign of the Determinant
We proved the basic results of linear algebra before introducing determinants in this final chapter. Although determinants have value as a research tool in more advanced subjects, they play little role in basic linear algebra (when the subject is done right).
Determinants do have one important application in undergraduate mathemat- ics, namely, in computing certain vol- umes and integrals. In this subsection we interpret the meaning of the sign of
the determinant on a real vector space. Then in the final subsection we will use the linear algebra we have learned to make clear the connection between determinants and these applications. Thus we will be dealing with a part of analysis that uses linear algebra.
Most applied mathematicians agree that determinants should rarely be used in serious numeric calculations.
We will begin with some purely linear algebra results that will also be useful when investigating volumes. Our setting will be inner product spaces. Recall that an isometry on an inner product space is an operator that preserves norms. The next result shows that every isometry has determinant with absolute value 1.
Proof First consider the case where V is a complex inner product space. Then all the eigenvalues of S have absolute value 1 (see the proof of 7.43). Thus the product of the eigenvalues of S, counting multiplicity, has absolute value one. In other words, jdet S j D 1, as desired.
Now suppose V is a real inner product space. We present two different proofs in this case.
Proof 1: With respect to the inner product on the complexification VC given by Exercise 3 in Section 9.B, it is easy to see that SC is an isometry on VC . Thus by the complex case that we have already done, we have jdet SC j D 1. By definition of the determinant on real vector spaces, we have det S D det SC and thus jdet S j D 1, completing the proof.
Proof 2: By 9.36, there is an orthonormal basis of V with respect to which M.S/ is a block diagonal matrix, where each block on the diagonal is a 1-by-1 matrix containing 1 or 1 or a 2-by-2 matrix of the form
cos sin sin cos ;
with 2 .0; /. Note that the determinant of each 2-by-2 matrix of the form above equals 1 (because cos2 C sin2 D 1). Thus the determinant of S, which is the product of the determinants of the blocks (see Exercise 6), is the product of 1’s and 1’s. Hence, jdet S j D 1, as desired.
The Real Spectral Theorem 7.29 states that a self-adjoint operator T on a real inner product space has an orthonormal basis consisting of eigenvectors. With respect to such a basis, the number of times each eigenvalue appears on the diagonal of M.T / is its multiplicity. Thus det T equals the product of its eigenvalues, counting multiplicity (of course, this holds for every operator, self-adjoint or not, on a complex vector space).
SECTION 10.B Determinant 321
10.45 Isometries have determinant with absolute value 1
Suppose V is an inner product space and S 2 L.V / is an isometry. Then jdetSj D 1.
322 CHAPTER 10 Trace and Determinant
RecallthatifV isaninnerproductspaceandT 2L.V/,thenTT isa positive operator and hence has a unique positive square root, denoted pT T (see 7.35 and 7.36). Because pT T is positive, all its eigenvalues are non- negative (again, see 7.35), and hence det pT T 0. These considerations play a role in next example.
10.46 Example Suppose V is a real inner product space and T 2 L.V / is invertible (and thus det T is either positive or negative). Attach a geometric meaning to the sign of det T.
Solution First we consider an isometry S 2 L.V /. By 10.45, the determinant of S equals 1 or 1. Note that
fv2V WSvDvg
is the eigenspace E.1; S/. Thinking geometrically, we could say that this is the subspace on which S reverses direction. An examination of proof 2 of 10.45 shows that detS D 1 if this subspace has even dimension and det S D 1 if this subspace has odd dimension.
Returning to our arbitrary invertible operator T 2 L.V /, by the Polar Decomposition (7.45) there is an isometry S 2 L.V / such that
We are not formally defining the phrase “reverses direction” be- cause these comments are meant only as an intuitive aid to our un- derstanding.
Now 10.44 tells us that
p TDS TT:
p
detT D.detS/.det TT/:
The remarks just before this example pointed out that det pT T 0. Thus whether det T is positive or negative depends on whether det S is positive or negative. As we saw in the paragraph above, this depends on whether the subspace on which S reverses direction has even or odd dimension.
Because T is the product of S and an operator that never reverses direction (namely, pT T ), we can reasonably say that whether det T is positive or negative depends on whether T reverses vectors an even or an odd number of times.
Volume
The next result will be a key tool in our investigation of volume. Recall that our remarks before Example 10.46 pointed out that det pT T 0.
Proof
By the Polar Decomposition (7.45), there is an isometry S 2 L.V / such
SECTION 10.B Determinant 323
10.47 jdetTj D detpTT
Suppose V is an inner product space and T 2 L.V /. Then
p jdetTj D det TT:
Another proof of this result is sug- gested in Exercise 8.
that
Thus
p TDS TT:
p jdetTj D jdetSjdet TT
p Ddet TT;
where the first equality follows from 10.44 and the second equality follows from 10.45.
Now we turn to the question of volume in Rn. Fix a positive integer n for the rest of this subsection. We will consider only the real inner product space Rn, with its standard inner product.
We would like to assign to each subset of Rn its n-dimensional volume (when n D 2, this is usually called area instead of volume). We begin with boxes, where we have a good intuitive notion of volume.
10.48 Definition box AboxinRn isasetoftheform
f.y1;:::;yn/2Rn Wxj
2 Suppose V is a real vector space with even dimension and T 2 L.V /. Suppose det T < 0. Prove that T has at least two distinct eigenvalues.
3 Suppose T 2 L.V/ and n D dimV > 2. Let 1;:::;n denote the eigenvalues of T (or of TC if V is a real vector space), repeated according to multiplicity.
(a) Find a formula for the coefficient of zn2 in the characteristic polynomial of T in terms of 1;:::;n.
(b) Find a formula for the coefficient of z in the characteristic polyno- mial of T in terms of 1;:::;n.
SECTION 10.B Determinant 331
4 SupposeT 2L.V/andc2F.Provethatdet.cT/DcdimV detT.
5 Prove or give a counterexample: if S; T 2 L.V /, then det.S C T / D det S C det T.
6 Suppose A is a block upper-triangular matrix
0 A1 1
detA D .detA1/.detAm/:
7 Suppose A is an n-by-n matrix with real entries. Let S 2 L.Cn/ denote the operator on Cn whose matrix equals A, and let T 2 L.Rn/ denote the operator on Rn whose matrix equals A. Prove that trace S D trace T and detS D detT.
8 Suppose V is an inner product space and T 2 L.V /. Prove that detT DdetT:
Use this to prove that jdet T j D det pT T , giving a different proof than was given in 10.47.
9 Suppose is an open subset of Rn and is a function from to Rn. Suppose x 2 and is differentiable at x. Prove that the operator T 2 L.Rn/ satisfying the equation in 10.56 is unique.
[This exercise shows that the notation 0.x/ is justified.]
10 Suppose T 2 L.Rn/ and x 2 Rn. Prove that T is differentiable at x and T 0.x/ D T.
11 Find a suitable hypothesis on and then prove 10.57.
12 Let a; b; c be positive numbers. Find the volume of the ellipsoid
n 3x2y2z2o .x;y;z/2R Wa2Cb2Cc2<1
by finding a set R3 whose volume you know and an operator T 2 L.R3/ such that T ./ equals the ellipsoid above.
A D B@ : : :
0 Am
CA ;
where each Aj along the diagonal is a square matrix. Prove that
Photo Credits
page 1: Pierre Louis Dumesnil; 1884 copy by Nils Forsberg/Public domain image from Wikimedia.
page 27: George M. Bergman/Archives of the Mathematisches Forschungsinstitut Oberwolfach.
page 51: Gottlieb Biermann; photo by A. Wittmann/Public domain image from Wikimedia.
page 117: Mostafa Azizi/Public domain image from Wikimedia.
page 131: Hans-Peter Postel/Public domain image from Wikimedia.
page 163: Public domain image from Wikimedia.
page 203: Public domain image from Wikimedia. Original painting is in Tate Britain.
page 224: Spiked Math.
page 241: Public domain image from Wikimedia.
page 275: Public domain image from Wikimedia. Original fresco is in the Vatican.
page 295: Public domain image from Wikimedia.
© Springer International Publishing 2015
S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics, DOI 10.1007/978-3-319-11080-6
333
Symbol Index
A1, 296 Aj; , 76 Aj;k , 70 A;k, 76 At, 109
C, 2
deg, 31
, 179
det, 307, 314 dim, 44
̊, 21
Dk, 328
E.; T /, 155
F, 4 F1, 13 Fm;n, 73 Fn, 6
FS , 14
G.; T /, 245
I , 52, 296 (), 207 Im, 118 1, 31
f , 327 L.V /, 86
L.V; W /, 52 M.T /, 70, 146
M.v/, 84 perm, 311
P.F/, 30 ,97 Pm.F/, 31 p.T /, 143 PU,195
R,2 Re, 118
0,327 ̈, 243
TQ , 9 7
p
R
T , 233 T 0, 103
T , 204
T 1, 80
T ./, 324 TC,277 Tm,143 TjU,132,137 T=U,137
U?, 193 U0, 104 hu; vi, 166
V , 16 jjvjj, 168 V 0, 101 V=U, 95 v, 15 VC, 276
v C U , 94
zN, 118 jzj, 118
© Springer International Publishing 2015
S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics, DOI 10.1007/978-3-319-11080-6
335
Index
absolute value, 118 addition
in quotient space, 96 of complex numbers, 2 of functions, 14
of linear maps, 55
of matrices, 72
of subspaces, 20
of vectors, 12
of vectors in Fn, 7
additive inverse in C, 3, 4
in Fn, 9
in vector space, 12, 15 additivity, 52
adjoint of a linear map, 204 affine subset, 94
algebraic multiplicity, 255 annihilator of a subspace, 104 Apollonius’s Identity, 179 associativity, 3, 12, 56
backward shift, 53, 59, 81, 86, 140 basis, 39
of eigenvectors, 157, 218, 221, 224, 268
of generalized eigenvectors, 254
Binet, Jacques, 317
Blake, William, 203
block diagonal matrix, 255
box in Rn, 323
Cauchy, Augustin-Louis, 171, 317 Cauchy–Schwarz Inequality, 172 Cayley, Arthur, 262 Cayley–Hamilton Theorem
on complex vector space, 261
on real vector space, 284 change of basis, 298
change of variables in integral,
328 characteristic polynomial
on complex vector space, 261
on real vector space, 283 characteristic value, 134 Christina, Queen of Sweden, 1 closed under addition, 18
closed under scalar multiplication,
18
column rank of a matrix, 111
commutativity, 3, 7, 12, 25, 56, 75, 79, 144, 212
complex conjugate, 118 complex number, 2
Complex Spectral Theorem, 218 complex vector space, 13 complexification
© Springer International Publishing 2015
S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics, DOI 10.1007/978-3-319-11080-6
337
of a vector space, 276
of an operator, 277 conjugate symmetry, 166
338 Index
conjugate transpose of a matrix, 207
coordinate, 6
cube root of an operator, 224 cubic formula, 124
degree of a polynomial, 31 derivative, 327
Descartes, René, 1 determinant
of a matrix, 314
of an operator, 307
diagonal matrix, 155
diagonal of a square matrix, 147 diagonalizable, 156 differentiable, 327
differentiation linear map, 53, 56,
59, 61, 62, 69, 72, 78,
144, 190, 248, 294 dimension, 44
of a sum of subspaces, 47 direct sum, 21, 42, 93
of a subspace and its orthogonal complement, 194
of nullTn and rangeTn, 243 distributive property, 3, 12, 16, 56,
79
Division Algorithm for
Polynomials, 121 division of complex numbers, 4
dot product, 164 double dual space, 116 dual
of a basis, 102
of a linear map, 103 of a vector space, 101
eigenspace, 155
eigenvalue of an operator, 134 eigenvector, 134
Euclid, 275
Euclidean inner product, 166
factor of a polynomial, 122 Fibonacci, 131
Fibonacci sequence, 161 field, 10
finite-dimensional vector space, 30 Flatland, 6
Fundamental Theorem of Algebra,
124
Fundamental Theorem of Linear
Maps, 63
Gauss, Carl Friedrich, 51 generalized eigenspace, 245 generalized eigenvector, 245 geometric multiplicity, 255 Gram, Jørgen, 182 Gram–Schmidt Procedure, 182 graph of a linear map, 98
Halmos, Paul, 27
Hamilton, William, 262 harmonic function, 179 Hermitian, 209
homogeneity, 52 homogeneous system of linear
equations, 65, 90 Hypatia, 241
identity map, 52, 56
identity matrix, 296
image, 62
imaginary part, 118 infinite-dimensional vector space,
31
inhomogeneous system of linear
equations, 66, 90 injective, 60
inner product, 166
inner product space, 167 integral, 327
invariant subspace, 132 inverse
of a linear map, 80
of a matrix, 296 invertible linear map, 80 invertible matrix, 296 isometry, 228, 292, 321 isomorphic vector spaces, 82 isomorphism, 82
Jordan basis, 273 Jordan Form, 273 Jordan, Camille, 272
kernel, 59
Khayyám, Omar, 117
Laplacian, 179
length of list, 5
Leonardo of Pisa, 131
linear combination, 28
Linear Dependence Lemma, 34 linear functional, 101, 187 linear map, 52
linear span, 29
linear subspace, 18
linear transformation, 52 linearly dependent, 33
linearly independent, 32
list, 5
of vectors, 28 Lovelace, Ada, 295
matrix, 70 multiplication, 75
of linear map, 70
of nilpotent operator, 249
of operator, 146
of product of linear maps, 75,
297
of T 0, 110 of T , 208 of vector, 84
minimal polynomial, 263, 279 minimizing distance, 198
monic polynomial, 262 multiplication, see product multiplicity of an eigenvalue, 254
Newton, Isaac, 203 nilpotent operator, 248, 271 nonsingular matrix, 296 norm, 164, 168
normal operator, 212, 287 null space, 59
of powers of an operator, 242 of T 0, 106
of T , 207
one-to-one, 60 onto, 62 operator, 86 orthogonal
complement, 193 operator, 229 projection, 195 vectors, 169
orthonormal basis, 181
list, 180
parallel affine subsets, 94 Parallelogram Equality, 174 permutation, 311
photo credits, 333
point, 13
polar coordinates, 329
Polar Decomposition, 233 polynomial, 30
positive operator, 225
positive semidefinite operator, 227
Index 339
340 Index
product
of complex numbers, 2
of linear maps, 55
of matrices, 75
of polynomials, 144
of scalar and linear map, 55 of scalar and vector, 12
of scalar and vector in Fn, 10 of vector spaces, 91
Pythagorean Theorem, 170
quotient map, 97
operator, 137 space, 95
range, 61
of powers of an operator, 251 of T 0, 107
of T , 207
rank of a matrix, 112
Raphael, 275
real part, 118
Real Spectral Theorem, 221 real vector space, 13
restriction operator, 137
Riesz Representation Theorem,
188 Riesz, Frigyes, 187
row rank of a matrix, 111
scalar, 4
scalar multiplication, 10, 12
in quotient space, 96 of linear maps, 55 of matrices, 73
Schmidt, Erhard, 182 School of Athens, 275 Schur’s Theorem, 186 Schur, Issai, 186 Schwarz, Hermann, 171
self-adjoint operator, 209
sign of a permutation, 312 signum, 313
singular matrix, 296
Singular Value Decomposition,
237 singular values, 236
span, 29
spans, 30
Spectral Theorem, 218, 221 spherical coordinates, 330 square root of an operator, 223,
225, 233, 259 standard basis, 39
subspace, 18
subtraction of complex numbers, 4 sum, see addition
Supreme Court, 174
surjective, 62
trace
of a matrix, 300
of an operator, 299 transpose of a matrix, 109, 207 Triangle Inequality, 173
tuple, 5
unitary operator, 229 upper-triangular matrix, 147, 256
vector, 8, 13 vector space, 12 volume, 324
zero of a polynomial, 122