### Transcription

Linear Algebra in Twenty Five LecturesTom Denton and Andrew WaldronMarch 27, 2012Edited by Katrina Glaeser, Rohit Thomas & Travis Scrimshaw1

Contents1 What is Linear Algebra?122 Gaussian Elimination192.1 Notation for Linear Systems . . . . . . . . . . . . . . . . . . . 192.2 Reduced Row Echelon Form . . . . . . . . . . . . . . . . . . . 213 Elementary Row Operations274 Solution Sets for Systems of Linear Equations344.1 Non-Leading Variables . . . . . . . . . . . . . . . . . . . . . . 355 Vectors in Space, n-Vectors435.1 Directions and Magnitudes . . . . . . . . . . . . . . . . . . . . 466 Vector Spaces537 Linear Transformations588 Matrices639 Properties of Matrices729.1 Block Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 729.2 The Algebra of Square Matrices . . . . . . . . . . . . . . . . 7310 Inverse Matrix10.1 Three Properties of the Inverse10.2 Finding Inverses . . . . . . . . .10.3 Linear Systems and Inverses . .10.4 Homogeneous Systems . . . . .10.5 Bit Matrices . . . . . . . . . . .79808182838411 LU Decomposition8811.1 Using LU Decomposition to Solve Linear Systems . . . . . . . 8911.2 Finding an LU Decomposition. . . . . . . . . . . . . . . . . . 9011.3 Block LDU Decomposition . . . . . . . . . . . . . . . . . . . . 942

12 Elementary Matrices and Determinants9612.1 Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . 9712.2 Elementary Matrices . . . . . . . . . . . . . . . . . . . . . . . 10013 Elementary Matrices and Determinants II10714 Properties of the Determinant11614.1 Determinant of the Inverse . . . . . . . . . . . . . . . . . . . . 11914.2 Adjoint of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . 12014.3 Application: Volume of a Parallelepiped . . . . . . . . . . . . 12215 Subspaces and Spanning Sets12415.1 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12415.2 Building Subspaces . . . . . . . . . . . . . . . . . . . . . . . . 12616 Linear Independence13117 Basis and Dimension139n17.1 Bases in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14218 Eigenvalues and Eigenvectors14718.1 Matrix of a Linear Transformation . . . . . . . . . . . . . . . 14718.2 Invariant Directions . . . . . . . . . . . . . . . . . . . . . . . . 15119 Eigenvalues and Eigenvectors II15919.1 Eigenspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16220 Diagonalization16520.1 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . 16520.2 Change of Basis . . . . . . . . . . . . . . . . . . . . . . . . . . 16621 Orthonormal Bases17321.1 Relating Orthonormal Bases . . . . . . . . . . . . . . . . . . . 17622 Gram-Schmidt and Orthogonal Complements18122.1 Orthogonal Complements . . . . . . . . . . . . . . . . . . . . 18523 Diagonalizing Symmetric Matrices3191

24 Kernel, Range, Nullity, Rank19724.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20125 Least Squares206A Sample Midterm I Problems and Solutions211B Sample Midterm II Problems and Solutions221C Sample Final Problems and Solutions231D Points Vs. Vectors256E Abstract ConceptsE.1 Dual Spaces . .E.2 Groups . . . . .E.3 Fields . . . . .E.4 Rings . . . . . .E.5 Algebras . . . .F Sine and Cosine as an Orthonormal BasisG Movie ScriptsG.1 Introductory Video . . . . . . . . . . . . . . . . . . . . . . .G.2 What is Linear Algebra: Overview . . . . . . . . . . . . . .G.3 What is Linear Algebra: 3 3 Matrix Example . . . . . . .G.4 What is Linear Algebra: Hint . . . . . . . . . . . . . . . . .G.5 Gaussian Elimination: Augmented Matrix Notation . . . . .G.6 Gaussian Elimination: Equivalence of Augmented Matrices .G.7 Gaussian Elimination: Hints for Review Questions 4 and 5 .G.8 Gaussian Elimination: 3 3 Example . . . . . . . . . . . . .G.9 Elementary Row Operations: Example . . . . . . . . . . . .G.10 Elementary Row Operations: Worked Examples . . . . . . .G.11 Elementary Row Operations: Explanation of Proof for Theorem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .G.12 Elementary Row Operations: Hint for Review Question 3 .G.13 Solution Sets for Systems of Linear Equations: Planes . . . .G.14 Solution Sets for Systems of Linear Equations: Pictures andExplanation . . . . . . . . . . . . . . . . . . . . . . . . . . .4.258258258259260261262264. 264. 265. 267. 268. 269. 270. 271. 273. 274. 277. 279. 281. 282. 283

G.15 Solution Sets for Systems of Linear Equations: Example . . . 285G.16 Solution Sets for Systems of Linear Equations: Hint . . . . . . 287G.17 Vectors in Space, n-Vectors: Overview . . . . . . . . . . . . . 288G.18 Vectors in Space, n-Vectors: Review of Parametric Notation . 289G.19 Vectors in Space, n-Vectors: The Story of Your Life . . . . . . 291G.20 Vector Spaces: Examples of Each Rule . . . . . . . . . . . . . 292G.21 Vector Spaces: Example of a Vector Space . . . . . . . . . . . 296G.22 Vector Spaces: Hint . . . . . . . . . . . . . . . . . . . . . . . . 297G.23 Linear Transformations: A Linear and A Non-Linear Example 298G.24 Linear Transformations: Derivative and Integral of (Real) Polynomials of Degree at Most 3 . . . . . . . . . . . . . . . . . . . 300G.25 Linear Transformations: Linear Transformations Hint . . . . . 302G.26 Matrices: Adjacency Matrix Example . . . . . . . . . . . . . . 304G.27 Matrices: Do Matrices Commute? . . . . . . . . . . . . . . . . 306G.28 Matrices: Hint for Review Question 4 . . . . . . . . . . . . . . 307G.29 Matrices: Hint for Review Question 5 . . . . . . . . . . . . . . 308G.30 Properties of Matrices: Matrix Exponential Example . . . . . 309G.31 Properties of Matrices: Explanation of the Proof . . . . . . . . 310G.32 Properties of Matrices: A Closer Look at the Trace Function . 312G.33 Properties of Matrices: Matrix Exponent Hint . . . . . . . . . 313G.34 Inverse Matrix: A 2 2 Example . . . . . . . . . . . . . . . . 315G.35 Inverse Matrix: Hints for Problem 3 . . . . . . . . . . . . . . . 316G.36 Inverse Matrix: Left and Right Inverses . . . . . . . . . . . . . 317G.37 LU Decomposition: Example: How to Use LU Decomposition 319G.38 LU Decomposition: Worked Example . . . . . . . . . . . . . . 321G.39 LU Decomposition: Block LDU Explanation . . . . . . . . . . 322G.40 Elementary Matrices and Determinants: Permutations . . . . 323G.41 Elementary Matrices and Determinants: Some Ideas Explained 324G.42 Elementary Matrices and Determinants: Hints for Problem 4 . 327G.43 Elementary Matrices and Determinants II: Elementary Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328G.44 Elementary Matrices and Determinants II: Determinants andInverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330G.45 Elementary Matrices and Determinants II: Product of Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332G.46 Properties of the Determinant: Practice taking Determinants 333G.47 Properties of the Determinant: The Adjoint Matrix . . . . . . 335G.48 Properties of the Determinant: Hint for Problem 3 . . . . . . 3385

G.49 Subspaces and Spanning Sets: Worked Example . . . . . . .G.50 Subspaces and Spanning Sets: Hint for Problem 2 . . . . . .G.51 Subspaces and Spanning Sets: Hint . . . . . . . . . . . . . .G.52 Linear Independence: Worked Example . . . . . . . . . . . .G.53 Linear Independence: Proof of Theorem 16.1 . . . . . . . . .G.54 Linear Independence: Hint for Problem 1 . . . . . . . . . . .G.55 Basis and Dimension: Proof of Theorem . . . . . . . . . . .G.56 Basis and Dimension: Worked Example . . . . . . . . . . . .G.57 Basis and Dimension: Hint for Problem 2 . . . . . . . . . . .G.58 Eigenvalues and Eigenvectors: Worked Example . . . . . . .G.59 Eigenvalues and Eigenvectors: 2 2 Example . . . . . . . .G.60 Eigenvalues and Eigenvectors: Jordan Cells . . . . . . . . . .G.61 Eigenvalues and Eigenvectors II: Eigenvalues . . . . . . . . .G.62 Eigenvalues and Eigenvectors II: Eigenspaces . . . . . . . . .G.63 Eigenvalues and Eigenvectors II: Hint . . . . . . . . . . . . .G.64 Diagonalization: Derivative Is Not Diagonalizable . . . . . .G.65 Diagonalization: Change of Basis Example . . . . . . . . . .G.66 Diagonalization: Diagionalizing Example . . . . . . . . . . .G.67 Orthonormal Bases: Sine and Cosine Form All OrthonormalBases for R2 . . . . . . . . . . . . . . . . . . . . . . . . . . .G.68 Orthonormal Bases: Hint for Question 2, Lecture 21 . . . . .G.69 Orthonormal Bases: Hint . . . . . . . . . . . . . . . . . . .G.70 Gram-Schmidt and Orthogonal Complements: 4 4 GramSchmidt Example . . . . . . . . . . . . . . . . . . . . . . . .G.71 Gram-Schmidt and Orthogonal Complements: Overview . .G.72 Gram-Schmidt and Orthogonal Complements: QR Decomposition Example . . . . . . . . . . . . . . . . . . . . . . . . .G.73 Gram-Schmidt and Orthogonal Complements: Hint for Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .G.74 Diagonalizing Symmetric Matrices: 3 3 Example . . . . . .G.75 Diagonalizing Symmetric Matrices: Hints for Problem 1 . . .G.76 Kernel, Range, Nullity, Rank: Invertibility Conditions . . . .G.77 Kernel, Range, Nullity, Rank: Hint for 1 . . . . . . . . . .G.78 Least Squares: Hint for Problem 1 . . . . . . . . . . . . . .G.79 Least Squares: Hint for Problem 2 . . . . . . . . . . . . . .H Student 358360361362363368. 370. 371. 372. 374. 376. 378.3803823843853863873883896

IOther Resources390J List of Symbols392Index3937

PrefaceThese linear algebra lecture notes are designed to be presented as twenty five,fifty minute lectures suitable for sophomores likely to use the material forapplications but still requiring a solid foundation in this fundamental branchof mathematics. The main idea of the course is to emphasize the conceptsof vector spaces and linear transformations as mathematical structures thatcan be used to model the world around us. Once “persuaded” of this truth,students learn explicit skills such as Gaussian elimination and diagonalizationin order that vectors and linear transformations become calculational tools,rather than abstract mathematics.In practical terms, the course aims to produce students who can performcomputations with large linear systems while at the same time understandthe concepts behind these techniques. Often-times when a problem can be reduced to one of linear algebra it is “solved”. These notes do not devote muchspace to applications (there are already a plethora of textbooks with titlesinvolving some permutation of the words “linear”, “algebra” and “applications”). Instead, they attempt to explain the fundamental concepts carefullyenough that students will realize for their own selves when the particularapplication they encounter in future studies is ripe for a solution via linearalgebra.There are relatively few worked examples or illustrations in these notes,this material is instead covered by a series of “linear algebra how-to videos”. The “scripts”They can be viewed by clicking on the take one iconfor these movies are found at the end of the notes if students prefer to readthis material in a traditional format and can be easily reached via the scripticon. Watch an introductory video below:Introductory VideoThe notes are designed to be used in conjunction with a set of onlinehomework exercises which help the students read the lecture notes and learnbasic linear algebra skills. Interspersed among the lecture notes are linksto simple online problems that test whether students are actively readingthe notes. In addition there are two sets of sample midterm problems withsolutions as well as a sample final exam. There are also a set of ten online assignments which are usually collected weekly. The first assignment8

is designed to ensure familiarity with some basic mathematic notions (sets,functions, logical quantifiers and basic methods of proof). The remainingnine assignments are devoted to the usual matrix and vector gymnasticsexpected from any sophomore linear algebra class. These exercises are allavailable WaldronWinter-2012/Webwork is an open source, online homework system which originated atthe University of Rochester. It can efficiently check whether a student hasanswered an explicit, typically computation-based, problem correctly. Theproblem sets chosen to accompany these notes could contribute roughly 20%of a student’s grade, and ensure that basic computational skills are mastered.Most students rapidly realize that it is best to print out the Webwork assignments and solve them on paper before entering the answers online. Thosewho do not tend to fare poorly on midterm examinations. We have foundthat there tend to be relatively few questions from students in office hoursabout the Webwork assignments. Instead, by assigning 20% of the gradeto written assignments drawn from problems chosen randomly from the review exercises at the end of each lecture, the student’s focus was primarilyon understanding ideas. They range from simple tests of understanding ofthe material in the lectures to more difficult problems, all of them requirethinking, rather than blind application of mathematical “recipes”. Officehour questions reflected this and offered an excellent chance to give studentstips how to present written answers in a way that would convince the persongrading their work that they deserved full credit!Each lecture concludes with references to the comprehensive online textbooks of Jim Hefferon and Rob //linear.ups.edu/index.htmland the notes are also hyperlinked to Wikipedia where students can rapidlyaccess further details and background material for many of the concepts.Videos of linear algebra lectu