Lernzettel: Linear Algebra Essentials

📋 Course Outline

1. Vector Operations
2. Matrix Types
3. Matrix Operations
4. Determinant Calculation
5. Linear Systems
6. Eigenvalues and Eigenvectors
7. Vector Space Properties
8. Subspaces and Basis
9. Linear Transformations

📖 1. Vector Operations

🔑 Key Concepts & Definitions

• Vector: An ordered set of numbers representing a point or direction in space, denoted as ( \mathbf{v} = (v_1, v_2, ..., v_n) ).
• Vector Addition: The operation of adding two vectors component-wise: ( \mathbf{u} + \mathbf{v} = (u_1 + v_1, u_2 + v_2, ..., u_n + v_n) ).
• Scalar Multiplication: Multiplying a vector by a scalar ( c ), scaling each component: ( c\mathbf{v} = (cv_1, cv_2, ..., cv_n) ).
• Dot Product: A scalar obtained from two vectors, calculated as ( \mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + ... + u_n v_n ), measuring similarity or projection.
• Magnitude (Norm): The length of a vector, given by ( |\mathbf{v}| = \sqrt{v_1^2 + v_2^2 + ... + v_n^2} ).

📝 Essential Points

• Vectors are fundamental in representing quantities with both magnitude and direction.
• Vector operations obey properties like commutativity of addition, distributivity, and associativity.
• The dot product is used to find angles between vectors and to project one vector onto another.
• Scalar multiplication changes the length of a vector without altering its direction (unless scalar is negative).
• Understanding vector operations is crucial for manipulating and analyzing geometric and algebraic problems.

💡 Key Takeaway

Vector operations form the foundation of linear algebra, enabling the representation and manipulation of multi-dimensional data, essential for applications across science, engineering, and computer science.

📖 2. Matrix Types

🔑 Key Concepts & Definitions

• Square Matrix: A matrix with the same number of rows and columns (n×n). It has special properties such as a determinant and inverse if non-singular.
• Diagonal Matrix: A square matrix where all off-diagonal elements are zero. The diagonal elements can be any values.
• Identity Matrix: A diagonal matrix with ones on the diagonal and zeros elsewhere, denoted as ( \mathbf{I} ). It acts as the multiplicative identity in matrix multiplication.
• Symmetric Matrix: A matrix equal to its transpose (( \mathbf{A} = \mathbf{A}^T )), meaning ( a_{ij} = a_{ji} ).
• Orthogonal Matrix: A square matrix whose transpose is its inverse (( \mathbf{A}^T = \mathbf{A}^{-1} )), satisfying ( \mathbf{A} \mathbf{A}^T = \mathbf{I} ).

📝 Essential Points

• Different matrix types have unique properties that simplify calculations (e.g., diagonal matrices make multiplication and determinant calculation straightforward).
• The identity matrix serves as the neutral element in matrix multiplication, similar to 1 in scalar multiplication.
• Symmetric and orthogonal matrices are important in transformations, with orthogonal matrices preserving lengths and angles.
• Recognizing matrix types helps in solving systems efficiently and understanding matrix behavior in linear transformations.

💡 Key Takeaway

Understanding the various types of matrices and their properties is fundamental in linear algebra, as it enables efficient computation and deeper insight into matrix operations and transformations.

📖 3. Matrix Operations

🔑 Key Concepts & Definitions

• Matrix Addition: The sum of two matrices of the same size, obtained by adding corresponding elements.
• Matrix Multiplication: Combining two matrices ( \mathbf{A} ) (size ( m \times n )) and ( \mathbf{B} ) (size ( n \times p )) to produce a matrix ( \mathbf{C} ) (size ( m \times p )), where each element ( c_{ij} ) is the dot product of the ( i )-th row of ( \mathbf{A} ) and the ( j )-th column of ( \mathbf{B} ).
• Transpose of a Matrix: Flipping a matrix over its diagonal, converting rows into columns and vice versa, denoted as ( \mathbf{A}^T ).
• Inverse Matrix: For a square matrix ( \mathbf{A} ), the matrix ( \mathbf{A}^{-1} ) such that ( \mathbf{A} \mathbf{A}^{-1} = \mathbf{I} ), exists only if ( \mathbf{A} ) is invertible (determinant ≠ 0).
• Scalar Multiplication: Multiplying every element of a matrix by a scalar ( c ).

📝 Essential Points

• Matrix addition and scalar multiplication are element-wise operations, straightforward and commutative.
• Matrix multiplication is associative but not commutative; ( \mathbf{A} \mathbf{B} \neq \mathbf{B} \mathbf{A} ) in general.
• The identity matrix ( \mathbf{I} ) acts as the multiplicative identity: ( \mathbf{A} \mathbf{I} = \mathbf{A} ).
• The inverse matrix exists only for square matrices with non-zero determinants; it is used to solve systems of equations via ( \mathbf{x} = \mathbf{A}^{-1} \mathbf{b} ).
• Transpose is useful in defining symmetric and orthogonal matrices, and in simplifying matrix expressions.
• Matrix operations underpin many advanced topics such as eigenvalues, linear transformations, and matrix decompositions.

💡 Key Takeaway

Matrix operations form the foundation of linear algebra, enabling the manipulation and solution of systems, transformations, and more complex matrix decompositions essential across scientific and engineering disciplines.

📖 4. Determinant Calculation

🔑 Key Concepts & Definitions

• Determinant: A scalar value associated with a square matrix that indicates whether the matrix is invertible and provides volume scaling factor for linear transformations.
• Invertibility: A matrix is invertible if and only if its determinant is non-zero (( \det(\mathbf{A}) \neq 0 )); otherwise, it is singular.
• Cofactor: The signed minor of an element in a matrix, used in determinant expansion; for element ( a_{ij} ), cofactor ( C_{ij} = (-1)^{i+j} M_{ij} ), where ( M_{ij} ) is the minor.
• Minor: The determinant of the submatrix formed by removing the ( i )-th row and ( j )-th column from the original matrix.
• Determinant Expansion (Laplace Expansion): Method of calculating determinants by expanding along a row or column using cofactors.
• Properties:
  • ( \det(\mathbf{A} \mathbf{B}) = \det(\mathbf{A}) \det(\mathbf{B}) )
  • Swapping two rows or columns multiplies the determinant by -1.
  • The determinant of a triangular matrix (upper or lower) is the product of its diagonal entries.

📝 Essential Points

• Determinants are only defined for square matrices.
• For a ( 2 \times 2 ) matrix ( \begin{pmatrix} a & b \ c & d \end{pmatrix} ), the determinant is ( ad - bc ).
• For larger matrices, determinants are computed via cofactor expansion or row reduction techniques.
• Row operations:
  • Swapping rows changes the sign of the determinant.
  • Multiplying a row by a scalar multiplies the determinant by that scalar.
  • Adding a multiple of one row to another does not change the determinant.
• Determinants are used to check matrix invertibility, compute eigenvalues, and in calculus for change of variables.

💡 Key Takeaway

The determinant is a fundamental scalar that reveals whether a matrix is invertible and encodes geometric and algebraic properties of linear transformations; mastering its calculation methods is essential for advanced linear algebra applications.

📖 5. Linear Systems

🔑 Key Concepts & Definitions

• Linear System: A collection of one or more linear equations involving the same set of variables. It can be written in matrix form as ( \mathbf{A} \mathbf{x} = \mathbf{b} ).
• Solution of a System: An assignment of values to variables that satisfies all equations simultaneously. Solutions can be unique, infinite, or none.
• Consistent System: A system with at least one solution.
• Inconsistent System: A system with no solutions; equations contradict each other.
• Homogeneous System: A system where ( \mathbf{b} = \mathbf{0} ); always has at least the trivial solution ( \mathbf{x} = \mathbf{0} ).
• Gaussian Elimination: A systematic method for solving linear systems by transforming the augmented matrix into row echelon form and then back-substituting.

📝 Essential Points

• The solution set of a linear system depends on the rank of the coefficient matrix ( \mathbf{A} ) and the augmented matrix ( [\mathbf{A} | \mathbf{b}] ).
• Unique solution exists if ( \text{rank}(\mathbf{A}) = \text{rank}([\mathbf{A} | \mathbf{b}]) = n ), where ( n ) is the number of variables.
• Infinite solutions occur if ( \text{rank}(\mathbf{A}) = \text{rank}([\mathbf{A} | \mathbf{b}]) < n ).
• No solution if ( \text{rank}(\mathbf{A}) \neq \text{rank}([\mathbf{A} | \mathbf{b}]) ).
• Gaussian elimination involves row operations: swapping, scaling, and adding multiples of rows to simplify the system.

💡 Key Takeaway

Linear systems can be efficiently solved using matrix methods like Gaussian elimination, and their solutions depend on the rank and consistency of the system. Understanding these relationships is crucial for analyzing whether solutions exist and how many solutions there are.

📖 6. Eigenvalues and Eigenvectors

🔑 Key Concepts & Definitions

• Eigenvalue (( \lambda )): A scalar associated with a square matrix ( \mathbf{A} ) such that there exists a non-zero vector ( \mathbf{v} ) satisfying ( \mathbf{A} \mathbf{v} = \lambda \mathbf{v} ). It represents the factor by which the eigenvector is scaled during the linear transformation.

• Eigenvector (( \mathbf{v} )): A non-zero vector that, when multiplied by matrix ( \mathbf{A} ), results in a scalar multiple of itself, i.e., ( \mathbf{A} \mathbf{v} = \lambda \mathbf{v} ). It indicates the direction preserved by the transformation.

• Characteristic Polynomial: The polynomial obtained from ( \det(\mathbf{A} - \lambda \mathbf{I}) = 0 ), used to find eigenvalues. It is a polynomial in ( \lambda ) whose roots are the eigenvalues.

• Eigenvalue Equation: The fundamental relation ( \mathbf{A} \mathbf{v} = \lambda \mathbf{v} ), defining the eigenvalues and eigenvectors of ( \mathbf{A} ).

• Diagonalization: The process of expressing a matrix ( \mathbf{A} ) as ( \mathbf{P} \mathbf{D} \mathbf{P}^{-1} ), where ( \mathbf{D} ) is a diagonal matrix of eigenvalues and ( \mathbf{P} ) contains the eigenvectors. It simplifies matrix powers and functions.

📝 Essential Points

• To find eigenvalues, solve ( \det(\mathbf{A} - \lambda \mathbf{I}) = 0 ) (characteristic equation).
• Eigenvalues can be real or complex; their multiplicity affects the matrix's diagonalizability.
• Eigenvectors corresponding to distinct eigenvalues are linearly independent.
• Eigenvalues provide insight into the matrix's properties, such as stability and spectral radius.
• Diagonalization is possible if the matrix has enough linearly independent eigenvectors (i.e., is diagonalizable).
• Eigenvalues are used in principal component analysis (PCA), stability analysis, quantum mechanics, and vibration analysis.

💡 Key Takeaway

Eigenvalues and eigenvectors reveal intrinsic characteristics of a matrix, allowing for simplified analysis of linear transformations, especially through diagonalization, which is essential in many applications across science and engineering.

📖 7. Vector Space Properties

🔑 Key Concepts & Definitions

• Vector Space: A set ( V ) of vectors over a field ( \mathbb{F} ) (such as ( \mathbb{R} )) where vector addition and scalar multiplication satisfy eight axioms (closure, associativity, commutativity, identity elements, inverse elements, distributivity, and compatibility).

• Closure: For all ( \mathbf{u}, \mathbf{v} \in V ), both ( \mathbf{u} + \mathbf{v} ) and ( c\mathbf{u} ) (for any scalar ( c \in \mathbb{F} )) are also in ( V ).

• Zero Vector: The additive identity ( \mathbf{0} \in V ) such that ( \mathbf{v} + \mathbf{0} = \mathbf{v} ) for all ( \mathbf{v} \in V ).

• Linear Independence: A set of vectors ( {\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k} ) is linearly independent if the only solution to ( c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \dots + c_k \mathbf{v}_k = \mathbf{0} ) is ( c_1 = c_2 = \dots = c_k = 0 ).

• Subspace: A subset ( W \subseteq V ) that is itself a vector space under the same addition and scalar multiplication, meaning it contains the zero vector, is closed under addition, and scalar multiplication.

📝 Essential Points

• Axioms of Vector Spaces: Must satisfy eight axioms, including closure under addition and scalar multiplication, existence of additive identity and inverses, distributivity, and compatibility of scalar multiplication.

• Subspace Criteria: To verify a subset ( W ) is a subspace, check if it contains the zero vector, and if it is closed under vector addition and scalar multiplication.

• Basis and Dimension: A basis is a minimal set of linearly independent vectors that span the entire vector space. The number of vectors in a basis defines the space's dimension.

• Properties of Subspaces: Any intersection of subspaces is a subspace; the span of any set of vectors is a subspace.

• Linear Dependence: If a set of vectors is linearly dependent, at least one vector can be written as a linear combination of others, indicating redundancy.

💡 Key Takeaway

A vector space is a structured set of vectors closed under addition and scalar multiplication, with subspaces being its smaller, self-contained vector spaces; understanding these properties is fundamental to analyzing the structure and dimension of vector spaces.

📖 8. Subspaces and Basis

🔑 Key Concepts & Definitions

• Subspace: A subset ( W ) of a vector space ( V ) that is itself a vector space under the same addition and scalar multiplication. It must contain the zero vector, be closed under addition, and be closed under scalar multiplication.

• Basis: A set of linearly independent vectors in a vector space ( V ) that spans ( V ). Every vector in ( V ) can be expressed uniquely as a linear combination of basis vectors.

• Dimension: The number of vectors in any basis of a vector space ( V ). It measures the "size" or degrees of freedom of the space.

• Linear Independence: A set of vectors ( { \mathbf{v}_1, \mathbf{v}_2, ..., \mathbf{v}_k } ) is linearly independent if the only solution to ( c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + ... + c_k \mathbf{v}_k = \mathbf{0} ) is ( c_1 = c_2 = ... = c_k = 0 ).

• Spanning Set: A set of vectors ( S ) such that every vector in ( V ) can be written as a linear combination of vectors in ( S ).

📝 Essential Points

• A subspace must include the zero vector and be closed under vector addition and scalar multiplication.
• The basis provides a minimal set of vectors needed to generate the entire space.
• The dimension of a vector space is equal to the number of vectors in any of its bases.
• Any two bases of a finite-dimensional vector space have the same number of vectors (the dimension).
• To determine if a set is a basis, verify linear independence and whether it spans the space.
• Subspaces include the zero vector space, lines through the origin, planes through the origin, and the entire space.

💡 Key Takeaway

A subspace is a smaller vector space within a larger one, and a basis is a minimal, independent set of vectors that fully describes the space; together, they define the structure and dimension of vector spaces.

📖 9. Linear Transformations

🔑 Key Concepts & Definitions

• Linear Transformation: A function ( T: V \to W ) between vector spaces ( V ) and ( W ) that preserves vector addition and scalar multiplication: [ T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v}), \quad T(c\mathbf{v}) = cT(\mathbf{v}) ]
• Matrix Representation: Every linear transformation ( T ) from ( \mathbb{R}^n ) to ( \mathbb{R}^m ) can be represented by an ( m \times n ) matrix ( \mathbf{A} ), such that: [ T(\mathbf{x}) = \mathbf{A} \mathbf{x} ]
• Kernel (Null Space): The set of all vectors ( \mathbf{v} ) in ( V ) such that ( T(\mathbf{v}) = \mathbf{0} ). It measures the vectors mapped to zero.
• Image (Range): The set of all vectors in ( W ) that are images of vectors in ( V ) under ( T ). It describes the output space of the transformation.
• Invertible Transformation: A linear transformation ( T ) that has an inverse ( T^{-1} ), satisfying: [ T^{-1}(T(\mathbf{v})) = \mathbf{v} ] and is represented by an invertible matrix.

📝 Essential Points

• Every linear transformation can be expressed as multiplication by a matrix, making matrix algebra fundamental to understanding transformations.
• The properties of a linear transformation (such as invertibility, rank, and nullity) are directly related to the properties of its matrix representation.
• The Rank-Nullity Theorem links the dimension of the domain, the null space, and the image: [ \text{dim}(V) = \text{rank}(T) + \text{nullity}(T) ]
• Linear transformations are used extensively in computer graphics (scaling, rotation, translation), data science (dimensionality reduction), and physics (coordinate changes).

💡 Key Takeaway

Linear transformations are functions that preserve vector operations and can be fully described by matrices, enabling their properties and effects to be analyzed through matrix algebra. Understanding their kernel, image, and invertibility is crucial for applications across science and engineering.

📊 Synthesis Tables

⚠️ Common Pitfalls & Confusions

1. Confusing vector addition with scalar multiplication; forgetting properties like commutativity.
2. Mixing up matrix multiplication order; ( \mathbf{A} \mathbf{B} \neq \mathbf{B} \mathbf{A} ) in general.
3. Assuming all matrices are invertible; determinant zero means no inverse.
4. Miscalculating determinants using incorrect minors or cofactors.
5. Overlooking that the transpose of a symmetric matrix equals itself.
6. Confusing eigenvalues and eigenvectors; eigenvalues are scalars, eigenvectors are vectors.
7. Assuming matrix operations are always commutative.
8. Forgetting that the identity matrix acts as the multiplicative identity.
9. Misapplying row operations that change the determinant sign or value.
10. Overlooking the difference between homogeneous and non-homogeneous systems.
11. Ignoring that the solution set of a linear system can be infinite or empty.

✅ Exam Checklist

• Define a vector and explain vector addition, scalar multiplication, and dot product.
• State properties of vector operations and their geometric meanings.
• Describe different matrix types: square, diagonal, identity, symmetric, orthogonal.
• Explain matrix addition, multiplication, transpose, and inverse.
• Calculate determinants for 2×2 and 3×3 matrices; understand cofactor expansion.
• State the conditions for matrix invertibility based on the determinant.
• Solve linear systems using Gaussian elimination and interpret solutions.
• Define eigenvalues and eigenvectors; describe their significance.
• Explain properties of vector spaces, subspaces, and basis.
• Describe linear transformations and their matrix representations.
• Understand the properties of subspaces: closure under addition and scalar multiplication.
• Recognize the importance of eigenvalues/eigenvectors in diagonalization.