1 - Matrix Review

• You can visit 01 Basic Matrix, 02 Operations with Matrices

Matrix Review and Notation

• Definition: A matrix is a rectangular array of (real/complex) numbers
  • Example: $A = \begin{bmatrix} 1 & 0 & -3 \\ 0 & -2 & 1/4 \end{bmatrix}$
• Terminology:
  • Each number is called an entry or element
  • $\mathbb{R}$ denotes the set of real numbers
  • $\mathbb{R}^{m \times n}$ denotes the set of all matrices with real entries having $m$ rows and $n$ columns
    • Called $m$-by-$n$ matrices or $m \times n$ matrices
  • Example: Matrix $A$ above is a 2-by-3 matrix ($A \in \mathbb{R}^{2 \times 3}$)

Matrix Notations

Using example: $A = \begin{bmatrix} 1 & 0 & -3 \\ 0 & -2 & 1/4 \end{bmatrix}$

• Matrix Naming:
  • Matrices are usually denoted by capital letters ($A$, $B$, $C$, etc.)
  • Corresponding lower case letter with subscript $ij$ denotes the $(i,j)$ entry
• Entry Notation:
  • $a_{13} = -3$
  • $a_{22} = -2$
  • Can also use $[A]_{ij}$ or $A(i,j)$
• Row and Column Notation:
  • $a_{i\bullet}$ denotes the $i$th row of $A$
    • Example: $a_{1\bullet} = \begin{bmatrix} 1 & 0 & -3 \end{bmatrix}$
  • $a_{\bullet j}$ denotes the $j$th column of $A$
    • Example: $a_{\bullet 2} = \begin{bmatrix} 0 \\ -2 \end{bmatrix}$

Matrix Operations

Addition

• Rule: If $A$ and $B$ are $m$-by-$n$ matrices, $C = A + B$ is a $m$-by-$n$ matrix with $c_{ij} = a_{ij} + b_{ij}$
• Example: $\begin{bmatrix} 1 & 0 & -3 \\ 0 & -2 & 1 \end{bmatrix} + \begin{bmatrix} -2 & 1 & 2 \\ 0 & 2 & 1 \end{bmatrix} = \begin{bmatrix} -1 & 1 & -1 \\ 0 & 0 & 2 \end{bmatrix}$

Subtraction

• Rule: If $A$ and $B$ are $m$-by-$n$ matrices, $C = A - B$ is a $m$-by-$n$ matrix with $c_{ij} = a_{ij} - b_{ij}$
• Example: $\begin{bmatrix} 1 & 0 & -3 \\ 0 & -2 & 1 \end{bmatrix} - \begin{bmatrix} -2 & 1 & 2 \\ 0 & 2 & 1 \end{bmatrix} = \begin{bmatrix} 3 & -1 & -5 \\ 0 & -4 & 0 \end{bmatrix}$

Scalar-Matrix Multiplication

• Rule: If $A$ is a matrix and $\alpha$ is a scalar, then $C = \alpha A$ has entries $c_{ij} = \alpha a_{ij}$
• Example: $2 \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} = \begin{bmatrix} 2 & 4 & 6 \\ 8 & 10 & 12 \end{bmatrix}$

Matrix-Matrix Multiplication

อันนี้ก็เป็นการคูณ Matrix ธรรมดาเนอะ ก็ตบ ๆ เอาได้เลย!!

• Rule: If $A$ is a $m$-by-$p$ matrix and $B$ is a $p$-by-$n$ matrix, $C = AB$ is a $m$-by-$n$ matrix with: $c_{ij} = \sum_{k=1}^{p} a_{ik}b_{kj}$
• Example 1: $\begin{bmatrix} 2 & -1 \\ 3 & 5 \end{bmatrix} \cdot \begin{bmatrix} 0 & 1 \\ 2 & -2 \end{bmatrix} = \begin{bmatrix} 2(0) + (-1)2 & 2(1) + (-1)(-2) \\ 3(0) + 5(2) & 3(1) + 5(-2) \end{bmatrix} = \begin{bmatrix} -2 & 4 \\ 10 & -7 \end{bmatrix}$
• Example 2: $\begin{bmatrix} 0 & 1 \\ 2 & -2 \end{bmatrix} \cdot \begin{bmatrix} 2 & -1 \\ 3 & 5 \end{bmatrix} = \begin{bmatrix} 0(2) + 1(3) & 0(-1) + 1(5) \\ 2(2) + (-2)(3) & 2(-1) + (-2)(5) \end{bmatrix} = \begin{bmatrix} 3 & 5 \\ -2 & -12 \end{bmatrix}$
• Important Note: $AB \neq BA$ in general

Transpose

• Definition: If $A$ is $m$-by-$n$, then its transpose (denoted $A^T$) is a $n$-by-$m$ matrix with $a_{ij}^T = a_{ji}$
• Example: $\begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}^T = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}$
• Properties:
  • $(A + B)^T = A^T + B^T$
  • $(A - B)^T = A^T - B^T$
  • $(AB)^T = B^T A^T$
  • $(ABC)^T = C^T B^T A^T$

Identity Matrix and Inverse

Identity Matrix

• Definition: An identity matrix $I_n$ is a $n$-by-$n$ matrix (square matrix) whose diagonal entries are 1 and nondiagonal entries are 0
• Example: $I_4 = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$
• Note: The subscript $n$ is usually omitted if its dimension can be inferred

Inverse

• Definition: If $A$ is $n$-by-$n$, its inverse (denoted $A^{-1}$) is a $n$-by-$n$ matrix satisfying: $AA^{-1} = I$
• Important: Some matrices do not have inverses

Linear Independence and Rank

• Visit Lecture 8 - Linear Combination and Span, Linearly Dependent Systems

Linear Independence

• Definition: Vectors $v_1, v_2, \ldots, v_m$ are said to be linearly independent if: $c_1v_1 + c_2v_2 + \cdots + c_mv_m = 0$ if and only if $c_1 = c_2 = \cdots = c_m = 0$ (scalar)

• Linear Dependence: Otherwise, they are called linearly dependent

  • If $v_1, \ldots, v_m$ are linearly dependent, then there are scalars $c_1, \ldots, c_m$, not all zeros at the same time, such that: $c_1v_1 + c_2v_2 + \cdots + c_mv_m = 0$

Rank

• Visit Lecture 9 - Basis, Dimension, Fundamental Matrix Spaces
• Definition: The rank of matrix $A$ is the maximal number of columns (or rows) of $A$ that are linearly independent

Nonsingular vs Singular Matrices

• Visit 03 Determinants
• Nonsingular Matrix: A $n$-by-$n$ matrix $A$ is said to be nonsingular if it satisfies any one of the following equivalent conditions:
  • $A$ has an inverse
  • $\det(A) \neq 0$
  • $\text{rank}(A) = n$
  • For any vector $z \neq 0$, $Az \neq 0$
• Singular Matrix: Otherwise, the matrix is singular

Orthogonal Matrices

• Definition: The matrix $Q \in \mathbb{R}^{n \times n}$ is said to be orthogonal if: $Q^TQ = QQ^T = I$
• Examples of Orthogonal Matrices:
  • $I$ (Identity matrix)
  • $\begin{bmatrix} 0.8 & 0.6 \\ -0.6 & 0.8 \end{bmatrix}$
  • $\begin{bmatrix} \cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{bmatrix}$ for any $\theta$