Matrix Multiplication

</aside>

Definition

If here are $A$ ($m \times n$) matrix and $B$ ($n\times p$) matrix, then their product $AB$ makes $C$ ($m \times p$ ) matrix.

Conditions for Matrix multiplication

The number of columns in $A$ must equal the number of rows in $B$.

In other words, the dimension of row vectors in $A$ must match the dimension of column vectors in $B$ for the dot product to be possible. </aside>

Formula

$$ A = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32} \end{bmatrix}

$$ B = \begin{bmatrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\

\end{bmatrix}

$$ C = AB = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32} \end{bmatrix} \times \begin{bmatrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\ \end{bmatrix} $$

Each element $c_{ij}$ of $C$ is calculated as the dot product of the $i$th row in $A$ and the $j$th column in $B$.

$c_{11} = a_{11} \cdot b_{11} + a_{12} \cdot b_{21}$
$c_{12} = a_{11} \cdot b_{12} + a_{12} \cdot b_{22}$
$c_{13} = a_{11} \cdot b_{13} + a_{12} \cdot b_{23}$
$c_{21} = a_{21} \cdot b_{11} + a_{22} \cdot b_{21}$
$c_{22} = a_{21} \cdot b_{12} + a_{22} \cdot b_{22}$
$c_{23} = a_{21} \cdot b_{13} + a_{22} \cdot b_{23}$
$c_{31} = a_{31} \cdot b_{11} + a_{32} \cdot b_{21}$
$c_{32} = a_{31} \cdot b_{12} + a_{32} \cdot b_{22}$
$c_{33} = a_{31} \cdot b_{13} + a_{32} \cdot b_{23}$

Practice with Python

A = np.array([[1,2,3],
              [4,5,6]])

B = np.array([[1,2],
              [4,5],
              [7,8]])

print(A@B)

When executed, this code will output like that.

# AB
[[30 36]
 [66 81]]

Let's break down the calculation step by step.
- $c_{11} = \color{red}{1} \color{black}× \color{blue}{1} \color{black}+ \color{red}{2} \color{black}× \color{blue}{4} \color{black}+ \color{red}{3} \color{black}× \color{blue}{7} \color{black}= \color{black}{30}$
- $c_{12} = \color{red}{1} \color{black} × \color{blue}{2} \color{black} + \color{red}{2} \color{black}× \color{blue}{5} \color{black}+ \color{red}{3} \color{black}× \color{blue}{8} \color{black}= \color{black}{36}$
- $c_{21} = \color{red}{4} \color{black}× \color{blue}{1} \color{black}+ \color{red}{5} \color{black}× \color{blue}{4} \color{black}+ \color{red}{6} \color{black}× \color{blue}{7} \color{black}= \color{black}{66}$
- $c_{22} = \color{red}{4} \color{black}× \color{blue}{2} \color{black}+ \color{red}{5} \color{black}× \color{blue}{5} \color{black}+ \color{red}{6} \color{black}× \color{blue}{8} \color{black}= \color{black}{81}$
- $AB = \begin{bmatrix} 30 & 36 \\ 66 & 81 \end{bmatrix}$

Properties

Non-Commutativity

It’s also a simple rule.

$$ AB \neq BA $$

For example, Here is $A$ = $2\times 3$ matrix, $B$ = $3\times 2$ matrix.

then $AB$ = $2 \times 2$ matrix
but $BA$ = $3 \times 3$ matrix!

Associativity

$$ (AB)C = A(BC) $$

This means that when multiplying several matrices, the placement of parentheses doesn’t affect the result.

Distributivity

$$ A(B+C) = AB + AC $$

This holds when $A$ = $m \times n$ matrix, $B$, $C$ = $n \times p$ matrices.

A = np.array([[1,2,3],
              [4,5,6],
              [7,8,9]])

B = np.array([[5,1],
              [1,7],
              [9,3]])

C = np.array([[4,6],
              [7,3],
              [1,5]])

print(A@(B+C))
print(A@B + A@C)

print(A@(B+C) == A@B + A@C)

You can check outputs about distributivity.

# A(B+C)
[[ 55  51]
 [136 126]
 [217 201]]

# AB + AC
[[ 55  51]
 [136 126]
 [217 201]]
 
# Consistency!
[[ True  True]
 [ True  True]
 [ True  True]]

Row Vector-Matrix Multiplication

The product($uA$) of row vector $u$, matrix $A$ is a linear combination of the row vectors of $A$.

Formula

$$ u = [u_1, u_2, u_3] $$

$$ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix} $$

$$ uA = [u_1, u_2, u_3] \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix} = [v_1, v_2, v_3] $$

Each element of $u$ is multiplied by the corresponding row of $A$ and the all are added together.

Practice with Python

A = np.array([[1,2,3],
              [4,5,6],
              [7,8,9]])
              
u = np.array([3,4,5])

print(A@u)

You can get the output like that.

[54 66 78]

Let's break down the calculation step by step.
In other expression.

Matrix-Column Vector Multiplication

The product($Ax$) of matrix $A$, column x is a linear combination of the column vectors of $A$.

Formula

$$ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix} $$

$$ x = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} $$

Each element of the resulting vector is calculated as the dot product of a row of matrix $A$ with the column vector $x$.

Practice with Python

A = np.array([[1,2,3],
              [4,5,6],
              [7,8,9]])

x = np.array([[3],
              [4],
              [5]])

print(A@x)

You can check output like below.

[[26]
 [62]
 [98]]

Let's break down the calculation step by step.
In other expression.