Matrix Algebra¶
Statisticians and data scientists use the language of vectors and matrices to organize data and work with models. We will take a close look at matrix algebra, taking time to relate matrix algebra to the more familar algebra that you have worked with in the past.
Recap on vectors and operations on vectors¶
Definition¶
A vector is an ordered list of real numbers. We denote a vector with a lower case letter and enclose the values in the vector with square brackets. We can write a vector as a row vector
or as a column vector
.
A vector has a length, defined as the number of values included in that vector. For example, the above vector is length 5.
We can create a vector in R by using the c operator.
``` {}
\PY{n}{v} \PY{o}{=}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{)}
```Vector times a scalar¶
A vector of length one is called a scalar. We can define the results of multiplying a vector by a scalar as each entry of the vector times the scalar .
For example, the vector times the scalar will result in a vector
R understands how to multiply vectors and scalars with no additional syntax.
``` {}
\PY{n}{v} \PY{o}{=} \PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{)}
\PY{n}{alpha} \PY{o}{=} \PY{l+m}{7}
\PY{n}{alpha}\PY{o}{*}\PY{n}{v}
```7
14
21
28Vector plus/minus a vector¶
A vector plus a vector creates a new vector that adds the individual entries of and
A vector minus a vector creates a new vector that subtracts the individual entries of from
R also understands vector addition and subtraction with no special syntax.
``` {}
\PY{n}{v} \PY{o}{=} \PY{n+nf}{c}\PY{p}{(}\PY{l+m}{4}\PY{p}{,} \PY{l+m}{\PYZhy{}1}\PY{p}{,} \PY{l+m}{7}\PY{p}{)}
\PY{n}{q} \PY{o}{=} \PY{n+nf}{c}\PY{p}{(}\PY{l+m}{0}\PY{p}{,} \PY{l+m}{32}\PY{p}{,} \PY{l+m}{9}\PY{p}{)}
\PY{n}{add} \PY{o}{=} \PY{n}{v}\PY{o}{+}\PY{n}{q}
\PY{n}{subtract} \PY{o}{=} \PY{n}{v}\PY{o}{\PYZhy{}}\PY{n}{q}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{add}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{subtract}\PY{p}{)}
```[1] 4 31 16
[1] 4 -33 -2The Matrix¶
A matrix is an ordered list of vectors.
A matrix has a dimension which is an ordered pair of numbers: the first number indicates the number of rows of the matrix and the second number indicates the number of columns. For example, he matrix above has dimension .
To build a matrix in R we can issue the matrix command. Matrix is a function that takes a vector as an argument and can take one of many optional arguments.
If we give the matrix function only a vector, the default behavior of this function is to create a vector.
``` {}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{)}\PY{p}{)}
\PY{n}{A}
```A matrix: 6 × 1 of type dbl
1 2 3 4 5 6¶
This is because a vector can be thought of as a matrix with one column. In other words, a matrix with dimension is a vector of length .
We can provide another arguement to the matrix function---ncol---to specify the number of columns we want to create.
``` {}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{)}\PY{p}{,} \PY{n}{ncol}\PY{o}{=}\PY{l+m}{2}\PY{p}{)}
\PY{n}{A}
```A matrix: 3 × 2 of type dbl
1 4 2 5 3 6
Above we used the ncol argument to specify a matrix with two columns. R will automatically determine the number of rows for the matrix and fill-in the values of the matrix column by column.
If we want, we can ask R to fill in values of the matrix row by row by including an additional argument in the matrix functionbyrow
``` {}
\PY{n}{v} \PY{o}{=} \PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{)}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n}{v}\PY{p}{,} \PY{n}{ncol}\PY{o}{=}\PY{l+m}{2}\PY{p}{)}
\PY{n}{B} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n}{v}\PY{p}{,} \PY{n}{ncol}\PY{o}{=}\PY{l+m}{2}\PY{p}{,} \PY{n}{byrow}\PY{o}{=}\PY{k+kc}{TRUE}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{A}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{B}\PY{p}{)}
```[,1] [,2]
[1,] 1 4
[2,] 2 5
[3,] 3 6
[,1] [,2]
[1,] 1 2
[2,] 3 4
[3,] 5 6We see that matrix A was “column filled” and matrix B was “row filled”.
There are similar operations for matrices as there are for vectors.
Matrix plus/minus a matrix¶
Given a matrix and matrix , the sum of and is a new matrix where the i,j entry of () is the sum of the corresponding entries from and (). To add two matrices they must have the same dimension.
R understands matrix addition and subtraction without any additional syntax.
``` {}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{\PYZhy{}1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{8}\PY{p}{)}\PY{p}{,} \PY{n}{ncol}\PY{o}{=}\PY{l+m}{3}\PY{p}{)}
\PY{n}{D} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{90}\PY{p}{,}\PY{l+m}{0.34}\PY{p}{,}\PY{l+m}{1.54}\PY{p}{,}\PY{l+m}{\PYZhy{}9.43}\PY{p}{,}\PY{l+m}{10}\PY{p}{,}\PY{l+m}{0}\PY{p}{)}\PY{p}{,} \PY{n}{ncol}\PY{o}{=}\PY{l+m}{3}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{A}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{D}\PY{p}{)}
\PY{n}{A}\PY{o}{+}\PY{n}{D}
```[,1] [,2] [,3]
[1,] -1 4 3
[2,] 2 1 8
[,1] [,2] [,3]
[1,] 90.00 1.54 10
[2,] 0.34 -9.43 0A matrix: 2 × 3 of type dbl
89.00 5.54 13 2.34 -8.43 8
``` {}
\PY{n}{A}\PY{o}{\PYZhy{}}\PY{n}{D}
```A matrix: 2 × 3 of type dbl
-91.00 2.46 -7 1.66 10.43 8
Matrix times a Matrix¶
Two matrices and can be multiplied together if the number of columns of is equal to the number of rows of . In other words, if the dimension of A is (a,b) and the dimension of B is (c,d) then b and c must be equal.
The i,j entry of this product, is the following sum of products:
For example, suppose that
and that
Then the product equals
The above definition and computation can feel cumbersome. We can simplify the above calculations by introducing the inner product.
The inner product between two vectors and , or , is
Let us use the inner product to simplify the above matrix multiplication. First, we rewrite the matrix A as a stack of two row vectors
where and
Second, we rewrite the matrix B as a stack of three column vectors
where $b_{1} =
1\
3$, $b_{2} =
2\
4$, and $b_{3} =
-1\
-2$
Then the product AB is a matrix of inner products
Matrix multiplication is different than mulitplication between two variables that represent real numbers because matrix multiplication is not commutative---the product is not rarely equal to .
To compute the product of two matrices in R we need the %*% operator
``` {}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{\PYZhy{}1}\PY{p}{,}\PY{l+m}{0}\PY{p}{,}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{\PYZhy{}2}\PY{p}{,}\PY{l+m}{\PYZhy{}1}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{)}\PY{p}{,}\PY{n}{ncol}\PY{o}{=}\PY{l+m}{3}\PY{p}{)}
\PY{n}{B} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{,}\PY{l+m}{7}\PY{p}{,}\PY{l+m}{8}\PY{p}{,}\PY{l+m}{9}\PY{p}{)}\PY{p}{,}\PY{n}{ncol}\PY{o}{=}\PY{l+m}{3}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{A}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{B}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{A}\PY{o}{\PYZpc{}*\PYZpc{}}\PY{n}{B}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{B}\PY{o}{\PYZpc{}*\PYZpc{}}\PY{n}{A}\PY{p}{)}
```[,1] [,2] [,3]
[1,] -1 -2 4
[2,] 0 -1 5
[3,] 1 3 6
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
[,1] [,2] [,3]
[1,] 7 10 13
[2,] 13 25 37
[3,] 25 55 85
[,1] [,2] [,3]
[1,] 6 15 66
[2,] 6 15 81
[3,] 6 15 96Note above that we computed the product AB and BA and these products resulted in different matrices.
Matrix times a vector¶
A matrix with dimension can be multiplied by a vector if the length of is . Then the product is a vector with the the i entry of defined as
For example, Let the matrix
and the vector
then
The definition above can also be simplifed by using inner products. Let
where , , and . Then the product simplifies to
We can use the %*% operator in R to multiply together a matrix and a vector
``` {}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{)}\PY{p}{,} \PY{n}{ncol}\PY{o}{=}\PY{l+m}{2}\PY{p}{,} \PY{n}{byrow}\PY{o}{=}\PY{k+kc}{TRUE}\PY{p}{)}
\PY{n}{v} \PY{o}{=} \PY{n+nf}{c}\PY{p}{(}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{\PYZhy{}3}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{Matrix A\PYZdq{}}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{A}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{Vector v\PYZdq{}}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{v}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{Av\PYZdq{}}\PY{p}{)}
\PY{n}{product} \PY{o}{=} \PY{n}{A}\PY{o}{\PYZpc{}*\PYZpc{}}\PY{n}{v}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{product}\PY{p}{)}
```[1] "Matrix A"
[,1] [,2]
[1,] 1 2
[2,] 3 4
[3,] 5 6
[1] "Vector v"
[1] 4 -3
[1] "Av"
[,1]
[1,] -2
[2,] 0
[3,] 2Matrix Transpose¶
Given a matrix , the tranpose of , or , is a matrix where the first row of corresponds to the first column of , the second row of corresponds to the second columns of and so on. If has dimension than has dimensions .
For example, if
then the tranpose of ,
We can use the t() operator to transpose a matrix in R.
``` {}
\PY{n}{M} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(} \PY{l+m}{1}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{,}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{2}\PY{p}{,}\PY{l+m}{13}\PY{p}{,}\PY{l+m}{90}\PY{p}{,}\PY{l+m}{23}\PY{p}{,}\PY{l+m}{0}\PY{p}{)}\PY{p}{,}\PY{n}{ncol}\PY{o}{=}\PY{l+m}{3}\PY{p}{,}\PY{n}{byrow}\PY{o}{=}\PY{k+kc}{TRUE}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{The matrix M\PYZdq{}}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{M}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{The transpose\PYZdq{}}\PY{p}{)}
\PY{n}{Mt} \PY{o}{=} \PY{n+nf}{t}\PY{p}{(}\PY{n}{M}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{Mt}\PY{p}{)}
```[1] "The matrix M"
[,1] [,2] [,3]
[1,] 1 2 3
[2,] 4 5 6
[3,] 3 2 13
[4,] 90 23 0
[1] "The transpose"
[,1] [,2] [,3] [,4]
[1,] 1 4 3 90
[2,] 2 5 2 23
[3,] 3 6 13 0Matrix inverse¶
For a super fun overview of matrix multiplication and the concept of identities and inverses, click here!
The identity matrix¶
The identity matrix of dimension , usually labled , is a matrix with rows and columns such that the diagonal elements of the matrix, the (1,1) entry, (2,2) entry, up to (r,r) entry are the number 1 and all other entries are 0. For example,
This matrix is called the identity matrix because any matrix times returns . To be more precise, this matrix is the multiplicative identity matrix. But the adjective multiplicative is usually dropped.
The inverse matrix¶
The inverse of a matrix ------is a matrix such that, if has the same number of rows and columns (called square) then
The idea of a matrix inverse is the same as in algebra. If we have a variable then the inverse of , called is the unique number such that
In matrix algebra, the identity matrix takes the place of the “1” in algebra.
We can compute the inverse of a matrix in R using the solve function.
``` {}
\PY{n}{A} \PY{o}{=} \PY{n+nf}{matrix}\PY{p}{(}\PY{n+nf}{c}\PY{p}{(}\PY{l+m}{3}\PY{p}{,}\PY{l+m}{4}\PY{p}{,}\PY{l+m}{5}\PY{p}{,}\PY{l+m}{6}\PY{p}{)}\PY{p}{,}\PY{n}{ncol}\PY{o}{=}\PY{l+m}{2}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{A\PYZdq{}}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{A}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{l+s}{\PYZdq{}}\PY{l+s}{A inverse\PYZdq{}}\PY{p}{)}
\PY{n}{Ainverse} \PY{o}{=} \PY{n+nf}{solve}\PY{p}{(}\PY{n}{A}\PY{p}{)}
\PY{n+nf}{print}\PY{p}{(}\PY{n}{Ainverse}\PY{p}{)}
```[1] "A"
[,1] [,2]
[1,] 3 5
[2,] 4 6
[1] "A inverse"
[,1] [,2]
[1,] -3 2.5
[2,] 2 -1.5Assignment¶
Define the vector v of length 3 with the following numbers: -1,0,1
Define the matrix A with dimension (2,3) by filling, column wise, the values: 1,2,3,4,5,6
Multiply A by v (Av)
Multiply v by A (vA). Why does R return an error?
Write a function that takes two argument which are both matrices and returns the product of these matrices.