Matrices and Matrix Calculations of Linear Algebra

[ExtravagantDreams]

My intent is to create a thread for people interested in Linear Algebra using Matrices. However, I will explicitly state that I am only a student of this class myself and that many things could end up being wrong or an improper way to teach.

I will merely be going along with the class, mostly using excerpts and questions from the book, "Linear Algebra and its Applications: Third Edition," by David C. Lay. So truthfully, this is more for myself. Looking things up and explaining it to others seems to be the best way to learn.

If people have any questions or comments, feel free to share. Also, I know there are many knowledgeable people on this board, so be sure to correct me or make suggestions. This will be very similar to my Intro. to Differential Equations thread.

Some background;

A linear equations is one that has only base variables standing alone or scaled. In most cases I will be giving examples using the variable x with subscripts of whole numbers (i.e 1, 2, ..., n) A linear equation is there for something in this form;

a₁x₁ + a₂x₂ + ... +a_nx_n

where the a's can be any real or complex number and the x's any variable.

For example: 3x₁ - [squ](6)x₂ = 10
is linear while: 2x₁x₂ + 6[squ](x₃) = 10 is not. This is because of both the x₁x₂ and [squ](x₃).

A system of linear equation is then just a set or combination of linear equations usually written one above the other with the symbol "{" denoting the set.

---

A system of a certain number of constants and equations can have three possibilities for solutions; No solution (inconsistent), one solution, or an infinite amount of solutions.

In general, if there are more variables than equations, then the system has infinite many solutions, and if the number of variables equal to the number of equations, the system will only have one solution. But this is not something to memorize, you will easily see this and the exceptions to this very soon. It can easily be shown using two variables and graphing the equations, the intersection will provide the solution.

Starting with the system;

3x₁ + 2x₂ = 6
1x₁ + 1x₂ = 1

A system can be solved by solving for one variable first, either by substitution or addition. Let's look at substitution of this example first.

Taking the second equation it can be rewritten as;
1x₁ = 1 - 1x₂

this can then be substituted into the first equations;
3(1 - 1x₂) + 2x₂ = 6

Now that there is only one variable the equation can be solved;
3 - 3x₂ + 2x₂ = 6
x₂ = -3

This can then be substituted into any of the original equations, let's use the second since it is much simpler;
1x₁ + 1(-3) = 1
1x₁ = 4

Therefore the answer;
x₁ = 4
x₂ = -3

This can become extremely tedious with many places for error as the variables increase so let us look at a different method.

Any equation can be multiplied by a scalar and still be the same equations, so let us take the original and multiply the second by negative three (-3) to obtain;

3x₁ + 2x₂ = 6
-3x₁ - 3x₂ = -3

An addition of the two equation can the be simply be preformed to obtain;
-x₂ = 3

and then this solution be plugged back into obtain the same answers;
x₁ = 4
x₂ = -3

The idea with this is to cancel out variables by addition of appropriate scalar quantities. But using this form can become very confusing and cluttering so let us transform this into a matrix which will allow for a much more organized and clean look.

Let us simply take the coefficients of the last example and write a coefficient matrix;

http://www.angelfire.com/tx6/extravagantdreams/matrix1.jpg

This is a 2X2 matrix, where the first number denotes the number of rows and the second the number of columns. You will later see why and how this can be done.

To actually solve this problem we will need the numbers for which each equation is equivilant to. Let us just add these to the end and make what is called an augmented matrix.

http://www.angelfire.com/tx6/extravagantdreams/matrix2.jpg

Then just writting the coefficients of the previous system we see that;

3x₁ + 2x₂ = 6
-3x₁ - 3x₂ = -3

http://www.angelfire.com/tx6/extravagantdreams/matrix3.jpg

3x₁ + 2x₂ = 6
0x₁ - 1x₂ = 3

http://www.angelfire.com/tx6/extravagantdreams/matrix4.jpg

The idea of a matrix is to have the first term in the first row equal to one and then adding a scalar quantity of that row to the ones below to cancel that first term. Then turning the second number in the second row to a one and adding a scalar quantity of that row to the rows below it to cancel the second term. This goes on until all rows are done and matrix such as this has been created;

http://www.angelfire.com/tx6/extravagantdreams/matrix5.jpg

Where the "k" denotes any real number.

This can be accomplished by using Elementary Row Operations:
1. Replacement - Replace one row by the sum of itself and a multiple of another row.
2. Interchange - Interchange two rows
3. Scaling - Multiply all entries in a row by a nonzero constant.

Let us try an example:
Given the system;
x₁ - 2x₂ + x₃ = 0
x₂ - 8₃ = 9
-3x₂ + 13x₃ = -9

Making sure we line up our x variable coefficient terms correctly we can write this in an augmented matrix

http://www.angelfire.com/tx6/extravagantdreams/matrix6.jpg

Since the first term in the first row is already one we can skip 3 of the Elementary Row Operations (ERO). Since the first terms in the subsequent rows are already zero we can also skip 1 of the ERO for the first term. Now let's make the second term in the second row equal one by using 3 of the ERO

http://www.angelfire.com/tx6/extravagantdreams/matrix7.jpg

Now let us use 1 to create this matrix;

http://www.angelfire.com/tx6/extravagantdreams/matrix8.jpg

The result can be written as;
x₁ - 2x₂ + x₃ = 0
x₂ - 4x₃ = 4
x₃ = 3

The solution is then;
x₁ = 29
x₂ = 16
x₃ = 3

Some extra links
http://www.ma.utexas.edu/users/kit/les082903web.pdf
http://www.ma.utexas.edu/users/kit/les090303web.pdf

 

[ExtravagantDreams]

Row Reduction and Echelon Forms

I will try not to use so many pictures again, it was just a pain in the butt to do. So instead I will combine them all into one picture and label them as steps and explain them by the step number, unless someone know of how to put this directly onto the computer.
I also ask that you only open the image once while reading the entire post, I have very limited bandwidth.
---

A Matrix is in Echelon Form if it has the following three properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading entry of the row above it.
3. All entries in a column below a leading entry are zeros.

This is shown in examples one and two of the picture.

If the matrix also meets the following requirements it is in Reduced Echelon Form:
4. The leading entry in each nonzero row is 1
5. each leading 1 is the only nonzero entry in its column (except for the last column if it is an augmented matrix.

This is shown in examples three and four of the picture. The fifth example is a general form where * depicts any number and the arrows show where there must be zeros.

Pivot Positions and columns

A pivot position is the position of the first nonzero value of a matrix in echelon form. A pivot column is a column with a pivot postion. Each variable that does not have a pivot position is a free variable, one that the answer to is unknown.

Example six is of an augmented matrix first forwards row reduced to create echelon form to show pivot positions and columns, then backwards row reduced to obtain the answer.

http://www.angelfire.com/tx6/extravagantdreams/Matrix2_1.jpg

http://www.ma.utexas.edu/users/kit/les090503web.pdf

 

[ExtravagantDreams]

Vector Equations

A Matrix with only one column is called a column vector or simple a vector.

Addition of Vectors; add vectors by addition of their components
Ex. 1

Scalar Product; multiply by constant
Ex. 2

Algebraic Properties of Rⁿ
for all u, v, w (where a bold variable will denote a vector) in Rⁿ and all scalars c and d;

1. u + v = v + u
2. (u + v) + w = u + (v + w)
3. u + 0 = 0 + u = u
4. u + (-u) = -u + u = 0
5. c(u + v) = cu + cv
6. (c + d)u = cu + du
7. c(du) = (cd)u

Let's take a look at an addition of three vectors a₁, a₂, and a₃ with components x₁, x₂, and x₃ equal to b
Ex. 3

Let V₁, ..., V_n be in IRⁿ. The set of all linear combinations of V₁, ..., V_n are denoted by the span {V₁, ..., V_n} and is called the sub set of IRⁿ spanned.

Span Rⁿ depends not on the number of vectors or coordinates but on the number of pivot positions. So a matrix of n vectors with x components each will span R^{Pivot Points}

http://www.angelfire.com/tx6/extravagantdreams/Matrix1_3.jpg
http://www.ma.utexas.edu/users/kit/les091203web.pdf

 

[ExtravagantDreams]

The Matrix Equations : Ax = b

If A is an m X n matrix, with columns a₁, ..., a_n, and if x is in Rⁿ, then the product of A and x, denoted by Ax, is the linear combination of the columns of A using the corresponding entries in x as weights;

Ex. 1

For a multiplication of a matrix and a vector to occur, the number of entries in a vector must equal the number of columns in the matrix.

To solve, multiply the first term in the vector by the first column in the matrix, then adding the second by the second and so on. It can also be done by multiply the first term in the vector by the first term of the first row in the matrix, then adding the second by the second in the first row and so on. Then repeating this for any subsequent rows.

Examples 2 and 3

In the previous matrices we sort of magically made the X_n terms vanish and just wrote their coefficients. In thruth, these x terms factor out and become vector of x.

Let us solve;
x₁ + 2x₂ -x₃ = 4
-5x₂ + 3x₃ = 1

Ex. 4

If the matrix is an identity matrix (a square matrix such that there are 1's across the downward diagonal and 0's elsewhere), the multiplication of any vector will result in that vector that spans R^{size of matrix}. For any n X n identity matrix, multiplied by a vector x with n terms, will result in the x for every x in Rⁿ.

Ex. 5

Properties of Matrix - Vector Product Ax
If A is an m X n matrix, u and v are vectors in Rⁿ, and c is a scalar, then:
1. A(u) + v = Au + Av (Ex. 6)
2. A(cu) = c(Au)

http://www.angelfire.com/tx6/extravagantdreams/Matrix1_4.jpg

http://www.ma.utexas.edu/users/kit/les091203web.pdf

 

[ExtravagantDreams]

Sol. Sets of Linear Systems and (in)dependence

Homogeneous Linear Systems

This is; equations equal to zero, as in Ax = 0
Where A is an m X n matrix and 0 is a vector in R^m

This system always has atleast one solution, namely x = 0 (the trivial solution). If one x has a non trivial solution, then other terms can be defined by the nontrivial solution.

Ex. 1

Parametric Vector Form



NonHomogeneous Linea Systems

This is; simply an equation equals to some number, as in Ax = b They are solved the same way as Homogeneous Linear Systems.

Ex. 2

w = p + v_h

Writting a solution set (of a consistent system) in Parametric Vector Form.


Linear Independence

Is the trivial solution the only one?

An index set of vectors {v₁, ..., v_p} in Rⁿ is said to be Linear Independent if the vector equation

x₁v₁ + x₂v₂ + ... + x_pv_p = 0

has only the trivial solution. The set {v₁, ..., v_p}is said to be linearly dependent if there exists c₁, ..., c_p, not all zero, such that

c₁v₁ + c₂v₂ + ... + c_pv_p = 0

Ex. 3 and Ex. 4

Linear Independence of Matrix Columns

Ex. 5


This is obviously not finished.

 

[ExtravagantDreams]

Matrix Operations

First, let's start by denoting the entries in a matrix. In a m X n (m rowns, n columns) matrix the entries can be written as a_{row number and column number}, such that the first entry in the first column is denoted by a₁₁. The second entry in this colum would then be a₂₁, all the way to a_m1. The first entry in the second row would then be a₁₂ and the last entry in the last row; a_mn.

Example 1

Sum of matrices.

Two matrices, A and B, can be added if each entry has a corresponding entry in the other matrix (i.e. they are the same size matrix). In such a case, matrix sums are the same as in vectors, where corresponding terms are added to form the sum.

Example 2

Also, matrices are scaled like vectors, where each entriy is multiplied by a scalar number c.

Example 3

Theorem
Let A, B and C be matricies of the same size, and let r and s be scalars.

a. A + B = B + A
b. (A + B) + C = A + (B + C)
c. A + 0 = A
d. r(A + B) = rA + rB
e. (r + s)A = rA + sA
f. r(sA) = (rs)A

Matrix Multiplication

Remember how to multiply a vector of n entries to a m X n matrix, the process is the same for two matrices. In the case of multiplying matrices, the two matrices must not be equal in size but the first must have the same number of columns as the second in rows (i.e. AB only if matrix A is a m X n matrix and B is a n X p matrix). the result will then be matrix C, a m X p matrix.

Row-Column Rule for computing C
If the product of AB is defined, then the entry in row i and column j of C is the sum of the products of the corresponding entries from row i of A and column j of B.

Example 4

Prperties of Matrix Multiplication
Recall that I_m represents the m X m identity matrix and I_mx = x for all x in R^m

Let A be an m X n matrix, and let B and C have sizes for which the indicated sums and products are defined. The term r is just a scalar

a. A(BC) = (AB)C
b. A(B + C) = AB + AC
c. (B + C)A = BA + CA
d. r(AB) = (rA)B = A(rB)
e. I_mA = A = A I_n

In general,
AB [x=] BA
If AC = AB, C [x=] B
If AB is the zero matrix, you cannot conclude that either A or B equal zero

Example 5

Powers of Matrix

If A is an n X n matrix and if k is a positive interfer, then A^k denotes the product of A k times.

If A is nonzero and if x is in Rⁿ, then A^kx is the resolt of left-multiplying x by A repeatedly k times. if k = 0, then A⁰x and should be x itself. Thus A⁰ is interpreted as the identity matrix.

Transpose of a Matrix
Given an m X n Matrix A, the transpose of A is the n X m matrix, denoted A^T, whose columns are formed from the corresponding rows of A.

Example 6

Theorem
Let A and B denot matrices whose sizes are appropriate for the following sums and products and r is a scalar,

a. (A^T)^T = A
b. (A + B)^T = A^T + B^T
c. (rA)^T = rA^T
d. (AB)^T = B^TA^T

http://www.angelfire.com./tx6/extravagantdreams/Matrix2_1_1.jpg
http://www.angelfire.com./tx6/extravagantdreams/Matrix2_1_2.jpg
http://www.ma.utexas.edu/users/kit/les091903web.pdf

 

[ExtravagantDreams]

Inverse of Matrix

Whoa! it has been way too long since I have last updated this. I have been too busy and concentrating too much on my other thread, even though it is way behind as well.

Let us speak briefly about matrix inversion, it is a relatively easy concept. It will be in the link below, I have changed format. I will no longer type here and include pictures, I will now instead use word documents with equation editor.

The Inverse of Matrix

I hope no one will have problems this way, if so be sure to tell me and I will try a different format.