Section 4.1 Vector Equations, Span
Subsection 4.1.1 Vector Equations
A vector equation is an equation involving vectors.
Consider the linear system,
\begin{equation*}
\begin{array}{rrrr}
2x \amp -y \amp = \amp 0 \\
-x \amp + 2y \amp = \amp 3
\end{array}
\end{equation*}
By definition of vector equality, this is equivalent to the (vector) equation,
\begin{equation*}
\begin{bmatrix} 2x - y \\ -x + 2y \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix}
\end{equation*}
Then, the left hand side can be expanded, to show how it depends on \(x\) and \(y\text{,}\)
\begin{align*}
\begin{bmatrix} 2x \\ -x \end{bmatrix} + \begin{bmatrix} -y \\ 2y \end{bmatrix} \amp = \begin{bmatrix} 0 \\ 3 \end{bmatrix}\\
x \begin{bmatrix} 2 \\ -1 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \end{bmatrix} \amp = \begin{bmatrix} 0 \\ 3 \end{bmatrix}
\end{align*}
Then, solving this linear system involves determining all possible linear combinations of the vectors \((2,-1)\) and \((-1,2)\) which are equal to \((0,3)\text{.}\) Further, the column vectors are precisely the columns of the coefficient matrix of the system,
\begin{equation*}
\begin{bmatrix} 2 \amp -1 \\ -1 \amp 2 \end{bmatrix}
\end{equation*}
Geometrically, we can solve this system using the parallelogram rule for addition. Let \(\vec{a}_1 = (2,-1), \vec{a}_2 = (-1,2)\) and \(\vec{b} = (0,3)\text{.}\) Sketch all of these vectors in standard position. Sketch a parallelogram with sides parallel to \(\vec{a}_1\) and \(\vec{a}_2\text{,}\) with one vertex at the origin, and opposite vertex at the the head of \(\vec{b}\text{.}\) Then, the sides parallel to \(\vec{a}_1\) and \(\vec{a}_2\) are multiples of \(\vec{a}_1\) and \(\vec{a}_2\) whose sum (by the parallelogram rule) is \(\vec{b}\text{.}\) These scalar multiples are the solution to the system.
More generally, consider a system of linear equations,
\begin{alignat*}{3}
a_{11}x_1 \amp + a_{12}x_2 \amp + \dots \amp + a_{1n}x_n \amp = b_1\\
a_{21}x_1 \amp + a_{22}x_2 \amp + \dots \amp + a_{2n}x_n \amp = b_2\\
\vdots \quad \amp \qquad \vdots \amp \amp \qquad \vdots \amp \vdots\\
a_{m1}x_1 \amp + a_{m2}x_2 \amp + \dots \amp + a_{mn}x_n \amp = b_m
\end{alignat*}
with augmented matrix,
\begin{equation*}
\left[\begin{array}{cccc|c}
a_{11} \amp a_{12} \amp \dots \amp a_{1n} \amp b_1 \\
a_{21} \amp a_{22} \amp \dots \amp a_{2n} \amp b_2 \\
\vdots \amp \vdots \amp \ddots \amp \vdots \amp \vdots \\
a_{m1} \amp a_{m2} \amp \dots \amp a_{mn} \amp b_n
\end{array}\right]
\end{equation*}
For brevity, and to focus on the column vectors making up the matrix, we will denote the columns of the coefficient matrix \(A\) as \(\vec{a}_1, \vec{a}_2, \dots, \vec{a}_n\text{,}\) and the constant vector as \(\vec{b}\text{.}\) Then, the augmented matrix can be denoted by,
\begin{equation*}
\begin{bmatrix} \vec{a}_1 \amp \vec{a}_2 \amp \dots \amp \vec{a}_n \amp \vec{b} \end{bmatrix}
\end{equation*}
Intuitively, this is a row vector with entries which are themselves column vectors, forming a matrix.
Then, in summary,
Theorem 4.1.2. Linear systems as vector equations..
The system of linear equations,
\begin{alignat*}{3}
a_{11}x_1 \amp + a_{12}x_2 \amp + \dots \amp + a_{1n}x_n \amp = b_1\\
a_{21}x_1 \amp + a_{22}x_2 \amp + \dots \amp + a_{2n}x_n \amp = b_2\\
\vdots \quad \amp \qquad \vdots \amp \amp \qquad \vdots \amp \vdots\\
a_{m1}x_1 \amp + a_{m2}x_2 \amp + \dots \amp + a_{mn}x_n \amp = b_m
\end{alignat*}
with corresponding augmented matrix,
\begin{equation*}
\begin{bmatrix} \vec{a}_1 \amp \vec{a}_2 \amp \dots \amp \vec{a}_n \amp \vec{b} \end{bmatrix}
\end{equation*}
has the same solution set as the vector equation,
\begin{equation*}
x_1 \vec{a}_1 + x_2 \vec{a}_2 + \dots + x_n \vec{a}_n = \vec{b}
\end{equation*}
In other words, solving a linear system can be thought of as determining all linear combinations of the column vectors \(\vec{a}_1, \dots, \vec{a}_n\) of the coefficient matrix \(A\) that are equal to the constant vector \(\vec{b}\text{.}\) This perspective is called the column picture.
Conversely, to solve a vector equation,
\begin{equation*}
x_1 \vec{a}_1 + \dots + x_n \vec{a}_n = \vec{b}
\end{equation*}
Form the associated system of linear equations in \(x_1, \dots, x_n\text{,}\) and form its augmented matrix, whose columns will be \(\vec{a}_1, \dots, \vec{a}_n, \vec{b}\text{.}\) Then, use row reduction to determine the solution.
Theorem 4.1.3. Vector equations as a linear system.
The vector equation,
\begin{equation*}
x_1 \vec{a}_1 + \dots + x_n \vec{a}_n = \vec{b}
\end{equation*}
has the same solution set as the system of linear equations in \(x_1, \dots, x_n\) which has augmented matrix given by,
\begin{equation*}
\begin{bmatrix} \vec{a}_1 \amp \vec{a}_2 \amp \dots \amp \vec{a}_n \amp \vec{b} \end{bmatrix}
\end{equation*}
Example 4.1.4.
\begin{equation*}
\begin{array}{rrrr}
x \amp + 3y \amp = -1 \\
-5x \amp + 2y \amp = 2
\end{array}
\end{equation*}
which has vector form,
\begin{equation*}
x \begin{bmatrix} 1 \\ -5 \end{bmatrix} + y \begin{bmatrix} 3 \\ 2 \end{bmatrix} = \begin{bmatrix} -1 \\ 2 \end{bmatrix}
\end{equation*}
From this perspective, solving this linear system involves finding all possible linear combinations of the vectors \((1, -5)\) and \((3, 2)\) in the plane, that are equal to the vector \((-1, 2)\text{.}\)Subsection 4.1.2 Introduction to Span of Vectors in \(\mathbb{R}^2, \mathbb{R}^3\)
One of the key ideas in linear algebra is to study the set of all vectors which can be written as a linear combination of a fixed set of vectors. We say that these are all the vectors “spanned”, or “generated”, by this set.
Definition 4.1.5.
Broadly, the span of a collection of vectors is the set of all of their linear combinations.Definition 4.1.6.
Let \(\vec{u}, \vec{v}\) be vectors. Then, the span of \(\vec{u}\) and \(\vec{v}\) is the set of all of their linear combinations, and is denoted by \(\Span{\vec{u}, \vec{v}}\text{.}\) In other words,
\begin{equation*}
\boxed{\Span{\vec{u}, \vec{v}} = \set{a \vec{u} + b \vec{v}: a, b \in \mathbb{R}}}
\end{equation*}
Intuitively, “span” means “the amount of space something covers”, and the span of vectors is all the possible vectors which can be “reached” by combining \(\vec{u}, \vec{v}\) using only the two fundamental operations of addition and scalar multiplication.
Example 4.1.7. A single vector in \(\mathbb{R}^2\).
Consider a single vector \(\vec{v}\) in \(\mathbb{R}^2\text{.}\) The span of \(\vec{v}\text{,}\) \(\Span{\vec{v}}\text{,}\) is the set of all linear combinations of the single vector \(\vec{v}\text{.}\) There is only one vector, so there is no addition. In this way, \(\Span{\vec{v}}\) is the set of all multiples of \(\vec{v}\text{,}\)
\begin{equation*}
\boxed{\Span{\vec{v}} = \set{a\vec{v}: a \in \mathbb{R}}}
\end{equation*}
Then, in \(\mathbb{R}^2\text{,}\) this forms a line, in particular a line through the origin \((0,0)\text{,}\) and through \(\vec{v}\text{.}\)
Example 4.1.8. Two vectors in \(\mathbb{R}^2\).
Consider two vectors \(\vec{u}, \vec{v}\) in \(\mathbb{R}^2\text{.}\) Then, by definition,
\begin{equation*}
\Span{\vec{u}, \vec{v}} = \set{a \vec{u} + b \vec{v}: a, b \in \mathbb{R}}
\end{equation*}
First, suppose that \(\vec{u}, \vec{v} \neq \vec{0}\) are non-zero. If \(\vec{u}, \vec{v}\) are not parallel, then it turns out that the span of \(\vec{u}\) and \(\vec{v}\) is all of \(\mathbb{R}^2\text{,}\) i.e. the entire plane. That is, every vector in \(\mathbb{R}^2\) can be written as a linear combination of \(\vec{u}, \vec{v}\text{.}\)
For example, a familiar case is for the standard basis vectors \(\ihat, \jhat\text{.}\) These vectors are not parallel (they are perpendicular), so,
\begin{equation*}
\Span{\ihat, \jhat} = \set{a \cdot \ihat + b \cdot \jhat : a, b \in \mathbb{R}}
\end{equation*}
and we know that every vector in \(\mathbb{R}^2\) can be written as a linear combination of \(\ihat\) and \(\jhat\text{.}\)
If \(\vec{u}, \vec{v}\) are parallel, that is, either vector is a scalar multiple of the other, then the span of \(\vec{u}, \vec{v}\) is a line (in the plane). Intuitively, the second vector doesn't “add” any new additional possible vectors. More precisely, if say \(\vec{u} = k \vec{v}\text{,}\) then,
\begin{align*}
a\vec{u} + b\vec{v} \amp = ak \vec{v} + b \vec{v}\\
\amp = (ak + b) \vec{v}
\end{align*}
That is, any linear combination of \(\vec{u}\) and \(\vec{v}\) is really just a multiple of \(\vec{v}\text{.}\) Conversely, any multiple of \(\vec{v}\) can be written as \(a\vec{u} + b \vec{v}\) for some weights \(a, b\text{.}\) All of this reasoning can be oriented in the opposite direction, to say that all linear combinations \(a\vec{u} + b\vec{v}\) are just a multiple of \(\vec{u}\text{.}\) That is,
\begin{equation*}
\Span{\vec{u}, \vec{v}} = \Span{\vec{v}} = \Span{\vec{u}}
\end{equation*}
In the most extremely pathological case, if \(\vec{u}, \vec{v}\) are both the zero vector, then their span is just the zero vector, i.e. a single point.
Example 4.1.9. Span of vectors in \(\mathbb{R}^3\).
Consider two vectors \(\vec{u}, \vec{v}\) in \(\mathbb{R}^3\text{.}\) If \(\vec{u}, \vec{v}\) are not parallel, then the span of \(\vec{u}, \vec{v}\) forms a plane in \(\mathbb{R}^3\text{.}\)
Adding a third vector \(\vec{w}\) to the span, which is not included in the plane spanned by \(\vec{u}\) and \(\vec{v}\text{,}\) results in the entire 3D space \(\mathbb{R}^3\text{.}\)
Definition 4.1.10.
Let \(\vec{u}, \vec{v}, \vec{w}\) be vectors. Then, the span of \(\vec{u}, \vec{v}, \vec{w}\) is given by,
\begin{equation*}
\boxed{\Span{\vec{u}, \vec{v}, \vec{w}} = \set{a \vec{u} + b \vec{v} + c \vec{w}: a, b, c \in \mathbb{R}}}
\end{equation*}
Subsection 4.1.3 Span of \(n\) Vectors
Definition 4.1.11.
Let \(\vec{v}_1, \dots, \vec{v}_p \in \mathbb{R}^n\) be vectors. The span of \(\vec{v}_1, \dots, \vec{v}_p\text{,}\) denoted by \(\Span{\vec{v}_1, \dots, \vec{v}_n}\text{,}\) is the set of all linear combinations of \(\vec{v}_1, \dots, \vec{v}_p\text{.}\) In other words,
\begin{equation*}
\Span{\vec{v}_1, \dots, \vec{v}_p} = \set{a_1 \vec{v}_1 + \dots a_n \vec{v}_p : a_1, \dots, a_p \in \mathbb{R}}
\end{equation*}
Notice that the span \(\Span{\vec{v}_1, \dots, \vec{v}_p}\) contains the zero vector, because \(\vec{0} = 0 \vec{v}_1 + \dots + 0 \vec{v}_p\text{.}\) Also, it contains every scalar multiple of every vector in the span, for example, every scalar multiple of \(\vec{v}_1\text{,}\) because \(c\vec{v}_1 = c\vec{v}_1 + 0\vec{v}_2 + \dots + \vec{v}_p\text{.}\)
More generally, by definition, a vector \(\vec{v}\) is in \(\Span{\vec{v}_1, \dots, \vec{v}_p}\) if and only if \(\vec{v}\) can be written as a linear combination of \(\vec{v}_1, \dots, \vec{v}_p\text{,}\) i.e. the following vector equation has a solution,
\begin{equation*}
x_1 \vec{v}_1 + x_2 \vec{v}_2 + \dots + x_p \vec{v}_p = \vec{v}
\end{equation*}
Thus, to determine if \(\vec{v}\) is in the span of \(\vec{v}_1, \dots, \vec{v}_n\text{,}\) solve the linear system with augmented matrix,
\begin{equation*}
\begin{bmatrix} \vec{v}_1 \amp \dots \amp \vec{v}_p \amp \vec{v} \end{bmatrix}
\end{equation*}
If this system has a solution, then \(\vec{v} \in \Span{\vec{v}_1, \dots, \vec{v}_n}\text{,}\) and any solution \(a_1, \dots, a_n\) provides the weights of the linear combination.
