Definition 6.1.1.
A vector space is a set of vectors \(V\) such that,
- If \(\vec{u}, \vec{v} \in V\text{,}\) then \(\vec{u} + \vec{v} \in V\) also.
- If \(\vec{v} \in V\) and \(c \in \mathbb{R}\text{,}\) then \(c\vec{v} \in V\) also.
Section 6.1 Real Coordinate Spaces, Column Space and Null Space
Subsection 6.1.1 Introduction to Vector Spaces
A vector space is a space of vectors. Broadly, a “space” is a set of objects which satisfy some specific conditions. In particular, vector spaces are sets of vectors for which we can perform the fundamental operations for vectors: addition and scalar multiplication. That is, we can take linear combinations.
Slightly more precisely, a vector space is a set \(V\) of vectors for which adding any two vectors in \(V\) is a vector that is also in \(V\text{,}\) and the scalar multiple of any vector in \(V\) is also in \(V\text{.}\) In other words, adding and scaling vectors in the space “stays” in the space.
This property that combining objects through some operation producing another object also within the set is called closure. Then, a vector space is a set of vectors which is closed under addition and scalar multiplication. More compactly, we can say a vector space is closed under taking linear combinations.
Example 6.1.2. The vector space \(\mathbb{R}^2\).
The set \(\mathbb{R}^2\) of all vectors with two real number components, is a vector space.
Example 6.1.3. The vector space \(\mathbb{R}\).
The real numbers \(\mathbb{R}\) can also be considered a vector space. The sum of two real numbers is a real number, and multiplying a real number by another real number is a real number.
Example 6.1.4. The vector space \(\mathbb{R}^n\).
More generally, \(\mathbb{R}^n\) is a vector space.
Subsection 6.1.2 Subspaces
Many interesting vector spaces are subsets of other vectors spaces. This naturally leads to the concept of a vector subspace.
Let \(V\) be a vector space. Then, a subset \(U \subseteq V\) is a vector subspace of \(V\) (or simply subspace) if \(U\) is itself a vector space.
For now, most of the vector spaces we encounter will be subspaces of \(\mathbb{R}^n\text{,}\) for some \(n\text{.}\)
Example 6.1.5. Subspaces of \(\mathbb{R}^2\).
Consider the possible subspaces of \(\mathbb{R}^2\text{.}\) Let \(U \subseteq \mathbb{R}^2\) be a subspace. First, notice that if \(\vec{v} \in U\) is any vector, then \(0 \vec{v} = \vec{0}\) is also in \(U\text{,}\) because \(U\) is closed under scalar multiplication. Thus, any subspace of \(\mathbb{R}^2\) must contain the zero vector. Indeed, the vector space containing only the zero vector \(U = \set{\vec{0}}\) is a vector space. This can be trivially verified, since any scalar multiple of \(\vec{0}\) is \(\vec{0}\text{,}\) and any sum of \(\vec{0}\) is \(\vec{0}\text{.}\)
Then, suppose \(U\) contains any non-zero vector \(\vec{d} \neq \vec{0}\text{.}\) Then, \(U\) must also contain all scalar multiples of \(\vec{d}\text{.}\) Thus, \(U\) must contain the line parallel to \(\vec{d}\) and through the origin. Indeed, this set, which is a line in \(\mathbb{R}^2\text{,}\) or the span of \(\vec{d}\text{,}\) is a vector space.
\begin{equation*}
U = \span{\vec{d}} = \set{c \vec{d}: c \in \mathbb{R}}
\end{equation*}
Indeed, if \(\vec{u}, \vec{v} \in U\text{,}\) then they are both scalar multiples of \(\vec{d}\text{.}\) Then, \(\vec{u} = a \vec{d}, \vec{v} = b \vec{d}\) for some scalars \(a, b \in \mathbb{R}\text{.}\) Then, \(\vec{u} + \vec{v} = a\vec{d} + b\vec{d} = (a + b) \vec{d}\) is also a scalar multiple of \(\vec{d}\text{,}\) and so is also in \(U\text{.}\) Also, for any scalar \(c\text{,}\) \(c\vec{v} = (cb) \vec{v}\) is also a scalar multiple of \(\vec{d}\text{,}\) and so is in \(U\text{.}\) Thus, \(U\) is a vector space.
Next, suppose that \(U\) contains another vector which is not parallel to \(\vec{d}\text{,}\) say these two vectors are now called \(\vec{v}_1\) and \(\vec{v}_2\text{.}\) Then, their span is a vector space. In this case, it makes up the entire plane \(\mathbb{R}^2\text{.}\)
In summary, the possible subspaces of \(\mathbb{R}^2\) are a single point (the origin), a line through the origin, and the entire plane.
Example 6.1.6. Subspaces of \(\mathbb{R}^3\).
The possible subspaces of \(\mathbb{R}^3\) are a single point (the origin), a line through the origin, a plane through the origin, or the entire 3D space.
Subsection 6.1.3 Spans of Vectors as a Subspace
Almost by definition, the span of vectors in \(\mathbb{R}^n\) is a subspace of \(\mathbb{R}^n\text{.}\) Let \(\vec{v}_1, \vec{v}_2 \in \mathbb{R}^n\text{.}\) Then, consider \(H = \Span{\vec{v}_1, \vec{v}_2}\text{.}\) For \(\vec{u}, \vec{v} \in H\text{,}\) \(\vec{u} = s_1 \vec{v}_1 + s_2 \vec{v}_2\) and \(\vec{v} = t_1 \vec{v}_1 + t_2 \vec{v}_2\) for some scalars \(s_1, s_2, t_1, t_2\text{.}\) Then,
\begin{equation*}
\vec{u} + \vec{v} = (s_1 + t_1) \vec{v}_1 + (s_2 + t_2) \vec{v}_2
\end{equation*}
so \(\vec{u} + \vec{v}\) is a linear combination of \(\vec{v}_1, \vec{v}_2\text{,}\) and so \(\vec{u} + \vec{v} \in H\text{.}\) Also, for a scalar \(c \in \mathbb{R}\text{,}\) \(c\vec{u} = c(s_1 \vec{v}_1 + s_2 \vec{v}) = (cs_1) \vec{v}_1 + (cs_2) \vec{v}_2\text{,}\) and so \(c\vec{u} \in H\) also.
More generally,
Definition 6.1.7.
For a collection of vectors \(\vec{v}_1, \dots, \vec{v}_p \in \mathbb{R}^n\text{,}\) their span \(\Span{\vec{v}_1, \dots, \vec{v}_p}\) is a subspace of \(\mathbb{R}^n\text{.}\) Then, \(\Span{\vec{v}_1, \dots, \vec{v}_p}\) is called the subspace spanned by \(\vec{v}_1, \dots, \vec{v}_p\text{.}\)Conversely, if \(H = \Span{\vec{v}_1, \dots, \vec{v}_p}\text{,}\) then the subspace \(H\) is spanned by \(\vec{v}_1, \dots, \vec{v}_p\text{.}\)
Also, the subspace \(\Span{\vec{v}_1, \dots, \vec{v}_p}\) is the smallest subspace of \(\mathbb{R}^n\) which contains the vectors \(\vec{v}_1, \dots, \vec{v}_p\text{.}\) Intuitively, if \(H\) is a subspace that contains \(\vec{v}_1, \dots, \vec{v}_p\text{,}\) then at minimum, \(H\) must contain all of the linear combinations of these vectors. If we stop there, we form precisely the span of these vectors.
Subsection 6.1.4 Vector Spaces of \(\mathbb{R}^n\)
Every vector space (and subspace) must contain the zero vector \(\vec{0}\text{.}\)
Subsection 6.1.5 Intersection of Subspaces is a Subspace
Subsection 6.1.6 The Column Space
Subspaces often arise in connection with some matrix \(A\text{.}\) The two most important ways to construct subspaces are called the column space and null space, of a matrix.
Definition 6.1.8.
The column space of a matrix \(A\) is the set of all linear combinations of the column vectors of \(A\text{.}\) More precisely, if \(A\) is an \(m \times n\) matrix, \(A = \begin{bmatrix} \vec{a}_1 \amp \dots \amp \vec{a}_n \end{bmatrix}\text{,}\) then the column space, denoted by \(C(A)\text{,}\) is defined by,
\begin{equation*}
\boxed{C(A) = \Span{\vec{a}_1, \dots, \vec{a}_n}}
\end{equation*}
The column space of an \(m \times n\) matrix \(A\) is a subspace of \(\mathbb{R}^m\) (since the column vectors are vectors in \(\mathbb{R}^m\)).
Recall that \(\vec{b}\) is in the span of \(\vec{a}_1, \dots, \vec{a}_n\) if and only if the linear system \(A\vec{x} = \vec{b}\) has a solution. Then, the column space is equiavlently the set of all vectors \(\vec{b}\) such that \(A\vec{x} = \vec{b}\) has a solution. Conversely, \(A\vec{x} = \vec{b}\) has a solution precisely when \(\vec{b}\) is in the column space of \(A\text{.}\)
Subsection 6.1.7 The Null Space
Definition 6.1.9.
Let \(A\) be an \(n \times m\) matrix. Then, the null space of \(A\text{,}\) \(N(A)\text{,}\) is the set of all solutions to the homogeneous system \(A\vec{x} = \vec{0}\text{.}\) In other words, it is the set of all vectors \(\vec{x}\) such that \(A\vec{x} = \vec{0}\text{,}\) or
\begin{equation*}
N(A) = \set{\vec{x} \in \mathbb{R}^n : A\vec{x} = \vec{0}}
\end{equation*}
Certainly, the zero vector \(\vec{0} \in N(A)\text{,}\) because \(A\vec{0} = \vec{0}\) for any matrix \(A\text{.}\)
Theorem 6.1.10.
The null space is a subspace. More precisely, if \(A\) is an \(n \times m\) matrix, then \(N(A)\) is a subspace of \(\mathbb{R}^n\text{.}\)
Proof.
Let \(\vec{x}, \vec{y} \in N(A), c \in \mathbb{R}\text{.}\) Then, \(A\vec{x} = \vec{0}\) and \(A\vec{y} = \vec{0}\text{.}\) Then,
\begin{align*}
A(\vec{x} + \vec{y}) \amp = A\vec{x} + A\vec{y}\\
\amp = \vec{0} + \vec{0}\\
\amp = \vec{0}
\end{align*}
and so \(\vec{x} + \vec{y} \in N(A)\text{.}\) Also,
\begin{align*}
A(c\vec{x}) \amp = c(A\vec{x})\\
\amp = c \cdot \vec{0}\\
\amp = \vec{0}
\end{align*}
and so \(c\vec{x} \in N(A)\text{.}\) Thus, \(N(A)\) is a subspace.
The null space is special, because for a general matrix \(A\) and vector \(\vec{b}\text{,}\) the set of all solutions \(\vec{x}\) such that \(A\vec{x} = \vec{b}\) is not a vector space (i.e. if \(\vec{b} \neq \vec{0}\)). For example, because \(\vec{0}\) is not a solution and so is not in the solution set. Graphically, the solution set may be a line or plane which does not go through the origin.
Subsection 6.1.8 Contrasting the Column and Null Space
The most basic difference between the column space and null space is that the column space is a set of output vectors in \(\mathbb{R}^m\text{,}\) whereas the null space is a set of input vectors in \(\mathbb{R}^n\text{.}\)
The null space is defined by specifying a condition for which its vectors must satisfy. On the other hand, the column space is defined by specifying how to construct each vector by linear combinations. For this reason, the null space is said to be defined implicitly, whereas the column space is defined explicitly.
Because the column space is defined explicitly, it is easy to construct elements of the column space, simply by specifing values for the coefficients. To determine an explicit description of \(N(A)\text{,}\) solve the homogeneous system \(A\vec{x} = \vec{0}\text{,}\) and write the solution set in parametric vector form. Then, recall that the solution set to a homogeneous system is always a span of vectors. Then, the null space can always be written explicitly as the span of some vectors.
On the other hand, because the null space is defined implicitly by a condition, it is easy to verify if a given vector \(\vec{x}\) is in the null space \(N(A)\text{,}\) simply by computing \(A\vec{x}\) and checking whether or not it is equal to the zero vector. To determine if \(\vec{b} \in C(A)\text{,}\) we have to consider the system of equations \(A\vec{x} = \vec{b}\text{,}\) and determine if there is a solution or not.
