Section 3.3 The Dot Product
Subsection 3.3.1 Vector Direction
Recall that every vector is associated with a unit vector. Slightly more generally, every direction is associcated with a unit vector. Further, every unit vector is associated with a particular direction, or angle. If \(\vec{u} = (a,b)\) is a unit vector, its terminal point is on the unit circle. Then, \(\vec{u} = (\cos{\theta}, \sin{\theta})\) where \(\theta\) is the angle from the direction of the positive x-axis to the direction of \(\vec{u}\text{.}\)
Then, a general vector \(\vec{v}\) can be written as,
\begin{equation*}
\vec{v} = \abs{\vec{v}} \vec{u}
\end{equation*}
where \(\vec{u}\) is the unit vector in the direction of \(\vec{v}\text{.}\) Alternatively,
\begin{equation*}
(x,y) = r (\cos{\theta}, \sin{\theta})
\end{equation*}
where \(r = \sqrt{x^2 + y^2}\) is the distance from \((x,y)\) to the origin, and \(\theta\) is the angle from the direction of the positive \(x\)-axis to the direction of \(\vec{v}\text{.}\) This represents the vector \((x,y)\) in polar coordinates. This may be familiar from pre-calculus or calculus.
We want to measure the degree to which two vectors point in the same direction, i.e. the degree to which they are parallel. Let \(\vec{u}, \vec{v}\) be vectors, given by \(\vec{u} = (u_1, u_2), \vec{v} = (v_1, v_2)\text{,}\) with representation,
\begin{align*}
\vec{u} \amp = r (\cos{\alpha}, \sin{\alpha})\\
\vec{v} \amp = l (\cos{\beta}, \sin{\beta})
\end{align*}
where \(r = \sqrt{u_x^2 + u_y^2}, l = \sqrt{v_x^2 + v_y^2}\text{,}\) and \(\alpha, \beta\) are their respective angles with the positive \(x\)-axis. Then, let \(\theta = \alpha - \beta\) be the angle between these vectors (without loss of generality, suppose that \(\alpha \geq \beta\)). Then, this angle can be at most \(180^{\circ}\text{,}\) or \(\pi\) radians.
Intuitively, if this angle is small, then the vectors are more parallel, and if this angle is close to \(90^{\circ}\text{,}\) then the vectors are less parallel. If the angle is close to \(180^{\circ}\text{,}\) then the vectors are close to being anti-parallel.
It turns out that we can use the cosine of the angle \(\theta\) to create a numerical measure of this parallel-ness. Here,
\begin{equation*}
\cos{\theta} = \cos{(\alpha - \beta)}
\end{equation*}
Then, using the cosine difference identity,
\begin{equation*}
= \cos{\alpha} \cos{\beta} + \sin{\alpha} \sin{\beta}
\end{equation*}
Then, recall that for the vectors, the cosines and sines of \(\alpha\) and \(\beta\) can be determined using the components of the vectors. In particular,
\begin{align*}
\amp = \frac{u_1}{r} \cdot \frac{v_1}{l} + \frac{u_2}{r} \cdot \frac{v_2}{l}\\
\cos{\theta} \amp = \frac{u_1 v_1 + u_2 v_2}{rl}
\end{align*}
Or, more explicitly,
\begin{equation*}
\cos{\theta} = \frac{u_1 v_1 + u_2 v_2}{\sqrt{u_1^2 + u_2^2} \cdot \sqrt{v_1^2 + v_2^2}}
\end{equation*}
This gives a measure of parallelness of \(\vec{u}\) and \(\vec{v}\text{,}\) in terms of their components. The numerator of this formula comes up often enough to be given its own name, called the dot product.
Subsection 3.3.2 Dot Product of Vectors
Here, we define the dot product of vectors. First, for vectors in \(\mathbb{R}^2\text{.}\)
Definition 3.3.1.
Let \(\vec{u} = (u_1, u_2), \vec{v} = (v_1, v_2) \in \mathbb{R}^2\) be vectors. Then, the dot product of \(\vec{u}\) and \(\vec{v}\text{,}\) denoted by \(\vec{u} \bullet \vec{v}\text{,}\) is the sum of the products of the corresponding components,
\begin{equation*}
\boxed{\vec{u} \bullet \vec{v} = u_1 v_1 + u_2 v_2}
\end{equation*}
The name “dot product” comes from the fact that the multiplication is denoted by a dot. The dot product is also called the scalar product, because the product produces a scalar value. It is also called the inner product, for more complicated reasons.
Subsection 3.3.3 Properties of the Dot Product
The dot product has some basic algebraic properties.
Theorem 3.3.2.
Let \(\vec{u}, \vec{v}, \vec{w} \in \mathbb{R}^2\) be vectors. Then,
Commutative property, \(\vec{u} \bullet \vec{v} = \vec{v} \bullet \vec{u}\text{.}\)
Distributive property, \(\vec{u} \bullet (\vec{v} + \vec{w}) = \vec{u} \bullet \vec{v} + \vec{u} \bullet \vec{w}\text{.}\)
Associativity with scalar multiplication, for
\(k \in \mathbb{R}\text{,}\)
\begin{equation*}
(k \vec{u}) \bullet \vec{v} = \vec{u} \bullet (k\vec{v}) = k(\vec{u} \bullet \vec{v})
\end{equation*}
Proof.
Each of these properties basically follows from the definition of a dot product, and the corresponding properties for real numbers. Let \(\vec{u} = (u_1, u_2), \vec{v} = (v_1, v_2), \vec{w} = (w_1, w_2)\text{.}\)
Then,
\begin{align*}
\vec{u} \bullet \vec{v} \amp = u_1 v_1 + u_2 v_2\\
\amp = v_1 u_1 + v_2 u_2\\
\amp = \vec{v} \bullet \vec{u}
\end{align*}
\begin{align*}
\vec{u} \bullet (\vec{v} + \vec{w}) \amp = (u_1, u_2) \bullet (v_1 + w_1, v_2 + w_2)\\
\amp = u_1(v_1 + w_1) + u_2(v_2 + w_2)\\
\amp = u_1 v_1 + u_1 w_1 + u_2 v_2 + u_2 w_2\\
\amp = \brac{u_1 v_1 + u_2 v_2} + \brac{u_1 w_1 + u_2 w_2}\\
\amp = \vec{u} \bullet \vec{v} + \vec{u} \bullet \vec{w}
\end{align*}
-
\begin{align*}
(k\vec{u}) \bullet \vec{v} \amp = (ku_1, ku_2) \bullet (v_1, v_2)\\
\amp = ku_1 v_1 + ku_2 v_2
\end{align*}
This expression is equal to,
\begin{equation*}
= k\brac{u_1 v_1 + u_2 v_2} = k (\vec{u} \bullet \vec{v})
\end{equation*}
This is also equal to,
\begin{equation*}
= u_1 (k v_1) + u_2 (k v_2) = \vec{u} \bullet (k\vec{v})
\end{equation*}
The dot product also relates to the magnitude of a vector. Consider the dot product of a vector \(\vec{u} = (u_1, u_2)\) with itself,
\begin{equation*}
\vec{u} \bullet \vec{u} = u_1 u_1 + u_2 u_2 = u_1^2 + u_2^2
\end{equation*}
This is precisely the square of the magnitude of \(\vec{u}\text{.}\) Thus,
\begin{equation*}
\boxed{\abs{\vec{u}} = \sqrt{\vec{u} \bullet \vec{u}}}
\end{equation*}
Subsection 3.3.4 Angle Between Vectors
Let \(\vec{v}, \vec{w}\) be non-zero vectors in \(\mathbb{R}^2\text{.}\) Then, positioning the vectors with a common tail, this determines a (unique) angle \(\theta\) between them. This is the smallest angle between the two vectors, which forces \(\theta\) to be in the range \(\theta \in [0,\pi]\text{.}\) This is called the angle between \(\vec{v}\) and \(\vec{w}\).
Subsection 3.3.5 Alternate Characterization for the Dot Product
Definition 3.3.3.
Let \(\vec{u}, \vec{v}\) be vectors, \(\theta\) be the angle between \(\vec{u}\) and \(\vec{v}\text{.}\) Then,
\begin{equation*}
\boxed{\vec{u} \bullet \vec{v} = \abs{\vec{u}} \abs{\vec{v}} \cos{\theta}}
\end{equation*}
This is an alternate characterization for the dot product, in that we could have instead defined the dot product using this formula, and then show that this definition implies the previous definition.
The proof of this fact follows from the geometry of triangles, in particular the law of cosines. Applying the law of cosines to a particular triangle of vectors shows the relationship between the dot product and the angle between vectors.
Proof.
From the vectors \(\vec{u}, \vec{v}\text{,}\) form a triangle by joining the heads of the two vectors to form the vector \(\vec{u} - \vec{v}\text{.}\) Then, by the law of cosines, we can relate the angle \(\theta\) between the vectors \(\vec{u}, \vec{v}\) with the lengths of the sides of this triangle (given by the magnitudes of the vectors involved). This gives,
\begin{equation*}
\abs{\vec{u}}^2 + \abs{\vec{v}}^2 - 2\abs{\vec{u}} \abs{\vec{v}} \cos{\theta} = \abs{\vec{u} - \vec{v}}^2
\end{equation*}
Then, expanding the right-hand side allows us to rewrite in terms of the dot product,
\begin{align*}
\abs{\vec{u} - \vec{v}}^2 \amp = (\vec{u} - \vec{v}) \bullet (\vec{u} - \vec{v})\\
\amp = (\vec{u} - \vec{v}) \bullet \vec{u} - (\vec{u} - \vec{v}) \bullet \vec{v} \amp\amp \text{by the distributive property}\\
\amp = \vec{u} \bullet \vec{u} - \vec{v} \bullet \vec{u} - \vec{u} \bullet \vec{v} + \vec{v} \bullet \vec{v}\\
\amp = \abs{\vec{u}}^2 + \abs{\vec{v}}^2 - 2\vec{u} \bullet \vec{v}
\end{align*}
Then,
\begin{align*}
\abs{\vec{u}}^2 + \abs{\vec{v}}^2 - 2\abs{\vec{u}} \abs{\vec{v}} \cos{\theta} \amp = \abs{\vec{u}}^2 + \abs{\vec{v}}^2 - 2\vec{u} \bullet \vec{v}\\
-2 \abs{\vec{u}} \abs{\vec{v}} \cos{\theta} \amp = -2\vec{u} \bullet \vec{v}\\
\vec{u} \bullet \vec{v} = \abs{\vec{u}} \abs{\vec{v}} \cos{\theta}
\end{align*}
as desired.
Subsection 3.3.6 Computing the Angle Between Vectors
The dot product can be written in terms of the cosine of the angle between two vectors, so it can be used to compute this angle.
Theorem 3.3.4. Angle between vectors.
Let \(\vec{u}, \vec{v} \in \mathbb{R}^2\) be non-zero vectors, \(\theta\) be the angle between them. Then,
\begin{equation*}
\theta = \arccos{\brac{\frac{\vec{u} \bullet \vec{v}}{\abs{\vec{u}} \abs{\vec{v}}}}}
\end{equation*}
Subsection 3.3.7 Dot Product and Parallel-ness
If the angle between \(\vec{u}\) and \(\vec{v}\) is a right angle, i.e. \(\theta = 90^{\circ}\text{,}\) then we say that \(\vec{u}\) and \(\vec{v}\) are perpndicular.
If \(\theta = 90^{\circ}\text{,}\) then the formula for the dot product becomes,
\begin{align*}
\vec{u} \bullet \vec{v} \amp = \abs{\vec{u}} \abs{\vec{v}} \underbrace{\cos{90^{\circ}}}_{0}\\
\amp = 0
\end{align*}
This provides a test to determine if \(\vec{u}\) and \(\vec{v}\) are perpendicular. That is, if \(\vec{u} \bullet \vec{v} = 0\text{.}\)
However, conversely, if \(\vec{u} \bullet \vec{v} = \abs{\vec{u}} \abs{\vec{v}} \cos{\theta} = 0\text{,}\) this implies that at least one of \(\abs{\vec{u}}, \abs{\vec{v}}, \cos{\theta}\) is equal to 0. If either of \(\abs{\vec{u}}, \abs{\vec{v}}\) are 0, then the corresponding vector is the zero vector. Recall that the zero vector is said to be perpendicular to all vectors (in fact, the reason for this convention is to make this theorem universal). In summary,
Theorem 3.3.5. Perpendicular if and only if zero dot product.
Vectors \(\vec{u}, \vec{v}\) are perpendicular if and only if \(\vec{u} \bullet \vec{v} = 0\text{.}\)
Subsection 3.3.8 Dot Product for Vectors in \(\mathbb{R}^3\)
The dot product for vectors in \(\mathbb{R}^3\) is defined in a similar way.
Definition 3.3.6.
Let \(\vec{u} = (u_1, u_2, u_3), \vec{v} = (v_1, v_2, v_3) \in \mathbb{R}^3\) be vectors. Then, the dot product of \(\vec{u}\) and \(\vec{v}\) is defined by,
\begin{equation*}
\boxed{\vec{u} \bullet \vec{v} = u_1 v_1 + u_2 v_2 + u_3 v_3}
\end{equation*}
This dot product shares many of the same properties. First, the algebraic properties.
Theorem 3.3.7.
Let \(\vec{u}, \vec{v}, \vec{w} \in \mathbb{R}^3\) be vectors. Then,
\(\vec{u} \bullet \vec{v} = \vec{v} \bullet \vec{u}\text{.}\)
\(\vec{u} \bullet (\vec{v} + \vec{w}) = \vec{u} \bullet \vec{v} + \vec{u} \bullet \vec{w}\text{.}\)
For
\(k \in \mathbb{R}\text{,}\)
\begin{equation*}
(k \vec{u}) \bullet \vec{v} = \vec{u} \bullet (k\vec{v}) = k(\vec{u} \bullet \vec{v})
\end{equation*}
\(\displaystyle \vec{u} \bullet \vec{u} = \abs{\vec{u}}^2\)
In addition, the angle properties of the dot product are still valid. However, it is more non-trivial to prove this result.
Theorem 3.3.8.
Let \(\vec{u}, \vec{v} \in \mathbb{R}^3\) be vectors, \(\theta\) be the angle between \(\vec{u}\) and \(\vec{v}\text{.}\) Then,
\begin{equation*}
\boxed{\vec{u} \bullet \vec{v} = \abs{\vec{u}} \abs{\vec{v}} \cos{\theta}}
\end{equation*}
And,
\begin{equation*}
\theta = \arccos{\brac{\frac{\vec{u} \bullet \vec{v}}{\abs{\vec{u}} \abs{\vec{v}}}}}
\end{equation*}
Subsection 3.3.9 Dot Product for Vectors in \(\mathbb{R}^n\) (Inner Product)
The dot product can be extended to \(\mathbb{R}^n\text{.}\) It is often called the inner product.
Definition 3.3.9.
Let \(\vec{u} = (u_1, \dots, u_n), \vec{v}_1 = (v_1, \dots, v_n) \in \mathbb{R}^n\) be vectors. Then, the dot product of \(\vec{u}\) and \(\vec{v}\) is defined by,
\begin{equation*}
\boxed{\vec{u} \bullet \vec{v} = u_1 v_1 + u_2 v_2 \dots + u_n v_n = \sum_{k=1}^n u_k v_k}
\end{equation*}
If \(\vec{u}, \vec{v}\) are vectors in \(\mathbb{R}^n\text{,}\) then recall that thtey are considered \(n \times 1\) matrices. Then, the transpose \(\vec{u}^T\) is a \(1 \times n\) matrix, and the matrix product \(\vec{u}^T \vec{v}\) is a \(1 \times 1\) matrix, whose entry is given by the dot product,
\begin{equation*}
\begin{bmatrix} u_1 \amp u_2 \amp \dots \amp u_n \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} = u_1 v_1 + u_2 v_2 + \dots + u_n v_n
\end{equation*}
Strictly speaking, the product is a \(1 \times 1\) matrix, however it is treated as a real number and written without brackets.
Then, the product \(\vec{u} \bullet \vec{v} = \vec{u}^T \vec{v}\) is called the inner product of \(\vec{u}\) and \(\vec{v}\text{.}\)
All of the properties of the dot product generalize for the inner product in \(\mathbb{R}^n\text{.}\)
Theorem 3.3.10.
Let \(\vec{u}, \vec{v}, \vec{w}\) be vectors in \(\mathbb{R}^n\text{.}\) Then,
Commutative property, \(\vec{u} \bullet \vec{v} = \vec{v} \bullet \vec{u}\text{.}\)
Distributive property, \(\vec{u} \bullet (\vec{v} + \vec{w}) = \vec{u} \bullet \vec{v} + \vec{u} \bullet \vec{w}\text{.}\)
Associativity with scalar multiplication, for
\(k \in \mathbb{R}\text{,}\)
\begin{equation*}
(k \vec{u}) \bullet \vec{v} = \vec{u} \bullet (k\vec{v}) = k(\vec{u} \bullet \vec{v})
\end{equation*}
Proof.
Let \(\vec{u} = (u_1, \dots, u_n), \vec{v} = (v_1, \dots, v_n), \vec{w} = (w_1, \dots, w_n)\text{.}\) Then, FINISH.
Also,
Theorem 3.3.11.
Let \(\vec{u} \in \mathbb{R}^n\text{.}\) Then, \(\vec{u} \bullet \vec{u} \geq 0\text{,}\) and \(\vec{u} \bullet \vec{u} = 0\) if and only if \(\vec{u} = 0\text{.}\)
Corollary 3.3.12.
In general,
\begin{equation*}
(a_1 \vec{v}_1 + \dots + a_p \vec{v}_p) \bullet \vec{w} = a_1 \vec{v}_1 \bullet \vec{w} + \dots + a_p \vec{v}_p \bullet \vec{w}
\end{equation*}
Subsection 3.3.10 Pythagorean Theorem
Theorem 3.3.13.
Two vectors \(\vec{u}, \vec{v}\) in \(\mathbb{R}^n\) are perpendicular if and only if \(\abs{\vec{u} + \vec{v}}^2 = \abs{\vec{u}}^2 + \abs{\vec{v}}^2\text{.}\)
Subsection 3.3.11 Angle Between Vectors
Recall that in \(\mathbb{R}^2\) and \(\mathbb{R}^3\text{,}\) the (geometric) angle between two vectors is given by the equation \(\abs{\vec{u}} \abs{\vec{v}} \cos{\theta} = \vec{u} \bullet \vec{v}\text{.}\) For higher dimensions, there is no physical notion of angle. However, that equation can be used to define the angle between \(\vec{u}, \vec{v} \in \mathbb{R}^n\text{.}\)
Definition 3.3.14. Angle between vectors in \(\mathbb{R}^n\).
\begin{equation*}
\theta = \arccos{\brac{\frac{\vec{x} \bullet \vec{y}}{\abs{\vec{x}} \abs{\vec{y}}}}}
\end{equation*}
In statistics, the value of \(\cos{\theta}\) is called the correlation coefficient, for suitable vectors \(\vec{u}, \vec{v}\text{.}\)
