Orthogonal Projection Finds the Best Approximation

When a system of equations has no solution, linear algebra offers an elegant workaround: find the closest possible answer. That is exactly what orthogonal projection does, and you will use it whenever a vector b sits outside the column space of a matrix A.

What is orthogonal projection and why does it matter?

Think of projection as a shadow. If you shine a light perpendicular to a line defined by vector a, the shadow that vector b casts on that line is its projection. That shadow is the point on the line closest to b, which makes it the best possible approximation when an exact solution does not exist.

This idea connects directly to the problem you saw before: in Ax = b, if b lives outside the column space of A, you cannot build it from the columns of A. But you can build the vector inside that space that gets you as close as possible. That vector is the projection.

What is orthogonal projection? It is the operation that finds the closest point on a line or subspace to a given vector, by dropping a perpendicular shadow onto it.

How do you calculate the projection of b onto a?

The formula uses tools you already know from the dot product and the norm [01:32]:

• Compute the dot product between b and a.
• Divide by the squared norm of a.
• Multiply that scalar by the vector a.

In symbols, the projection of b onto a equals (b · a) / ||a||² times a. The fraction tells you how much of a you need; multiplying by a gives you the final shadow vector.

Worked example with b = (2, 3) and a = (4, 0)

Let's run the numbers step by step [02:15]:

• Dot product: 2·4 + 3·0 = 8.
• Squared norm of a: √16 squared cancels back to 16.
• Scalar: 8/16 = 1/2.
• Final projection: (1/2)·(4, 0) = (2, 0).

Graphically, vector a lies flat on the x axis, b points up and to the right at (2, 3), and the shadow lands exactly at (2, 0). That point is the closest you can get to b while staying on the line defined by a.

How does projection solve a system without a solution?

Here is where projection becomes a problem solver. Take the inconsistent system from the previous class: x + y = 2 and x + y = 4. The coefficient matrix A has columns (1, 1) and (1, 1), so its column space is just the line through (1, 1). The result vector B = (2, 4) does not live on that line, so there is no exact answer.

The fix: project B onto A [05:48].

• Dot product of B and A: 2·1 + 4·1 = 6.
• Squared norm of A: (√2)² = 2.
• Scalar: 6/2 = 3.
• Projection: 3·(1, 1) = (3, 3).

When you plot this, the column space of A is the diagonal line y = x, and (3, 3) sits right on it. It is the closest point on that line to B = (2, 4). That point is your best approximate solution.

Why is the projection the best approximation? Because it is the unique point inside the column space whose distance to b is the smallest possible, thanks to the perpendicular drop.

Key concepts you just used

A few ideas are doing the heavy lifting here, and it helps to name them clearly.

• Column space of A: the set of all vectors you can build by combining the columns of A. If b is not in it, Ax = b has no exact solution [04:58].
• Dot product: measures alignment between two vectors and powers the projection formula [02:30].
• Norm squared: the length of a vector squared, which scales the projection correctly [02:50].
• Best approximation: the projected vector, which minimizes the distance between b and the column space of A.

Practice exercise

Try this on your own. Given vector V = (3, 0) and vector U = (1, 1), calculate the projection of U onto V. Share the resulting vector in the comments and compare your answer with others.

You just used projection to find one closest vector. But what if you wanted to take an entire basis of vectors and turn it into a clean, fully orthogonal basis? That is the next step, and it builds directly on what you did here.