When the cosine similarity between two vectors equals zero, those vectors share no linear relationship and form a 90 degree angle. In linear algebra, that property is called orthogonality, and it becomes the foundation for building independent, efficient, and stable machine learning models.

If you work with data, understanding orthogonal and orthonormal vectors helps you remove redundancy, speed up algorithms, and unlock advanced techniques like PCA. Here is how the concept works and how to verify it with NumPy.

Why does orthogonality matter in machine learning?

Think of giving directions on a map using north-south and east-west axes. Walking ten steps north does not change your east-west position at all. That is orthogonality in action: each axis carries new, non-redundant information [00:20].

This independence pays off in three concrete ways:

• Removes redundancy: when features are orthogonal, each one contributes new information, so models learn more efficiently.
• Adds stability and speed: math operations on orthogonal vectors are simpler and faster, making algorithms more robust.
• Enables PCA: Principal Component Analysis transforms correlated features into a new set of orthogonal features that capture the essence of your data [00:55].

What does it mean for two vectors to be orthogonal? It means their dot product equals zero and they form a 90 degree angle. Geometrically, neither vector projects onto the other.

How many orthogonal vectors fit in a space?

A key rule: an n-dimensional space (Rⁿ) holds at most n mutually orthogonal vectors. In R² (a plane) you get two, like the X and Y axes. In R³ you get three, like the corner of a room where X, Y, and Z meet [01:30]. You cannot squeeze in one more without breaking the perpendicularity.

What is the difference between orthogonal and orthonormal vectors?

A set of vectors is orthonormal when it satisfies two conditions at the same time:

1. Every pair is orthogonal (their dot product is zero).
2. Each vector has a norm of one, meaning they are unit vectors.

The standard basis vectors î = (1, 0) and ĵ = (0, 1) are the textbook example. Their dot product is zero and both measure one unit long [02:10]. Together they form an orthonormal basis you can combine to build any other vector in the plane.

What is an orthonormal basis? A set of mutually orthogonal unit vectors that can reconstruct any vector in the space through linear combinations.

How do I check orthogonality and orthonormality with NumPy?

Let's translate the theory into Python. Start by importing NumPy and defining the standard basis vectors:

python import numpy as np

i = np.array([1, 0]) j = np.array([0, 1])

print(f"Dot product i·j: {i @ j}")

The @ operator computes the dot product. The result is zero, confirming orthogonality [03:20]. To verify the norm:

python norm_i = np.linalg.norm(i) norm_j = np.linalg.norm(j) print(f"Norm i: {norm_i}, Norm j: {norm_j}")

Both norms equal one, so î and ĵ are orthonormal.

Can the standard basis reconstruct any vector?

Yes, and that is the whole point of a basis. Take a vector v = (5, 3) and rebuild it as a linear combination of î and ĵ:

python v = np.array([5, 3]) v_reconstructed = 5 * i + 3 * j print(f"Can the standard basis build v? {np.array_equal(v, v_reconstructed)}")

The output is True, proving that any 2D vector lives inside the span of î and ĵ [04:40].

How do I normalize vectors that are orthogonal but not unit length?

Now create vectors that are orthogonal but not orthonormal. Take v1 = (4, -2) and v2 = (2, 4). Their dot product is zero, so they are orthogonal. But their norms are not one, so they fail the orthonormal test [06:00].

A third vector v3 = (3, 3) gives a dot product of 6 with v1, confirming it adds redundant information.

To turn v1 and v2 into an orthonormal pair, divide each one by its norm:

python v1 = np.array([4, -2]) v2 = np.array([2, 4])

v1_normalized = v1 / np.linalg.norm(v1) v2_normalized = v2 / np.linalg.norm(v2)

print(f"Dot product: {v1_normalized @ v2_normalized:.2f}") print(f"Norm v1: {np.linalg.norm(v1_normalized):.2f}") print(f"Norm v2: {np.linalg.norm(v2_normalized):.2f}")

The dot product stays at zero and both norms now equal one. You have just built a custom orthonormal basis [07:30].

How can I visualize orthonormal vectors?

Matplotlib lets you confirm the geometry visually with plt.quiver:

python import matplotlib.pyplot as plt

plt.figure(figsize=(6, 6)) plt.quiver(0, 0, v1_normalized[0], v1_normalized[1], angles='xy', scale_units='xy', scale=1, color='blue', label='v1 normalized') plt.quiver(0, 0, v2_normalized[0], v2_normalized[1], angles='xy', scale_units='xy', scale=1, color='red', label='v2 normalized') plt.xlim(-5, 5); plt.ylim(-5, 5) plt.title('Orthonormal vectors visualization') plt.legend() plt.grid(True) plt.show()

The arrows look small because their length is exactly one, but the 90 degree angle between them is clear [09:30].

Your turn: practice with your own vectors

Create two 2D vectors you believe are orthogonal. Use NumPy to verify your intuition with the dot product, then normalize them and share your findings in the comments. With this toolkit you can already add and scale vectors, measure magnitudes with the norm, and test independence with the dot product.