15) Vector Norms

Vector Norms

Vector Norms: Measuring Vector Magnitude

1. What is a Vector Norm?

A vector norm is a function that assigns a positive length to a vector, measuring its size.

Real-World Meaning

  • Physics: The norm of velocity gives the speed of an object.
  • Machine Learning: Norms measure distances between data points.
  • Optimization: Used in gradient descent to control updates.

2. Types of Vector Norms

(a) Euclidean Norm (L2 Norm)

Measures straight-line distance.

||v||₂ = sqrt(v₁² + v₂² + ... + vₙ²)

Real-World Use:

  • GPS Navigation: Finding shortest distance.
  • Physics: Computing magnitudes of forces.

(b) Manhattan Norm (L1 Norm)

Sum of absolute values of vector components.

||v||₁ = |v₁| + |v₂| + ... + |vₙ|

Real-World Use:

  • City Blocks: Distance in grid-like street layouts.
  • AI: Used in Lasso Regression for feature selection.

(c) Max Norm (L∞ Norm)

Largest absolute value of any vector component.

||v||∞ = max(|v₁|, |v₂|, ..., |vₙ|)

Real-World Use:

  • Quality Control: Finding maximum deviation.
  • Robotics: Avoiding worst-case movement errors.

(d) p-Norm (Lp Norm)

Generalized norm formula.

||v||ₚ = (∑ |vᵢ|ᵖ)^(1/p)

Real-World Use:

  • Deep Learning: Used in error functions.
  • Image Processing: Measures pixel differences.

3. Properties of Norms

  • Non-negativity: ||v|| ≥ 0
  • Zero Vector Property: ||v|| = 0 ⇔ v = 0
  • Triangle Inequality: ||u + v|| ≤ ||u|| + ||v||
  • Scaling: ||c v|| = |c| ||v||

Real-World Meaning:

  • Triangle inequality ensures the shortest path is direct.
  • Scaling means doubling force doubles its effect.

4. Choosing the Right Norm

  • L2 norm: Used when true distance matters.
  • L1 norm: Used in grid-like systems.
  • L∞ norm: Used when worst-case scenarios matter.
  • Lp norm: Used in generalized settings.

Comments

Post a Comment

Popular posts from this blog

4) Vector Spaces

Vector Spaces with Real-World Context Vector Spaces with Real-World Context 1. Definition of a Vector Space A vector space is a set of vectors that can be added together and multiplied by scalars, following specific rules (axioms). These properties allow vectors to be manipulated in a consistent way. Real-World Example: Geometric Space: Any position or direction in a 2D or 3D world can be represented as a vector. Function Spaces: The set of all polynomials with real coefficients forms a vector space. 2. Properties of a Vector Space (Axioms) Addition Properties: u + v = v + u (Commutativity) (u + v) + w = u + (v + w) (Associativity) v + 0 = v (Zero Vector Exists) v + (-v) = 0 (Additive Inverse Exists) Scalar Multiplication Properties: c (d v) = (cd) v (Associativity of Scalar Multiplication) c (u + v) = c u + c v (Distributivity of Scalar over Vector Add...
Read more

7) Transformation Over Time

Transformations Over Time Transformations Over Time 1. What Does “Over Time” Mean? When we apply a transformation repeatedly or continuously over time, the result is a dynamic system . The transformation evolves based on time ( t ) or steps ( n ). Example: Rotating a shape frame by frame in an animation. Example: A car moving along a curved path using translations and rotations. 2. Linear Transformations Over Time A linear transformation over time follows the equation: v t+1 = A v t where: v t is the vector at time t . A is the transformation matrix. v t+1 is the transformed vector at t+1 . Example: Scaling Over Time A = [ 2 0 ] [ 0 2 ] Applying it repeatedly leads to exponential growth : v1 = A v0, v2 = A v1, v3 = A v2 3. Continuous Transformations (Differential Equations) Some transformations happen smoothly over time and are described by differential equations : d v / dt = ...
Read more

2) Prerequisites for understanding vectors

Prerequisites for Understanding Vectors (with Real-World Meaning) Prerequisites for Understanding Vectors 1. Numbers and Arithmetic Natural Numbers (ℕ): Counting numbers (1, 2, 3, ...). Real-World Meaning: Used in everyday counting—like the number of books on a shelf or apples in a basket. Integers (ℤ): Includes negative numbers (-2, -1, 0, 1, 2, ...). Real-World Meaning: Represent situations like temperatures (below zero), financial debts, or elevations above and below sea level. Rational Numbers (ℚ): Fractions and decimals. Real-World Meaning: Used in measurements and everyday situations like recipes or sharing a pizza equally. Real Numbers (ℝ): Continuous values including π and √2. Real-World Meaning:...
Read more