10) Single Value Decomposition

Singular Value Decomposition (SVD)

Singular Value Decomposition (SVD)

1. What is SVD?

Singular Value Decomposition (SVD) is a matrix factorization technique that breaks down a matrix into three special matrices:

A = U Σ Vᵀ
  • U: Orthogonal matrix (left singular vectors)
  • Σ: Diagonal matrix (singular values)
  • Vᵀ: Orthogonal matrix (right singular vectors)

Real-World Meaning: SVD extracts important features from data and is widely used in machine learning, image processing, and recommendation systems.

2. Structure of SVD Components

If A is an (m × n) matrix:

  • U (m × m): Represents variance in row space.
  • Σ (m × n): Contains singular values (importance of components).
  • Vᵀ (n × n): Represents variance in column space.

Real-World Meaning: Like breaking down an image into fundamental parts where some details are more important than others.

3. How SVD Works (Step-by-Step Algorithm)

  1. Compute Aᵀ A and A Aᵀ to find eigenvalues and eigenvectors.
  2. Compute singular values as the square roots of eigenvalues.
  3. Construct U, Σ, Vᵀ using eigenvectors and singular values.

Real-World Meaning: Similar to compressing an image by keeping the most important features and discarding the less important ones.

4. Applications of SVD

  • Image Compression: High singular values store the most important image details.
  • Recommendation Systems: Helps identify hidden patterns in user preferences.
  • Noise Reduction: Filtering out small singular values removes noise.
  • Dimensionality Reduction (LSA in NLP): Helps in text processing and understanding document similarities.

5. Rank Approximation & Data Compression

  • Low-rank approximation: Keeping only top singular values approximates the original data.
  • Compression: Reduces data size while maintaining quality.

Real-World Meaning: Like reducing an image's resolution while keeping it recognizable.

Conclusion

SVD is a powerful tool for data compression, noise reduction, and feature extraction. It makes large datasets manageable while preserving important patterns.

Comments

Post a Comment

Popular posts from this blog

4) Vector Spaces

Vector Spaces with Real-World Context Vector Spaces with Real-World Context 1. Definition of a Vector Space A vector space is a set of vectors that can be added together and multiplied by scalars, following specific rules (axioms). These properties allow vectors to be manipulated in a consistent way. Real-World Example: Geometric Space: Any position or direction in a 2D or 3D world can be represented as a vector. Function Spaces: The set of all polynomials with real coefficients forms a vector space. 2. Properties of a Vector Space (Axioms) Addition Properties: u + v = v + u (Commutativity) (u + v) + w = u + (v + w) (Associativity) v + 0 = v (Zero Vector Exists) v + (-v) = 0 (Additive Inverse Exists) Scalar Multiplication Properties: c (d v) = (cd) v (Associativity of Scalar Multiplication) c (u + v) = c u + c v (Distributivity of Scalar over Vector Add...
Read more

7) Transformation Over Time

Transformations Over Time Transformations Over Time 1. What Does “Over Time” Mean? When we apply a transformation repeatedly or continuously over time, the result is a dynamic system . The transformation evolves based on time ( t ) or steps ( n ). Example: Rotating a shape frame by frame in an animation. Example: A car moving along a curved path using translations and rotations. 2. Linear Transformations Over Time A linear transformation over time follows the equation: v t+1 = A v t where: v t is the vector at time t . A is the transformation matrix. v t+1 is the transformed vector at t+1 . Example: Scaling Over Time A = [ 2 0 ] [ 0 2 ] Applying it repeatedly leads to exponential growth : v1 = A v0, v2 = A v1, v3 = A v2 3. Continuous Transformations (Differential Equations) Some transformations happen smoothly over time and are described by differential equations : d v / dt = ...
Read more

2) Prerequisites for understanding vectors

Prerequisites for Understanding Vectors (with Real-World Meaning) Prerequisites for Understanding Vectors 1. Numbers and Arithmetic Natural Numbers (ℕ): Counting numbers (1, 2, 3, ...). Real-World Meaning: Used in everyday counting—like the number of books on a shelf or apples in a basket. Integers (ℤ): Includes negative numbers (-2, -1, 0, 1, 2, ...). Real-World Meaning: Represent situations like temperatures (below zero), financial debts, or elevations above and below sea level. Rational Numbers (ℚ): Fractions and decimals. Real-World Meaning: Used in measurements and everyday situations like recipes or sharing a pizza equally. Real Numbers (ℝ): Continuous values including π and √2. Real-World Meaning:...
Read more