12) Markov Chain

Markov Chains Explained

Markov Chains Explained

1. What is a Markov Chain?

A Markov Chain is a mathematical system that undergoes transitions from one state to another based on certain probabilities. Its key property, known as the Markov Property, is that the future state depends only on the current state, not on the sequence of events that preceded it.

Real-World Examples:

  • Weather Prediction: Tomorrow’s weather depends only on today’s, not on the entire past week.
  • Stock Market Trends: A stock's price tomorrow depends on today's market condition.
  • Google PageRank: The probability of moving from one webpage to another depends on the current browsing behavior.

2. Components of a Markov Chain

A Markov Chain consists of the following key components:

  • States: The possible conditions or positions of the system (e.g., sunny, cloudy, rainy).
  • Transition Probabilities: The probabilities of moving from one state to another.
  • State Transition Matrix: A matrix representing all these transition probabilities.

Real-World Meaning: In customer behavior modeling, states could be "Browsing," "Adding to Cart," "Purchasing," etc., with probabilities assigned to moving from one state to another.

3. Transition Matrix (P)

The transition matrix stores the probabilities of moving from one state to another. For a system with states A, B, and C, it might look like:

P = [ [P(A→A), P(A→B), P(A→C)]
[P(B→A), P(B→B), P(B→C)]
[P(C→A), P(C→B), P(C→C)] ]

Each row sums to 1 because the system must transition to some state.

Real-World Meaning: In speech recognition, this matrix represents how likely one word is to follow another.

4. Types of Markov Chains

  • Absorbing Markov Chain: Contains at least one state that, once entered, cannot be left (e.g., permanently unsubscribing from a service).
  • Ergodic Markov Chain: Every state can eventually be reached from every other state (e.g., long-term stock price movements).

Real-World Meaning: In customer retention, an absorbing state might represent a customer permanently leaving, while an ergodic state implies they could eventually return.

5. Steady-State Probability

Over time, the probabilities of being in each state tend to stabilize. This is expressed as:

πP = π

where π is the steady-state vector.

Real-World Meaning: Google's PageRank is based on the steady-state probabilities of a random surfer visiting webpages.

6. First Passage Time & Hitting Probability

First Passage Time: The expected number of steps before a certain state is reached.

Hitting Probability: The probability of eventually reaching a particular state.

Real-World Meaning: In disease spread modeling, these metrics can predict how long before a disease reaches a specific region, or in board games, the number of moves to reach a winning position.

7. Hidden Markov Models (HMMs)

In a Hidden Markov Model, the states are not directly observed but are inferred from observable events.

Real-World Meaning: In speech recognition, the actual words (states) are inferred from audio signals (observations). In finance, the true market state (bullish/bearish) is inferred from stock prices.

8. Summary of Real-World Applications

  • AI & Machine Learning: Speech recognition, natural language processing, and predictive modeling.
  • Finance: Modeling stock price movements and risk analysis.
  • Biology: DNA sequence analysis and disease spread modeling.
    • \n
  • Marketing: Predicting customer behavior and retention strategies.
    • \n
\n
\n
\n\n"

Comments

Post a Comment

Popular posts from this blog

4) Vector Spaces

Vector Spaces with Real-World Context Vector Spaces with Real-World Context 1. Definition of a Vector Space A vector space is a set of vectors that can be added together and multiplied by scalars, following specific rules (axioms). These properties allow vectors to be manipulated in a consistent way. Real-World Example: Geometric Space: Any position or direction in a 2D or 3D world can be represented as a vector. Function Spaces: The set of all polynomials with real coefficients forms a vector space. 2. Properties of a Vector Space (Axioms) Addition Properties: u + v = v + u (Commutativity) (u + v) + w = u + (v + w) (Associativity) v + 0 = v (Zero Vector Exists) v + (-v) = 0 (Additive Inverse Exists) Scalar Multiplication Properties: c (d v) = (cd) v (Associativity of Scalar Multiplication) c (u + v) = c u + c v (Distributivity of Scalar over Vector Add...
Read more

7) Transformation Over Time

Transformations Over Time Transformations Over Time 1. What Does “Over Time” Mean? When we apply a transformation repeatedly or continuously over time, the result is a dynamic system . The transformation evolves based on time ( t ) or steps ( n ). Example: Rotating a shape frame by frame in an animation. Example: A car moving along a curved path using translations and rotations. 2. Linear Transformations Over Time A linear transformation over time follows the equation: v t+1 = A v t where: v t is the vector at time t . A is the transformation matrix. v t+1 is the transformed vector at t+1 . Example: Scaling Over Time A = [ 2 0 ] [ 0 2 ] Applying it repeatedly leads to exponential growth : v1 = A v0, v2 = A v1, v3 = A v2 3. Continuous Transformations (Differential Equations) Some transformations happen smoothly over time and are described by differential equations : d v / dt = ...
Read more

2) Prerequisites for understanding vectors

Prerequisites for Understanding Vectors (with Real-World Meaning) Prerequisites for Understanding Vectors 1. Numbers and Arithmetic Natural Numbers (ℕ): Counting numbers (1, 2, 3, ...). Real-World Meaning: Used in everyday counting—like the number of books on a shelf or apples in a basket. Integers (ℤ): Includes negative numbers (-2, -1, 0, 1, 2, ...). Real-World Meaning: Represent situations like temperatures (below zero), financial debts, or elevations above and below sea level. Rational Numbers (ℚ): Fractions and decimals. Real-World Meaning: Used in measurements and everyday situations like recipes or sharing a pizza equally. Real Numbers (ℝ): Continuous values including π and √2. Real-World Meaning:...
Read more