Topic Modeling Studio

Topic Overview

How LDA Works

Understanding Latent Dirichlet Allocation

Graphical Model

M documents

θ Doc-Topic

N words

z Topic

w Word

φ Topic-Word

K topics

Key Concepts

α (Alpha)

Document-topic concentration. Lower α means documents focus on fewer topics. Higher α means documents spread across many topics.

β (Beta)

Topic-word concentration. Lower β means topics have distinct vocabulary. Higher β means topics share more words.

θ (Theta)

Per-document topic distribution. Each document has its own θ defining how much it belongs to each topic.

φ (Phi)

Per-topic word distribution. Each topic has its own φ defining probabilities over the vocabulary.

Generative Process

For each topic k

Draw word distribution φ_k ~ Dirichlet(β)

For each document d

Draw topic distribution θ_d ~ Dirichlet(α)

For each word n in document d

Draw topic z_d,n ~ Multinomial(θ_d)

Draw word w_d,n ~ Multinomial(φ_z)

Inference: Gibbs Sampling

Since we observe only words (shaded node), we need to infer the hidden topics. This demo uses Collapsed Gibbs Sampling:

Topic Assignment Update:

P(z_i = k | z_-i, w) ∝ (n_d,k^-i + α) × (n_k,w^-i + β) / (n_k^-i + Vβ)

Probability of assigning word i to topic k depends on: how much topic k appears in this document, and how often this word appears in topic k.

Statistics

Documents: 0

Vocabulary Size: 0

Topics: 0

Avg. Doc Length: 0

Perplexity: -