BERT Masked Language Modeling

Interactive exploration of BERT's masked token prediction with attention visualization

Model Configuration

BERT Model:

Prediction Settings

Top-K Predictions: 5 Minimum Probability: 0.01

Show Probabilities Show as Percentages Auto-predict on mask

Example Sentences

How to Use

Click a word to mask it and see predictions
Multiple masks: Click multiple words for simultaneous prediction
View attention: See how BERT attends to each token
Probabilities: Higher bars = more confident predictions
Load examples: Click example buttons to try different sentences

About BERT MLM

BERT (Bidirectional Encoder Representations from Transformers) is pre-trained using masked language modeling. It predicts masked tokens based on bidirectional context.

Input Text

Type or paste your text below, then click words to mask them:

Interactive Text

Click on any word to mask it:

Process text to begin...

Statistics

Tokens: 0

Masked: 0

Predictions: 0

Avg. Confidence: 0%

BERT Masked Language Modeling

Input Text

Interactive Text

Predictions

Attention Visualization

Statistics