X-Hour 1: ELIZA Deep Dive¶

CS 165: Natural Language Processing
Week 1 - Thursday X-Hour


Learning Objectives¶

By the end of this session, you will:

  1. Understand how ELIZA's pattern matching works
  2. Implement and extend ELIZA's conversation engine
  3. Debug common issues in rule-based dialogue systems
  4. Explore the limitations and strengths of pattern-based NLP

Setup¶

This notebook is designed to run in Google Colab or any Jupyter environment. No external dependencies required!

Part 1: Understanding ELIZA's Core Mechanism¶

ELIZA (1966) by Joseph Weizenbaum was one of the first chatbots. It simulated a Rogerian psychotherapist using:

  • Pattern matching: Regular expressions to detect keywords and phrases
  • Transformation rules: Converting "I am" to "you are", etc.
  • Response templates: Canned responses with placeholders

Let's build this step by step.

In [ ]:
import re
import random
from typing import List, Tuple, Optional

Step 1: Word Reflections¶

ELIZA needs to swap pronouns when reflecting statements back to the user.

In [ ]:
# Reflection dictionary: swaps first-person to second-person and vice versa
REFLECTIONS = {
    "am": "are",
    "was": "were",
    "i": "you",
    "i'd": "you would",
    "i've": "you have",
    "i'll": "you will",
    "my": "your",
    "are": "am",
    "you've": "I have",
    "you'll": "I will",
    "your": "my",
    "yours": "mine",
    "you": "me",
    "me": "you",
    "myself": "yourself",
    "yourself": "myself"
}

def reflect(text: str) -> str:
    """
    Reflect pronouns in the text from first to second person and vice versa.
    
    Args:
        text: Input string to reflect
    
    Returns:
        Reflected string
    """
    words = text.lower().split()
    reflected_words = [REFLECTIONS.get(word, word) for word in words]
    return ' '.join(reflected_words)

# Test the reflection function
print("Original: 'I am sad'")
print(f"Reflected: '{reflect('I am sad')}'")
print()
print("Original: 'my mother thinks you are crazy'")
print(f"Reflected: '{reflect('my mother thinks you are crazy')}'")

💡 Exercise 1: Test Reflections¶

Try these inputs and predict the output before running:

  1. "I think my father hates me"
  2. "you are not listening to me"
  3. "I've been feeling lonely"
In [ ]:
# Your code here
test_sentences = [
    "I think my father hates me",
    "you are not listening to me",
    "I've been feeling lonely"
]

for sentence in test_sentences:
    print(f"Original: {sentence}")
    print(f"Reflected: {reflect(sentence)}")
    print()

Step 2: Pattern-Response Pairs¶

ELIZA's power comes from its pattern-response database. Each pattern has:

  • A regex to match user input
  • One or more response templates
  • A priority (some patterns are more specific than others)
In [ ]:
# Pattern-response database
# Format: (pattern, [responses])
# Use {0}, {1}, etc. for captured groups
PATTERNS = [
    # High priority: specific patterns
    (r'quit|exit|bye|goodbye',
     ["Goodbye. It was nice talking to you.",
      "Thank you for talking with me. Goodbye."]),
    
    (r'hello|hi|hey',
     ["Hello. How are you feeling today?",
      "Hi there. What brings you here today?"]),
    
    # Family patterns
    (r'.*\bmother\b.*',
     ["Tell me more about your mother.",
      "How is your relationship with your mother?",
      "What else comes to mind when you think of your mother?"]),
    
    (r'.*\bfather\b.*',
     ["Tell me more about your father.",
      "How do you feel about your father?",
      "What role does your father play in your life?"]),
    
    (r'.*\bfamily\b.*',
     ["Tell me more about your family.",
      "How is your relationship with your family?"]),
    
    # Emotion patterns with capture groups
    (r'i am (sad|depressed|unhappy).*',
     ["I'm sorry to hear that you are {0}. Can you tell me why?",
      "Why do you think you are {0}?",
      "How long have you been {0}?"]),
    
    (r'i am (happy|glad|excited).*',
     ["That's wonderful that you are {0}! What made you feel this way?",
      "It's great to hear you are {0}. Tell me more."]),
    
    (r'i (feel|felt) (.*)',
     ["Why do you feel {1}?",
      "Tell me more about feeling {1}.",
      "When did you start feeling {1}?"]),
    
    # "I am" patterns
    (r'i am (.*)',
     ["Why do you say you are {0}?",
      "How long have you been {0}?",
      "Do you believe it is normal to be {0}?",
      "Do you enjoy being {0}?"]),
    
    # "I want/need" patterns
    (r'i (want|need) (.*)',
     ["Why do you {0} {1}?",
      "What would it mean to you if you got {1}?",
      "What if you never got {1}?"]),
    
    # Dream patterns
    (r'i (dream|dreamed|dreamt) (.*)',
     ["What does that dream suggest to you?",
      "Do you dream often about {1}?",
      "What do you think that dream means?"]),
    
    # Question patterns
    (r'why (.*)',
     ["Why do you think {0}?",
      "What makes you ask that?",
      "Does that question interest you?"]),
    
    (r'how (.*)',
     ["How do you suppose {0}?",
      "What makes you think about {0}?"]),
    
    # "I remember" patterns
    (r'i remember (.*)',
     ["Do you often think of {0}?",
      "What else do you remember about {0}?",
      "Why do you recall {0} right now?"]),
    
    # "I think" patterns  
    (r'i think (.*)',
     ["Do you really think {0}?",
      "What makes you think {0}?",
      "Do you doubt that {0}?"]),
    
    # "Yes/No" responses
    (r'^yes$|^yeah$|^yep$',
     ["You seem quite certain.",
      "I see. Can you elaborate?"]),
    
    (r'^no$|^nope$',
     ["Why not?",
      "Are you sure?",
      "Can you explain?"]),
    
    # Catch-all patterns
    (r'.*',
     ["Can you elaborate on that?",
      "I see. Please continue.",
      "Very interesting. Tell me more.",
      "How does that make you feel?",
      "What does that suggest to you?"])
]

print(f"Loaded {len(PATTERNS)} pattern-response pairs")

Step 3: The ELIZA Engine¶

Now we'll implement the core matching and response generation logic.

In [ ]:
class Eliza:
    """
    ELIZA chatbot implementation using pattern matching.
    """
    
    def __init__(self, patterns: List[Tuple[str, List[str]]]):
        """
        Initialize ELIZA with a pattern database.
        
        Args:
            patterns: List of (pattern, responses) tuples
        """
        self.patterns = [(re.compile(p, re.IGNORECASE), r) for p, r in patterns]
        self.history = []  # Store conversation history
    
    def respond(self, user_input: str) -> str:
        """
        Generate a response to user input.
        
        Args:
            user_input: User's message
        
        Returns:
            ELIZA's response
        """
        # Store input in history
        self.history.append(("User", user_input))
        
        # Clean and normalize input
        user_input = user_input.strip().lower()
        
        # Try to match patterns in order
        for pattern, responses in self.patterns:
            match = pattern.match(user_input)
            if match:
                # Select a random response template
                response = random.choice(responses)
                
                # Fill in captured groups
                if match.groups():
                    # Reflect the captured text
                    reflected_groups = [reflect(g) if g else '' for g in match.groups()]
                    response = response.format(*reflected_groups)
                
                self.history.append(("ELIZA", response))
                return response
        
        # Should never reach here due to catch-all pattern
        default = "I'm not sure I understand. Can you rephrase?"
        self.history.append(("ELIZA", default))
        return default
    
    def print_history(self, n: Optional[int] = None):
        """
        Print conversation history.
        
        Args:
            n: Number of recent exchanges to print (None for all)
        """
        history = self.history[-n:] if n else self.history
        for speaker, message in history:
            print(f"{speaker}: {message}")

# Create an ELIZA instance
eliza = Eliza(PATTERNS)
print("ELIZA initialized and ready!")

Let's Test ELIZA!¶

In [ ]:
# Test conversation
test_inputs = [
    "Hello",
    "I am feeling sad",
    "I think my mother doesn't understand me",
    "I want to be happy",
    "Why do you ask so many questions?"
]

for user_input in test_inputs:
    response = eliza.respond(user_input)
    print(f"User: {user_input}")
    print(f"ELIZA: {response}")
    print()

Part 2: Interactive Testing¶

Now let's have a real conversation with ELIZA!

In [ ]:
def chat_with_eliza(max_turns: int = 10):
    """
    Interactive chat session with ELIZA.
    
    Args:
        max_turns: Maximum number of conversation turns
    """
    eliza_bot = Eliza(PATTERNS)
    print("ELIZA: Hello. I am ELIZA. How can I help you today?")
    print("(Type 'quit' to exit)\n")
    
    for turn in range(max_turns):
        user_input = input("You: ").strip()
        
        if not user_input:
            continue
            
        response = eliza_bot.respond(user_input)
        print(f"ELIZA: {response}\n")
        
        if re.match(r'quit|exit|bye|goodbye', user_input.lower()):
            break
    
    return eliza_bot

# Uncomment to chat:
# my_eliza = chat_with_eliza()

Part 3: Pattern Debugging and Analysis¶

Let's build tools to understand which patterns are matching and why.

In [ ]:
def debug_pattern_match(user_input: str, patterns: List[Tuple[str, List[str]]]):
    """
    Show which patterns match a given input and in what order.
    
    Args:
        user_input: Input text to test
        patterns: List of (pattern, responses) tuples
    """
    user_input = user_input.strip().lower()
    print(f"Testing input: '{user_input}'\n")
    print("Matching patterns (in priority order):")
    print("-" * 80)
    
    matches_found = 0
    for i, (pattern_str, responses) in enumerate(patterns):
        pattern = re.compile(pattern_str, re.IGNORECASE)
        match = pattern.match(user_input)
        
        if match:
            matches_found += 1
            print(f"\n✓ Pattern {i+1}: {pattern_str}")
            print(f"  Captured groups: {match.groups()}")
            print(f"  Possible responses ({len(responses)}):")
            for j, resp in enumerate(responses[:3], 1):  # Show first 3
                print(f"    {j}. {resp}")
            
            # Show what the response would be
            response = responses[0]
            if match.groups():
                reflected_groups = [reflect(g) if g else '' for g in match.groups()]
                response = response.format(*reflected_groups)
            print(f"  → Example output: '{response}'")
            
            if matches_found == 1:
                print("  ⭐ This pattern would be used (first match)")
    
    if matches_found == 0:
        print("No patterns matched!")
    else:
        print(f"\n{matches_found} pattern(s) matched total")

# Test it
debug_pattern_match("I am feeling sad about my mother", PATTERNS)

💡 Exercise 2: Debug These Inputs¶

Use debug_pattern_match() to see which patterns match these inputs:

  1. "I think I need help"
  2. "My father was very strict"
  3. "Why am I always so anxious?"

For each, identify:

  • Which pattern matched first?
  • What groups were captured?
  • How were the groups reflected?
In [ ]:
# Your code here

Part 4: Extending ELIZA¶

Now it's your turn to improve ELIZA!

💡 Exercise 3: Add New Patterns¶

Add patterns for these scenarios:

  1. Work/job related: Detect mentions of "work", "job", "boss", "colleague"
  2. Relationships: Detect "friend", "boyfriend", "girlfriend", "relationship"
  3. Anxiety/worry: Detect "worried", "anxious", "scared", "afraid"

For each pattern, create 2-3 appropriate response templates.

In [ ]:
# Extended patterns - add your new patterns here
EXTENDED_PATTERNS = PATTERNS.copy()

# TODO: Add work-related patterns
# Example:
# EXTENDED_PATTERNS.insert(10, (
#     r'.*\bwork\b.*',
#     ["Tell me more about your work.",
#      "How do you feel about your job?"]
# ))

# Your patterns here:


# Test your extended ELIZA
extended_eliza = Eliza(EXTENDED_PATTERNS)

test_cases = [
    "I hate my job",
    "My boss is terrible",
    "I'm worried about my relationship",
    "I feel anxious all the time"
]

for test in test_cases:
    print(f"User: {test}")
    print(f"ELIZA: {extended_eliza.respond(test)}")
    print()

💡 Exercise 4: Add Context Memory¶

ELIZA has no memory - it doesn't remember what was discussed earlier. Implement a simple memory system:

  1. Track mentioned topics (e.g., "mother", "father", "work")
  2. Occasionally reference previously mentioned topics
  3. Example: "Earlier you mentioned your mother. How does that relate to this?"
In [ ]:
class ElizaWithMemory(Eliza):
    """
    Extended ELIZA with simple topic memory.
    """
    
    def __init__(self, patterns: List[Tuple[str, List[str]]]):
        super().__init__(patterns)
        self.topics_mentioned = set()
        self.memory_keywords = ['mother', 'father', 'family', 'work', 'job', 
                                'friend', 'relationship', 'dream']
    
    def respond(self, user_input: str) -> str:
        # TODO: Extract topics from user input
        # TODO: Occasionally reference past topics
        # TODO: Call parent respond method
        
        # Starter code:
        user_lower = user_input.lower()
        
        # Extract topics
        for keyword in self.memory_keywords:
            if keyword in user_lower:
                self.topics_mentioned.add(keyword)
        
        # Your implementation here
        
        return super().respond(user_input)

# Test your implementation
# memory_eliza = ElizaWithMemory(EXTENDED_PATTERNS)

Part 5: Analysis and Limitations¶

Let's explore what makes ELIZA work and where it fails.

Pattern Coverage Analysis¶

In [ ]:
def analyze_pattern_coverage(test_sentences: List[str], patterns: List[Tuple[str, List[str]]]):
    """
    Analyze which patterns are used most often on a test set.
    """
    pattern_usage = {}
    
    for sentence in test_sentences:
        sentence = sentence.strip().lower()
        
        for i, (pattern_str, _) in enumerate(patterns):
            pattern = re.compile(pattern_str, re.IGNORECASE)
            if pattern.match(sentence):
                pattern_usage[i] = pattern_usage.get(i, 0) + 1
                break  # Only count first match
    
    print("Pattern Usage Statistics:")
    print("-" * 80)
    
    sorted_patterns = sorted(pattern_usage.items(), key=lambda x: x[1], reverse=True)
    
    for pattern_idx, count in sorted_patterns[:10]:  # Top 10
        pattern_str = patterns[pattern_idx][0]
        print(f"Pattern {pattern_idx}: {pattern_str[:50]}... → {count} times")
    
    catch_all_idx = len(patterns) - 1
    catch_all_count = pattern_usage.get(catch_all_idx, 0)
    print(f"\nCatch-all pattern used: {catch_all_count} times ({catch_all_count/len(test_sentences)*100:.1f}%)")

# Test with some realistic inputs
realistic_inputs = [
    "Hello, how are you?",
    "I'm feeling really depressed lately",
    "My mother never understood me",
    "I want to be successful",
    "Why do I always feel this way?",
    "I had a terrible dream last night",
    "I think something is wrong with me",
    "Nobody at work respects me",
    "I need someone to talk to",
    "Sometimes I just feel empty inside"
]

analyze_pattern_coverage(realistic_inputs, PATTERNS)

💡 Exercise 5: Breaking ELIZA¶

Find inputs that expose ELIZA's limitations:

  1. Repetition: What happens if you repeat the same input?
  2. Nonsense: Does ELIZA handle gibberish well?
  3. Complex syntax: Try long, complex sentences
  4. Sarcasm/humor: Can ELIZA detect non-literal language?
  5. Context: Ask a follow-up question that requires memory

Document your findings below:

In [ ]:
# Test cases that break ELIZA
breaking_inputs = [
    # Add your test cases here
    "I am sad",
    "I am sad",  # Repetition
    "I am sad",
    # More tests...
]

test_eliza = Eliza(PATTERNS)
for inp in breaking_inputs:
    print(f"User: {inp}")
    print(f"ELIZA: {test_eliza.respond(inp)}")
    print()

Part 6: Discussion Questions¶

Discuss these questions with your group:

  1. The ELIZA Effect: Why did people form emotional connections with ELIZA, even knowing it was a simple program? What does this tell us about human conversation?

  2. Pattern Priority: Why does the order of patterns matter? Give an example where reordering patterns would change ELIZA's behavior.

  3. Limitations: What are the fundamental limitations of pattern-matching approaches to NLP? What can they never do?

  4. Modern Applications: Where do we still use pattern matching in modern NLP systems? (Think: preprocessing, rule-based components, etc.)

  5. Evaluation: How would you evaluate whether ELIZA is "good"? What metrics would you use?

  6. Ethics: Weizenbaum was disturbed when people became emotionally attached to ELIZA. What ethical considerations should we have when building conversational AI?

Write your answers here:


(Your discussion notes)


Part 7: Challenge Exercise (Optional)¶

Build a Domain-Specific ELIZA¶

Create an ELIZA-style chatbot for a different domain:

  • Tech support bot: Helps with computer problems
  • Fitness coach: Motivates and advises on exercise
  • Study buddy: Helps with homework and studying
  • Career counselor: Gives career advice

Requirements:

  1. At least 15 domain-specific patterns
  2. Appropriate reflections for your domain
  3. Test with realistic conversations
  4. Document limitations specific to your domain
In [ ]:
# Your domain-specific ELIZA here
DOMAIN_PATTERNS = [
    # Add your patterns
]

DOMAIN_REFLECTIONS = {
    # Add domain-specific reflections if needed
}

# Implementation...

Summary and Takeaways¶

Today you learned:

  1. ✅ Pattern Matching: How regex patterns can drive conversation
  2. ✅ Pronoun Reflection: Converting first-person to second-person and vice versa
  3. ✅ Template Responses: Using placeholders and captured groups
  4. ✅ Debugging: Analyzing which patterns match and why
  5. ✅ Limitations: Understanding what rule-based systems can't do

Key Insights:¶

  • ELIZA demonstrates that simple pattern matching can create surprisingly engaging conversations
  • The "ELIZA effect" shows humans are predisposed to read intelligence into systems
  • Pattern-based approaches are brittle and don't scale well
  • Modern NLP has moved beyond patterns, but they're still useful for preprocessing and specific tasks

Next Week:¶

We'll move from rule-based systems to statistical approaches - text classification using machine learning!


Questions? Bring them to office hours or post on Piazza!