Confidently Wrong: AI, Uncertainty, and Open Source

This is a special episode of WTBAI in which Kimberly sits down with her former colleague Derek Hanson to unpack what language research reveals about today’s AI systems, and together they consider where builders risk going wrong.

Kimberly brings a corpus linguistics lens to large language models, reframing them as pattern-recognition systems trained on messy, biased “corpora” of the web. Her early insight was that AI is as powerful for feedback as it is for generation, and that this is an important distinction for education, ethics, and product design.

Drawing from her EdTech startup (Moxie), she explains how embedding linguistic frameworks (e.g., Swales’ move-step analysis) enabled structured feedback ... until frontier models caught up. The conversation then turns to open source and WordPress, where AI integration is accelerating across a massive ecosystem.

Key themes:

• Corpus vs. model: what LLMs are actually sampling
• “Normalized overconfidence” and confidently wrong outputs
• Why feedback > generation in many real-world use cases
• Guardrails, prompt design, and early “agent-like” systems
• Auditability gap: code transparency vs. output transparency
• Bias sources: training data + human annotators
• Missing voices: humanities, age diversity, non-developers
• Friction as a feature: slowing down for rigor and care
• A critical question for builders: how does your system handle uncertainty?

The practical takeaway for builders is that before shipping AI features, ask whether your system surfaces or suppresses uncertainty, and whether a human could actually defend its outputs.

Links:

• Women Talk About AI: https://womentalkaboutai.com
• Kimberly Pace Becker (LinkedIn): https://www.linkedin.com/kimberlypacebecker
• “Stochastic Parrots” paper (Bender et al., 2021): https://dl.acm.org/doi/10.1145/3442188.3445922