The Gap Between What We Build and What People Actually Want
Last week I shipped a feature that automatically categorizes user data. It took three weeks. It works perfectly. Nobody asked for it. I built it because the architecture made it trivial—just another API call, a quick database schema redesign, maybe 40 lines of clean code. But when I watched someone use it, they looked... uncomfortable. Not confused. Uncomfortable. That's when it hit me: we're not solving for people anymore. We're solving for elegance.
The thing nobody in tech wants to admit is that the people experiencing our tools every single day don't experience them the way we do. When I see an API integration opportunity, I see possibility. When my mom sees AI in Google Search, she sees search results that got worse. When a junior designer sees Figma's new AI feature, they're calculating how long until that job doesn't exist. I'm not sure this gap is actually closeable with better product design, which is what we usually reach for.
- We think about databases. They think about losing control.
- We see automation as liberation from repetitive tasks—what I wouldn't give to have AI write my database migrations—and we assume everyone else wants the same freedom from their work. They don't.
- We're moving faster than ever. That's not always good.
The Honest Problem With Building AI-First
I've been building products at the intersection of AI and user experience since 2022. Early on, I made a decision that felt obvious: let the AI dictate the UX. If the model can process 10,000 data points simultaneously, design around that capability. If the system works best with structured inputs, ask users to structure their chaos. Make them legible to the machine.
I'm not sure this was the right move, but I kept doing it anyway because everyone else was. The feedback loops rewarded it. Investors rewarded it. Github Copilot worked because engineers were already thinking in code. But I watched the same approach fail catastrophically when we tried to apply it to non-technical users. The people who didn't think in databases didn't want to start.
Building AI tools means constantly choosing between two paths. Path one: make the tool adapt to how humans actually work, which is messy and contradictory and full of context that changes minute to minute. Path two: ask humans to adapt to the tool, to structure their lives so the algorithm can see them clearly. Path two is technically easier. Path two is also why people are growing to hate these products.
What We're Actually Asking For
When we integrate Claude or GPT into a product, we're not just adding intelligence. We're asking for permission to surveil. To collect. To pattern-match against your behavior. Every note-taking app that now has AI summaries is asking you to let it read everything you write. Every email client with AI drafting is asking you to give it your communication patterns, your voice, your relationships.
And the payoff? Slight convenience. Maybe 15 minutes saved per week if you're optimistic. Against that we're asking people to accept:
- Cybersecurity risks that even the companies building this stuff don't fully understand
- The knowledge that their data is being used to train the next model, which competes with them
- A slow erosion of skills—why remember how to write a professional email when the AI can do it
I genuinely don't know if that trade is worth it from a user's perspective. From a builder's perspective it's irresistible. From a human perspective it might be a disaster we're all pretending is progress.
The Thing Nobody Says Out Loud
The companies leading this charge—OpenAI spending $200 million on marketing, Microsoft trying to convince governments that AI is worth the energy crisis, Anthropic positioning enterprise automation as salvation—they're not wrong about the technology. The tools work. They're genuinely useful in specific contexts. But they're also operating from a fundamental assumption that I'm starting to question:
That optimization is always good. That automation is always progress. That if something can be made more efficient, it should be.
People don't experience their lives as databases to be optimized. They experience them as messy, contradictory, full of meaning that can't be quantified. And we keep building tools that ignore that. We keep asking them to become more legible to our systems instead of making our systems more legible to their actual lives.
I'm still building AI products. I still believe there are real problems it can solve. But I'm doing it with less certainty than I started with, and more
