We're Pretending. Let's Stop.
Last month, I watched a client spend forty minutes asking ChatGPT to rebuild a Looker Studio dashboard query. The AI gave an answer. He took it. Nobody questioned whether that answer made sense for his actual business problem. That's not collaboration. That's abdication dressed up in buzzwords.
Real collaboration requires friction. Requires pushback. Requires someone—human or otherwise—saying "wait, that metric doesn't measure what you think it measures." When you're working in Power BI or Looker Studio, you're not just moving data around. You're making assumptions about what matters. Those assumptions deserve interrogation.
I'm not sure this framing is entirely fair, honestly. Some teams do push back on AI outputs, do dig into the logic. But most? Most take the path of least resistance. They ask a question, get a formula, drop it into their dashboard, and move forward. I've done it myself. It's faster. Feels productive. Feels wrong afterward.
What Actually Collaborative AI Work Looks Like
- The human defines the business problem, not just the technical question. This takes time. Most skip it.
- AI generates multiple approaches with explicit trade-offs listed—here's fast but less accurate, here's slower but captures nuance in your customer retention data.
- The human reads those trade-offs. Actually reads them. Doesn't just pick "accurate."
- Back and forth. The human says "your definition of churn is too generous—we only care about customers who haven't paid in 90+ days." The AI recalibrates. The human checks the recalibration against three months of historical data to make sure it actually works.
- Someone asks why.
That last point matters more than the others. In every Power BI project I've seen fail, it's because someone accepted a calculation without asking why it was built that way. A metric looked reasonable on the surface, moved into production, and six weeks later the finance team discovered it was overcounting regional revenue by fifteen percent because nobody questioned the join logic.
The Asymmetry Problem
Here's where I'm genuinely uncertain about where this is heading: AI systems are becoming good enough that they *sound* authoritative. A language model won't hedge unless you specifically ask it to. It won't say "I'm not sure if this DAX formula works across your data model because I haven't seen your schema." It'll generate something that compiles, that runs, that produces numbers.
Numbers that might be wrong in subtle, consequential ways.
In 2023, according to Gartner's analytics report, 62% of organizations said their biggest data challenge wasn't *collecting* data—it was *trusting* it. Not trusting the tools. Trusting the logic inside the tools. And now we're adding another layer: AI-generated logic that most humans don't fully understand, embedded in dashboards that executives use to allocate budgets.
I've started telling clients to document the *reasoning*, not just the results. Why did we choose this metric? What were the alternatives? What breaks if this assumption changes? That documentation forces the collaboration to happen. Forces someone to slow down.
What Teams Should Actually Do
If you're building dashboards in Looker Studio or Power BI and you're using AI to accelerate that work, structure it differently:
- Use AI for hypothesis generation and variation exploration. Not for final answers.
- Require a human to articulate the business rule before accepting the technical implementation. "We measure engagement as daily active users" is not the same as "daily active users where session duration > 5 minutes."
- Build review gates. Especially for anything touching revenue, churn, or customer health metrics.
- Ask for the opposite calculation. If AI builds you a formula for customers who stayed, make it build the inverse and compare. They should add up to your total.
Most of this is basic quality control. We're acting like it's revolutionary because we slapped "AI" on it.
The Uncomfortable Part
I don't actually know if this is solvable at scale. Organizations are under pressure to move fast. Documentation slows things down. Review gates slow things down. Asking why slows things down.
And the tools—they're getting better at generating plausible answers faster. Which makes the pressure to just accept them stronger.
