Reviewing is not thinking
People say that you will get dumber if you use AI to do your work, unless you know what you’re doing and design your workflow deliberately so AI becomes your biggest multiplier instead of your greatest threat. I didn’t fully understand this until one day one of my engineers shared his story on how he uses AI on our slack channel.
He shared that he had become too dependent on AI. For a while, he used AI to answer almost every coding problem he ran into. Then one day he tried to build a little side project by hand. While building it, he realized even his basic skills had gotten rusty. Whenever he hit a problem, instead of working out the solution himself, his immediate response was to reach for an LLM answer.
After realizing this, he changed how he works. Now he uses AI purely as a thinking partner, not as a tool to solve the problem directly. He asks it to point him toward resources and concepts to learn, so he arrives at the answer himself. He said while he is not coding as fast as before, he is more aware of his work. Most importantly, his confidence in his coding skills has improved.
Hearing his story made me curious enough to go looking for research on how using AI affects our reasoning and critical thinking. Two studies in particular stood out to me:
- A study by Microsoft and Carnegie Mellon from June 2025. They surveyed knowledge workers who use GenAI weekly, collecting 936 real-world AI use cases. They found that higher confidence and trust in AI correlated with less critical thinking effort. AI users produced “a less diverse set of outcomes” and showed weaker memory retention. One line from the research stuck with me regarding automation:
“…deprive the user of the routine opportunities to practice their judgement… leaving them atrophied and unprepared when the exceptions do arise.” – The Impact of Generative AI on Critical Thinking
- A working paper from Wharton researchers. They ran 3 preregistered experiments across more than 1,300 participants, where the AI was sometimes programmed to give faulty answers. They found that when the AI was wrong, people still followed it about 80% of the time, performing worse than they would have with no AI at all.
My engineer’s story and the result of those studies made me reflect on my own journey of using AI in my PM work. Do I use it to sharpen my thinking and keep building my PM skills, or do I let myself outsource everything to it and deliberately make myself rusty?
Judgement under ambiguity
As PMs, we don’t have any visible skills (no code, no mockup to show off) because our work is fused into other roles’ output. The real PM skill is the judgement we make: what’s in or out of scope, what to prioritise, what the risks are, what to push back on. That judgement shapes the code the engineers write and the mockup the designer makes. Letting AI make those judgements for me is the PM equivalent of an engineer forgetting how to write clean, usable code.
As agency PMs especially, we work under constraints that traditional in-house PMs don’t face, mainly:
- No positional authority.
- No product ownership.
- No direct access to the end user or product data.
- Undefined or shifting success criteria.
Under those constraints, agency PMs have to develop a skill I call judgement under ambiguity. A good agency PM thrives under them, making decisions that lead not only to a successful product release, but also to client satisfaction and a respectable margin.
That skill is my edge, so I have every reason to keep sharpening it.
That’s exactly why AI gave me pause. When I first encountered it, I saw how powerful it was and how easy it would be to hand all my thinking over to it. But outsourcing everything would mean losing what makes me good at this: the judgement muscle I’d built over years. At the same time, refusing to touch AI felt naive. I’d fall behind the people using it every day to learn and enhance their own work.
With that in mind, I decided to build an approval gate for every workflow automation I run. I ask the LLM never to edit or execute its idea without my approval. I plan everything with AI and read all of its reasoning and summaries. If it looks good and makes sense, I approve and the AI executes. If I don’t, I tell the AI what’s wrong, and it readjusts or pushes back until I finally approve the draft.
I thought that having me in the loop and reviewing the AI’s work before it executes would guard me from losing my edge.
I was wrong.
This was an illusion I built to convince myself I was still in control. At first it wasn’t apparent, but slowly I realized I’d started agreeing with the AI almost immediately. I had become too trusting of it. Now you may ask, “Is that bad? As long as the AI is giving the right call, isn’t it okay?” And since I’m the approval gate, it’s not as if I don’t understand what the AI is doing. I could explain the reasoning behind any of it. So I’m still doing the work, right?
Well, that’s the trap.
Recognizing vs Generating
The core finding from the Microsoft/Carnegie Mellon study I mentioned earlier maps directly onto my situation: the higher your trust in the AI, the less critical thinking you do.
In reality, what I’ve done is feed the problem to the AI and it does all the thinking. The approval gate is only me reviewing it. My job is now just to say yes or no to the AI’s work, and as my trust in it grows, I put in less and less real thinking each time, without ever noticing it happen. My “review” silently degrades into rubber-stamping, and before long I’ll lose my edge and blindly trust whatever the AI says.
I’ve since learned there’s a name for this failure mode: automation complacency. When AI proposes a polished, confident-sounding answer, evaluating it is cognitively harder than it looks. You feel like a judge, but you’re just agreeing.
What I was doing was only recognition (does this look right?), which is low-effort work. Doing it continuously will slowly but surely degrades my thinking and PM muscle. What keeps the skill sharp is generation (what would I have done?), which is the higher-effort work. By handling the routine generation, the AI removes my reps at producing judgement from scratch and leaves me the easier task of recognizing a plausible answer. And that generation step, producing the answer myself, is exactly how skill gets built.
When I make a call: drop this feature, prioritise that ticket, push back on a requirement, decide whether something’s worth the timeline, I reason through the data I have and commit to what I think is best. Then I find out the consequence, good or bad. Both get ingrained in my memory as the stored lesson. And the next time I face a similar situation, my brain pulls up that lesson as one more variable to weigh.
So by only doing an approval workflow, I stripped out one of the core components of that chain: producing the answer. I handed the AI the job of thinking through the solution before I’d reasoned it out myself.
My core PM judgement is being generated by AI, and I’m only evaluating whether its call looks right. For the first few iterations I might still push back with my own view, but over time, as the AI gets better and my judgement deteriorates, I’ll end up blindly following it.
It’s a frightening future. At that point, I’d have no one to blame if AI replaced me.
What I do differently now
Clearly, I need to change how I work with AI. Now, I no longer let it do the thinking first and me just reviewing it. What I do is add friction that forces the generation part of my thinking, through a few practices:
- Form my own answer first, then look at what AI comes up with. In all of my flow, I have to feed the AI what I would have done. Only once I give it my opinion does the AI move to the next step.
- Reject with a reason, not just a no. Whenever I reject a decision the AI made, instead of simply saying no, I force myself to state why I’m rejecting it and what a better alternative would be.
- Periodically do the work with no AI at all. It’s how I find where I’ve gotten rusty.
- Be suspicious of the easy yes. It’s a guard I keep in the back of my mind. Whenever I catch myself agreeing too often and too fast, I stop and reflect.
That shift from passively running an approval gate to actively forcing my own generation is the whole point. Generation is what keeps the muscle working.
Those principles keeps me working with AI while still making my judgement and PM skill sharp and evolving. Here’s a snapshot of the concept in one of my UPDATE workflows; I’ll follow up with a full post going through each of my PM workflows in depth.
When working with AI, what you need is selectivity: knowing which tasks are skill-building and which are throughput. For a PM, the tasks where you want to keep generating first are the job itself: what to build and for whom, how to sequence, which tradeoff to make, how to frame the strategic narrative, or what the real problem is underneath a feature request. The tasks where you can happily let AI lead are the routine ones: formatting, drafting low-stakes comms, or summarizing a doc.
Mastering that separation is the key to using AI as a tool. That is how AI becomes a multiplier instead of a threat. You keep the edge that makes you valuable, and let the tool absorb everything that was only ever friction.