The Invisible Erosion

I was in a room recently watching someone prepare for an important conversation.

They had done their homework. Pages of it. Structured, thorough, professionally formatted. Everything you’d want to see from someone taking the work seriously.

Then I started reading it.

The questions were generic. Interchangeable. The kind that sound rigorous until you realize they could apply to any situation, any industry, any context whatsoever. When I asked the person to walk me through what a good response would look like, there was a long pause.

They didn’t know.

They had the document. They didn’t have the thinking behind it. AI had done the preparation and the human had done the printing. That’s not productivity. That’s the appearance of productivity with the substance quietly hollowed out.

The tell nobody talks about

A few days ago I wrote about a small but revealing signal: the em-dash.

AI loves em-dashes. Overuses them. And when I see a document full of them, submitted without a single edit, it’s one signal among several that something happened. The person may not have written it. More importantly, they may not have read it.

I got pushback. A couple of PhDs took genuine offence. And they had a point worth acknowledging.

Em-dashes have a long and legitimate history in academic and professional writing. Plenty of strong writers use them deliberately and well. My observation wasn’t that every em-dash is an AI tell. It’s that a specific pattern is worth paying attention to. Em-dashes combined with uniform sentence length. Suspiciously clean structure. An author who can’t explain their own work.

No single signal is the tell. The combination is.

A document that reads too smoothly. Sentences that are all roughly the same length. Transitions that are a little too elegant. No rough edges anywhere. Real writing has rough edges. It has a voice that’s inconsistent in small ways because humans are inconsistent in small ways. When everything is polished to exactly the same degree it starts to feel like it came from the same place.

Because it did.

The question isn’t whether you use em-dashes. It’s whether you can defend what you wrote, explain why you made the choices you made, and stand behind the thinking behind the words. If the answer is yes, write however you want.

What deskilling actually looks like

Deskilling doesn’t announce itself.

It doesn’t arrive as a dramatic moment where someone loses a skill overnight. It accumulates quietly, one substitution at a time. Each individual shortcut feels reasonable. The deadline was tight. The output looked fine. Nobody complained.

But the cognitive muscle that develops through struggle, through forming a hypothesis, testing it, refining it, defending it, doesn’t develop when the struggle is removed. It atrophies. Slowly and invisibly. Until the moment it’s needed and it isn’t there.

The research on this is clear and has been for years. Dratsch and colleagues documented it in healthcare in 2023. Less experienced workers are the most susceptible. They defer to the AI output not because they’re lazy but because they don’t yet have the expertise to know when the output is wrong. The AI fills the gap that experience was supposed to fill. And experience never develops.

We are running this experiment at scale right now inside most organizations. The results won’t be visible for another two or three years. By then the damage will be structural.

The mechanism nobody is designing around

Here’s what I observe most often.

People use AI the way they used to use Google. Type in a question. Get an answer. Copy the answer. Move on.

The problem is Google returned links. AI returns conclusions. Confident, well-structured, authoritative-sounding conclusions. And the human brain, particularly under time pressure, is wired to accept confident-looking outputs without interrogation. Parasuraman and Riley documented this cognitive pattern in 1997. They called it automation bias. They were writing about industrial control systems. The mechanism is identical.

The antidote is not to use AI less. It’s to use it differently.

The people who get the most from AI are not the ones asking it for answers. They’re the ones arriving with a hypothesis and using AI to pressure-test it. They bring their judgment first. AI sharpens it. The output is theirs. The thinking is theirs. The accountability is theirs.

That’s not a natural behaviour. It has to be taught, modelled, and deliberately designed into how work actually gets done. Most organizations haven’t started that conversation yet.

The governance gap

Most organizations are failing this in one of two ways.

Either the guardrails are so restrictive that nobody can experiment and the learning never happens. Or there are no guardrails at all and every employee is running their own version of the wild west, feeding whatever they want into whatever tool they want, with no framework for what good looks like.

Both produce the same outcome. An organization full of people who don’t know how to think with AI deliberately. Who use it as a search engine. Who submit the output without interrogating it. Who are slowly, invisibly, becoming less capable than they were before the tools arrived.

The fix isn’t a policy. Policies tell people what they can’t do. What’s needed is a philosophy, a clear organizational answer to the question: how do we think with AI rather than just use it?

That question is not being asked in most boardrooms. It’s not on most leadership agendas. It’s not in most job descriptions or performance reviews or onboarding programs.

It should be.

What this costs

The bill for this period of undesigned AI adoption will come due.

It will show up as junior staff who can’t do foundational work without AI assistance. Senior staff spending time re-checking outputs they shouldn’t have to re-check. Organizations that adopted AI early and fast discovering they built speed on a foundation of eroding judgment.

The technology is not the problem. The absence of deliberate design is the problem.

Thirty years of human factors research tells us exactly what happens when powerful technology gets deployed without designing the human layer. We watched it happen with automation in manufacturing. We watched it happen with electronic health records in medicine. We are watching it happen again right now with AI in knowledge work.

The question was never whether AI is powerful enough.

It’s whether your organization is designed to think with it.

---

References

Parasuraman, R. & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230-253. https://journals.sagepub.com/doi/10.1518/001872097778543886

Dratsch, T., Chen, X., Rezazade Mehrizi, M., Kloeckner, R., Mähringer-Kunz, A., Püsken, M., Baeßler, B., Sauer, S., Maintz, D., & Pinto Dos Santos, D. (2023). Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance. Radiology, 307(4), e222176. https://doi.org/10.1148/radiol.222176

Kücking, F. et al. (2024). Automation bias in AI-decision support: Results from an empirical study. Studies in Health Technology and Informatics, 317, 298-304. https://doi.org/10.3233/SHTI240871

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257-285. https://onlinelibrary.wiley.com/doi/10.1207/s15516709cog1202_4