The quiet erosion nobody is tracking — and why it will cost companies their most durable competitive advantage.
E veryone is watching the headcount number. That’s the metric boards ask about, the one that ends up in earnings calls, the one that makes headlines. And I get it; it’s concrete. You can count on it. But after sitting inside a lot of AI transformations, the thing that actually keeps me up at night isn’t the jobs that go away. It’s the knowledge that goes with them and more precisely, the knowledge that never gets built in the first place.
We are, right now, in the early stages of a slow-motion institutional memory crisis. And almost nobody is talking about it.
The Expertise You Don’t Know You’re Losing
Here’s what typically happens when a company automates a knowledge-intensive task with AI. A senior person, someone who spent five, eight, ten years developing genuine judgment gets their workflow restructured around a model. They stop doing the hard parts. The model does them.
Six months in, productivity metrics look great. Throughput is up. Error rates are down. Leadership declares the initiative as a success.
Two years in, that senior person left. And suddenly the team realizes something uncomfortable: they have no one who can evaluate whether the model is making good decisions anymore. The human judgment that used to provide the quality check on the process is gone. Not because the AI replaced it, but because it was never exercised long enough to be passed on.
This isn’t hypothetical. I’ve watched it happen. The model becomes load bearing before anyone realizes the expertise underneath it has quietly atrophied.
The model becomes load bearing before anyone realizes the expertise underneath it has quietly atrophied.
The Apprenticeship Problem Nobody Is Solving
There’s a reason every skilled discipline in history has been built around apprenticeship. You don’t become a good underwriter by reading coverage manuals. You become one by sitting next to someone who’s seen a thousand edge cases and developed instincts that can’t be written down. Same for surgeons, lawyers, engineers, claims adjusters, financial analysts, any domain where judgment is the product.
AI breaks that chain. Not malicious. Just structurally.
When a junior analyst’s job becomes reviewing and approving AI-generated outputs instead of building the analysis themselves, they learn a very different skill set. They learn to be good editors of machine output. That is a skill, but it is not the same skill as knowing how to construct the analysis in the first place. And critically, it doesn’t build the mental model you need to know when the machine is wrong.
The organizations that figure out how to thread this needle and how to use AI to amplify expertise without short-circuiting its development are going to have a genuinely durable competitive advantage. Not because they’re more efficient today. Because they’ll have humans who can supervise, interrogate, and improve the AI five years from now.
The organizations that don’t figure this out will have highly efficient operations staffed by people who can operate the system but can’t evaluate it. That’s fragile architecture.
There’s a Second Problem, and It’s Less Obvious
The first problem is expertise that doesn’t get built. The second problem is expertise that gets extracted and then lost when the model changes.
When you fine-tune a model on your historical decisions, you’re essentially encoding your institutional knowledge into a system you don’t fully control. That’s not inherently bad. But it creates a dependency most organizations haven’t thought carefully about.
What happens when the underlying model gets updated and starts behaving differently? What happens when you want to understand why the model is making a specific recommendation and the people who could explain the reasoning it was trained on have all moved on? What happens when a regulator asks you to explain a decision, and your answer is essentially “the model said so”?
The companies that treated AI deployment as a knowledge management strategy actively documenting the logic, maintaining the human expertise alongside the model, building systems that make the reasoning visible, those are the companies that retain optionality. They can audit the model. They can override it intelligently. They can improve it.
The companies that treated it as a cost reduction exercise will find themselves, at some point, unable to do any of those things.
Efficiency is a commodity. The ability to interrogate your own systems is not.
What to Actually Do About It
This isn’t an argument against AI adoption. That ship has sailed, and honestly the efficiency gains are real and significant. This is an argument for being intentional about what you’re trading away in pursuit of those gains.
A few things worth building into any serious AI strategy:
Deliberate expertise in preservation
Identify the judgment-intensive capabilities that your AI is replacing or shortcutting. Then decide — explicitly, not by accident — which of those capabilities you need to maintain in human form, and how. This might mean keeping a small team doing things manually that the AI does at scale. It might mean structured knowledge capture before experienced people leave. It almost certainly means rethinking how you develop junior talent in a world where the “do it the hard way first” path has been automated.
Adversarial review as a standing practice
The people responsible for your AI “systems” should include people whose job is to break them, to find the edge cases, to probe the recommendations, to ask “why did it say that?” regularly enough that someone still knows how to answer. This function atrophies fast if it’s not deliberately resourced.
Legible reasoning is not negotiable
Any AI system making consequential decisions should produce a decision trace that a human expert can evaluate. Not just for compliance. For the organizational immune system. The day you can no longer evaluate whether your AI is reasoning well is the day it owns you, not the other way around.
The efficiency story around AI is mostly right. The knowledge story is mostly unwritten. And right now, the companies that get ahead of it have a window that won’t stay open long.
How We Think About This
Our work with F&I administrators and operations teams is grounded in exactly this tension: how do you capture the efficiency gains from AI without hollowing out the organizational expertise that makes those gains sustainable?
The answer we’ve built toward is a tiered model where AI amplifies human judgment rather than replacing it, keeping experts in the loop on high-stakes decisions, building feedback mechanisms that retrain models on real outcomes, and designing workflows that develop junior talent even as the routine work gets automated.
If this is a conversation you want to have with someone who’s thought about it seriously — PCMI welcomes the exchange. Just a direct conversation about what a thoughtful AI strategy looks like for your F&I Administration and Claims handling, starting with the questions most vendors don’t want to ask.
Reach out. The best time to think about this is before it becomes obvious.