Reading about generative AI and knowledge work for my PhD got me thinking about what happens to people's skills as AI takes on more of their work. The productivity benefits are useful, but I found myself wondering what people are learning while that work gets done.
Consider two junior consultants preparing recommendations for a client. Both use generative AI, with one spending much of the project polishing an analysis supplied by the system, while the other uses it to explore possible explanations, test a hypothesis and work through what the client's evidence might mean.
Their finished reports could look very similar, even though the opportunities they have had to practise their judgement could be quite different.
Later, an unexpected problem comes up in a client conversation and each consultant needs to recognise what matters and decide how to respond. If you were the client, what would you want the consultant to understand well enough to discuss with you?
That is the question I would bring back to whoever is responsible for AI across the organisation: what are these consultants getting the opportunity to learn, and what might they be missing?
What happens while the work gets done?
I am an avid supporter of using AI and automation in the workplace, including handing over tasks such as moving transcripts, organising an inbox or scheduling meetings where that makes sense. Giving people time back from the admin they would happily spend less of their day doing is a benefit worth having.
Some tasks also give people the practice they need to become good at their work, even some of the boring ones that we wish we could just hand off.
For a junior consultant, forming a hypothesis and explaining why one interpretation fits the evidence better than another are part of learning how to consult. If AI takes on that work, how does the junior get the practice they will need when the next client brings a less familiar problem?
There is a distinction here between losing a skill someone already possesses and never developing a skill they might otherwise have acquired. Neither is an automatic consequence of using AI, but both should be considered when an organisation is planning its future capability.
Checking an AI-generated answer can itself require considerable expertise. A consultant who challenges its assumptions, identifies missing context and explains why a recommendation is unsuitable may be exercising exactly the judgement we want to develop. Equally, a long conversation with an AI system doesn't always give us assurance about what the person has understood.
What interests me is whether an organisation could see good work coming out of a team and assume its people are developing too. I would want to understand both what they are getting better at and what they might not be developing through a well-intended use of AI and automation, particularly if they are expected to take on more responsibility in a few years.
What the research tells us
Research carried out through Anthropic's Fellows Program examined how developers learnt an unfamiliar programming library with and without AI assistance. Those using AI performed worse on a subsequent unaided assessment of understanding. This is relevant to how people acquire new skills, although the short duration means this study alone cannot tell us whether experienced professionals lose established skills over months or years. [1]
A peer-reviewed study published in PNAS found that high-school students using a GPT-4 assistant resembling standard ChatGPT performed better during mathematics practice, but worse on subsequent unaided exams than students who practised without AI. A version designed to provide teacher-informed hints largely avoided that disadvantage, without producing a detectable improvement in unaided exam performance over the control group. These were short-term assessments in a school, so the findings need care when applied to professional development. [2]
There is also encouraging evidence from the workplace. A peer-reviewed study in The Quarterly Journal of Economics found productivity gains among customer-support agents using AI assistance. During temporary system outages, previously assisted agents continued to handle chats faster than before AI introduction, a pattern consistent with learning. The outages were rare and the estimates noisy, however, so this is suggestive evidence of learning through AI assistance, not a demonstration of long-term skill retention. [3]
The studies examined different tasks, people and forms of assistance, which is why I would be cautious about turning them into a general verdict on AI. They give us a reason to look more closely at how people use it and what they learn in the process.
Different ways of using the same technology
I find it useful to think about AI doing work for us, working with us and taking over parts of the work, with all three potentially having a place within the same project.
In a marketing team, staff might only be allowed to check an AI-generated output, or they might work within a fixed format while choosing the wording themselves. Another arrangement could give them scope to develop a campaign within agreed brand guidelines. These differences tell us how much freedom people have to shape the work, although understanding what they learn needs a closer look, because checking a fixed output could still involve demanding judgement.
A similar question arises in legal work, where someone might insert suggested wording into a contract or examine why that wording suits the client and what trade-offs it creates. Both involve AI-generated text, but the person has different opportunities to work through the reasoning.
For our consultants, that means considering what they are learning to notice, explain and challenge during a client project, including how to spot a need that the original brief missed.
An organisation may decide that some skills need less investment as its work changes, or accept a trade-off to meet an immediate need. That can be a reasonable choice, provided it has considered the implications for the people it expects to become its experienced advisers.
Follow the saved time
One of my first questions would be what happens to any time saved, because that helps us understand what the change has made possible for the people doing the work.
Does the consultant get more opportunity to attend client conversations, explore a difficult problem or receive feedback? Does the team take on more of the same work? Does someone simply finish at a more reasonable hour?
I would want to understand which of those is happening before drawing conclusions about what the organisation has gained.
I would start by speaking to the people doing the work and, where practical, asking them to show me the process. What does the AI actually do? What do they believe it does? Which decisions remain theirs? How would they approach the task without it?
For a team, I might start with a few contrasting tasks and the people doing them. At a strategic level, I would first map where AI is being used and then look at selected workflows. A small sample helps identify questions for further investigation.
Who is considering the longer-term impact?
These questions apply to the useful automations employees build for themselves and to systems introduced through a vendor. I would want to encourage that innovation while making sure someone considers consequences that neither the user nor the supplier may initially have anticipated.
Individuals can reflect on how they use AI, while managers influence the work they are given and the feedback they receive. My view is that overall responsibility should sit with whoever leads the organisation's AI strategy, whether or not that person is the CTO, and should include the capabilities the organisation intends to retain and develop.
This does not need to become a lengthy assessment of every small automation, but it does mean asking deeper questions where AI changes how people practise judgement, learn their work or prepare for greater responsibility.
For a strategic leader, a useful starting point is to choose one important AI-enabled workflow and ask: if we continue working this way, what will our people have more opportunity to become good at, and what will they have less opportunity to practise?
The answer may confirm that the trade-off is worthwhile, or it may show where people need more practice, feedback or responsibility.
For me, that is part of deciding how to use AI well: understanding what we gain from it today and considering what our choices could mean for the people and expertise the business will need tomorrow.