One of the questions I have been thinking about increasingly over the last few months is what exactly we need to know in a world where AI seems to know everything. The question sounds slightly silly when put like that, but I think it is an important one.

If I want to understand a statistical concept, I can ask ChatGPT. If I want to write a piece of code, Claude can write it better than I can. If I need to understand an unfamiliar industry, create a financial model, analyse a document or research a subject, AI can help me do all of these things.

So why should I spend years developing expertise in something that a machine can already do better than I can? Why do I need to learn arithmetic when a calculator can do it faster and more accurately than I can? I recently finished Ethan Mollick’s Co-Intelligence, and one of the most interesting parts of the book approaches this question from almost exactly the opposite direction.

Perhaps AI doesn’t make expertise less important. Perhaps it makes expertise more important. Mollick uses a fairly straightforward example. An experienced architect can look at an AI-generated building plan and judge whether it makes sense. A skilled physician can examine a diagnosis suggested by AI and identify where it may be wrong.

A person who knows nothing about architecture or medicine cannot do either particularly well.

This creates an interesting problem.

The better AI becomes at producing answers, the more important our ability to evaluate those answers becomes. And evaluating an answer requires us to know something ourselves. That knowledge cannot entirely live inside the machine.

Mollick spends some time explaining why. Our working memory is limited. When we encounter a new problem, we use what is already stored in our long-term memory to understand it. The facts, concepts and connections that we have accumulated over time allow us to make sense of new information. Having Google, ChatGPT or some future superintelligent AI available does not make what we know irrelevant.

We still need something inside our own heads. And that brings us to the next problem.

How Do We Become Experts?

Simply doing something repeatedly does not necessarily make us experts at it. Mollick illustrates this through two hypothetical piano students.

One spends hours playing pieces she already knows. She gets better through repetition, but she largely remains within her comfort zone. The other practises differently. She works on things she finds difficult. Someone points out her mistakes. The difficulty increases as she improves. She receives feedback and works specifically on her weaknesses.

Both may spend the same number of hours practising. Their results can be very different. This is the idea of deliberate practice.

Expertise requires knowledge, but it also requires repeatedly operating close to the edge of our ability. We need to make mistakes, understand those mistakes, correct them and then attempt something slightly harder.

There is, however, a problem with deliberate practice. You need a good coach. And good coaches are scarce.

A coach needs to understand the subject better than you do. They need to observe what you are doing, identify where you are going wrong and give you useful feedback. Ideally, they should do this frequently enough that you can correct yourself before bad habits become embedded. Most of us don’t have such a person sitting next to us while we work.

This is where Mollick introduces an idea that I found much more interesting than the usual discussion about AI making us more productive.

What if AI becomes the coach?

AI as a Coach

Imagine two young architects. The first works in the traditional way. He creates designs, studies the work of accomplished architects and occasionally gets feedback from a senior colleague.

The second does all of that, but also has an AI assistant looking at every design he produces. The AI can point out structural problems. It can suggest alternative materials. It can compare his design with thousands of other buildings. It can ask why he made a particular choice. It can identify patterns in the mistakes he keeps making.

Most importantly, it can do this every time he creates something.

The first architect might receive detailed feedback once a week. The second can receive it after every attempt. Mollick describes an experiment at Wharton that takes this idea even further.

Students learning how to pitch an idea interact with multiple AI roles. One helps teach them. Another behaves like a venture capitalist and interrogates their pitch. Another evaluates their performance. Finally, an AI mentor helps them understand what went well, what went wrong and what they should improve before trying again.

That is not AI doing the work for the student. The student is still doing the work. AI is making the practice better. And suddenly something that has historically been scarce becomes much more abundant.

Feedback.

The Strange Paradox of AI and Expertise

This leaves us with a rather strange situation. AI threatens expertise because it can increasingly perform tasks that previously required experts. At exactly the same time, AI may make it dramatically easier to acquire expertise because everyone can potentially have access to a patient, knowledgeable coach.

There is another twist. If everyone has access to the same AI, it does not necessarily follow that everyone becomes equally capable. An expert may actually get considerably more out of AI than a novice.

The expert knows what questions to ask. He can recognise when an answer looks suspicious. He understands the exceptions. He knows when the model is confidently talking nonsense. And because he understands the underlying subject, he can push the AI considerably further.

Which creates an interesting loop. Expertise helps you use AI better. AI helps you develop expertise faster. That expertise then helps you use AI even better. And so on.

That possibility interests me much more than the endless debate about which jobs AI will replace.

I realised that, while gaining familiarity with AI over the past year and a half, I have also inadvertently been running versions of Mollick’s experiment myself.

I have used AI to write software despite not being a professional programmer. I have used it to challenge things I thought I understood. I have asked it to explain unfamiliar concepts, attack my arguments, find holes in my reasoning and sometimes simply tell me that an idea I was rather pleased with doesn’t survive scrutiny.

Sometimes I have used AI to get the answer.

Increasingly, I find the more interesting use is to get better at arriving at the answer myself.

There are things I may no longer need to spend years learning merely so that I can execute them mechanically. But if understanding something helps me recognise a bad answer, ask a better question, understand an exception or make a better decision, then perhaps that knowledge has become more valuable rather than less.

Which leads to a rather different way of thinking about education in the age of AI.

Maybe we shouldn’t ask:

What do I no longer need to learn because AI can do it for me?

Perhaps we should ask:

What should I learn deeply because AI can now help me learn it better?

And if AI really can become an extraordinarily patient coach, available whenever we want to practise, perhaps the arrival of artificial intelligence is not an excuse to stop developing expertise.

It may be the greatest opportunity we have ever had to develop more of it.

You may also like:

You Look Like a Thing and I Love You by Janelle Shane: Of Giraffes, Glitches, and Gentle Warnings – An introduction to AI, AI weirdness style

The Age of AI – Notes That Stayed – A discussion on what the future might be.

Part One: Simulated Masters and the Myth of Neo – Reflection on Melanie Mitchell’s book Artificial Intelligence for thinking humans