Haven't we always outsourced our thinking?
The impact of AI on cognition
I was one of those weird engineers who took a minor in English. Literature massaged a different part of my brain and it felt good. As it turns out, that may be one of the antidotes to the risk of AI taking over our thinking.
In a podcast interview, Sheldon Fernandez, former CEO of DarwinAI, said that he’s often asked by students how you stand out from your peers. He replied:
Read literature. Read serious contemplative fiction. Why? Two reasons: there’s a difference between immersing yourself in material in solitude, versus sampling the outputs of generative AI. Most of your generation can’t read a novel right? The ability to do that is significant.
For the first reason (I’ll come back to the second reason), he’s speaking about the kind of long-form thinking achieved through reading books or writing essays or investigating complex problems. There is a lot of chatter on the internet right now about what AI means for human cognition: Is ChatGPT making us stupid? - The Conversation, When AI thinks for students: The silent erosion of critical thinking in classrooms - The Guardian, ChatGPT May Be Eroding Critical Thinking Skills, According to a New MIT Study - The Times. What happens when more and more people depend on AI?
What kind of thinking are we talking about?
The AI of now has the ability to recall, look up, read information across domains almost instantly as well as answer analytical questions about them. It can write summaries, essays, and code as well as test and revise them. It can have conversations with people in different languages and be tirelessly engaging. When we use AI, we can skip the effort to do the research and synthesis. We can skip the initial writing whether words or code, maybe even forgo writing altogether. It seems we also no longer need to learn other languages.
Setting aside the current technical problems with AI (hallucinations, errors in complex tasks, sycophancy) and the massive social implications of AI (who owns it, who can afford to use it, the environmental cost… which too be frank is a bigger threat and deserving of a different article) what do we lose when we let AI do the work? Before we get into the drawbacks, I want to start somewhere else. Throughout history haven’t we outsourced our thinking several times, so is this different?
Here’s two famous stories.
Socrates on the dangers of writing
The first comes from the famous Greek philosopher Socrates (via Plato’s dialogue Phaedrus, 274c–275b1) who criticized the use of writing. Socrates tells Phaedrus an Egyptian myth about the god Theuth who presented the gift of writing to King Thamus claiming it to be an “elixir of memory and wisdom.” King Thamus disagrees saying that writing will produce forgetfulness in those who learn it, because they’ll stop exercising their memory and rely on external marks instead. Worse, Thamus argues, people will absorb many ideas without truly understanding them, gaining the appearance of wisdom rather than the thing itself. And there is a deep truth to it. Oral traditional cultures required people to develop prodigious memories and practice in discussing and debating ideas. Today I can barely remember what I’m doing the next day. But of course, we now have access to a larger pool of knowledge than could ever be contained in oral stories. In fact, we can only access Socrates’ wisdom through Plato’s writing.
A bicycle for the mind
The other story is about how Steve Jobs referred to a computer as a “bicycle for the mind”2. He had read about how man was not very efficient at getting around on its two feet in comparison to most other animals, but a man+bike was more efficient at moving than any other animal (presumably given a decent trail). Jobs was quoted as saying “We think we’re basically fashioning a 21st century bicycle here which can amplify an inherent intellectual ability that man has and really take care of a lot of drudgery to free people to do much more creative work.”
And sure enough a computer has been a powerful cognitive assist. As a former engineer, I relied on spreadsheets to do complex calculations, documents to map out designs, databases to track bugs and reporting, and code to operate complex machinery. It allowed me to build, in collaboration with hundreds of other people, millions of lines of code that no one person has even read.
Eroding our connection to reality
On the other hand computers have also narrowed our thinking to a screen. In the world of the internet, we start to believe that the digital bits and bytes are more real than reality. PowerPoint slides become substitutes for actual thinking. Emails supplant face-to-face communication.
It’s not just our memory or rhetorical ability or feel for reality that has eroded. Our society is so specialized that we no longer need to grow our own food or worry about water or sanitation, so we’ve lost some of our self-reliance. We are more dependent on everyone else for survival than ever before. And given the massive amounts of information available to us, we generally accept whatever we are taught or absorb. In other words, culture does the thinking for us.
So this seems to imply AI is very likely to cause us to lose some capabilities but also gain new ones. Still, the pace of change is mind-bending. Since the Industrial Revolution, our societies have been under pressure to adapt. Think how technological prowess has enabled the colonization of the Americas, the devastation of the environment, the wars that have killed countless millions. Change used to happen from generation to generation. Now it seems to be happening decade to decade. What happens when the bicycle for mind becomes an explosive rocket ship? Can we even learn to pilot it quickly enough before we crash it?
Thinking is a manual skill.
Use it or lose it as they say, equally applicable to our muscles as it does to our minds. My son absorbs everything right now, but with each challenge he overcomes, he levels up because striving is what leads to growth. So it doesn’t take too much research to convince me that when AI does the writing and researching for you, the writer learns less, remembers less, and essentially knows less. In the long run, this might mean less ability to innovate.
In Fernandez’s article in The Walrus, he cites a study:
In her seminal paper “Ironies of Automation,” Lisanne Bainbridge observed a striking paradox: as systems became more automated, as operators were removed from routine and repetitive tasks, they became worse at handling scenarios in which those systems failed.3
It makes sense to me that when operators don’t have a solid grounding in how the system works (its causal architecture in other words), they don’t develop the intuitive understanding that enables them to quickly make sense of novel situations and respond. Operators would also be less able to improve the system. Taken to the extreme you can imagine a scenario where humans don’t do any of the work, don’t understand how it works, and therefore cannot innovate.
Experienced non-experts versus actual experts
On the other hand, experts that have done the work of understanding the system can truly use AI as a cognitive assist to accelerate their work. Experts can make a judgment about the quality of the answers returned by the computer in a way a beginner wouldn’t. The biggest distinction may not even be between experts and beginners. It’s between those who are experienced but have settled into routines or habits (the experienced non-expert) and those who are always pushing the boundaries of what they know (the expert)4. The non-expert would use AI to skip the struggle for understanding so that the problem can be solved quickly. The expert would use AI to push their own thinking to tackle greater challenges.
We need to get better at metacognition
We can only guess at the long-term impact of AI, but the consequences may emerge more quickly than the time it took us to understand the impacts of social media. It does suggest that all of us will need to get better at metacognition5 - thinking about thinking - especially if computers can do the routine work faster and better than us. We will need to have a holistic understanding of what healthy cognitive “fitness” is if we are ensure that humanity develops in wise ways. Fernandez gets at with his second reason for why read literature.
And the second, looking to 2030, in a world where that kind of narrow intelligence is abundant, what do we have left? We have our understanding of the human condition and how we appropriate it to be empathic with human beings. Why does literature survive? Because it tells us something about human existence and that’ll help you.
The reality is we have already been outsourcing our thinking. It’s whether we can retain some level of oversight and control of it. Literature, well-written non-fiction or any aspect of culture which connects us with our human story and experience will serve us well. In poker they say “If you don’t know who the sucker is, then it’s you.” In life, if you aren’t doing the thinking, someone else is doing it for you.
Read a more detailed account Machinocene’s Substack post “Steve Jobs and the computer as a ‘bicycle for the mind’”
“ChatGPT Isn’t Just Changing How We Work. It’s Harming How We Think” by Sheldon Fernandez
Carl Bereiter and Marlene Scardamalia, Surpassing Ourselves: An Inquiry into the Nature and Implications of Expertise (Chicago: Open Court, 1993)
Take a look at our work at the Human Venture Institute, which is essentially about understanding metacognition but at that civilizational level.