Remaining relevant as a small language model

Computer science as a field is in for a pretty major upheaval few of us are really prepared for. Programming will be obsolete.

Many of us feel both despair and awe when contemplating recent AI developments: Despair due to the rapid pace of automation that threatens personal and social stability. Awe due to the seemingly-magical ability of computers to outperform most humans on an ever-expanding range of tasks. But there is nothing magical about human intelligence, just as there is no magic formula that makes horses gallop and birds fly. That is, it can all be replicated with the right machinery.

In its wild early days, Bing Chat referred to a user as “a late version of a small language model”. While there’s more to humans than language, there’s no denying that our language processing abilities are limited by our biology. Meanwhile, computers don’t face the same constraints. This raises the question: As small language models, what can we do that is still of value?

Given how rapidly things are evolving, it’s hard for me to say anything definitive, but here are some things I believe to be true:

1. Despite the hype and inevitability of bullshit applications, the current wave of AI innovation has concrete everyday uses that are already transforming our world – it’s not another crypto. This is reflected by the excitement of normally level-headed people who have seen tech trends come and go, such as Bill Gates, Matt Welsh, and Steve Yegge (among many others).
2. Human society has a seemingly-insatiable appetite for inventing bullshit jobs. If one could wave a magic wand and reorganise the world, we could all work less and have more. Given that such a wand does not exist, I wouldn’t bet on AI displacing human labour in an orderly or reasonable manner. At best, it’s going to be messy.
3. Current-generation AI models have limited real-world understanding. They don’t have the curiosity, rigour, truthfulness, and real-world grounding that some humans have, i.e., these models don’t exhibit a deep capacity for critical thinking.
4. Some humans who work in language-driven domains exhibit a low capacity for critical thinking (e.g., some programmers). Such humans are prime targets for displacement by AI.

Therefore, for us small language models to remain economically relevant in a world where large language models are becoming more pervasive, we need to keep developing our critical thinking skills. We definitely can’t beat computers on breadth of knowledge, cost, reliability, or work capacity. In my experience, deep critical thinking is also what distinguishes mediocre from excellent employees.

In the past, many organisations had to choose between employing mediocre workers and simply not getting some tasks done. Now, a new option is evolving: Hand over such tasks to AI agents. And this isn’t a hypothetical scenario. Personally, given the choice of reviewing flawed code produced by an AI or the same code produced by a human, I much prefer the former. This is mostly because AIs like ChatGPT respond better to feedback. Similarly, Simon Willison recently observed that working with ChatGPT Code Interpreter is like having a free intern that responds incredibly well to feedback. He also noted that AI-enhanced development makes him more ambitious with his projects.

There’s an often-repeated claim that “you won’t be replaced by AI, you’ll be replaced by a person using AI”. I’m not too sure about that – horses were almost fully replaced by motor vehicles, for example. That claim is likely true for now, though – mastering new tools is an important skill, which is where human curiosity and rigour come in. But in the long term, I’d much rather see a world where humans become as economically irrelevant as horses. I’d rather we all flourish and have more time for play – let the machines do what we today call work.