Be Your Own LLM | Customers, Etc.
Tools like ChatGPT are powerful, but know where you add unique value.
If you read a modern translation of the works of St. Augustine, the fifth century bishop of Hippo, it’s common to see references to biblical texts scattered throughout. Depending on the translation, you’ll often find references to the chapter and verse from scripture directly in the text.
The original texts of Augustine, of course, didn’t have these references. The systematic division of the bible into chapters and verses didn’t happen until the 13th and 15th centuries, respectively. When Augustine quoted scripture, he didn’t use quotation marks or italics. It was just there.
Augustine quoted scripture so frequently in his writing that’s it’s sometimes difficult to tell where the quotes from scripture end and his own personal thoughts begin. It’s like he had internalized the scriptures and it just flowed out of him, almost as if he had it all memorized.
That’s probably because he did have it all memorized.
Just a few pages ahead in the book pictured above, Augustine writes:
Now I arrive in the fields and vast mansions of memory, where are treasured innumerable images brought in there from objects of every conceivable kind perceived by the senses…. Sojourning there I command something I want to present itself, and immediately certain things emerge, while others have to be pursued from some time and dug out from remote crannies.
It was common in classical rhetoric to memorize large bodies of text verbatim and that’s very likely what Augustine is talking about. You can read entire books that discuss both the history of memorization and the technique itself.
Predicting the next word
This post is called “Be Your Own LLM” so let’s start connecting this back to things like ChatGPT and Large Language Models (LLMs). Tim Berners Lee, the guy who invented the World Wide Web, wrote a really great post called Large language models, explained with a minimum of math and jargon. It’s a very good primer, but it’s also very long, so for a quick summary, let’s just ask ChatGPT:
ChatGPT and other large language models (LLMs) operate through a process that involves training on massive amounts of text data and then generating responses to user inputs based on patterns and information in that data. Initially, during the training phase, the model learns by adjusting its internal parameters to predict the next word in a sentence, given all the previous words in it. It does this by analyzing countless sentences and gradually improving its predictions.
When an LLM is trained on a large amount of data, is able to produce a response that demonstrates it “understands” the data. It doesn’t really understand anything—it’s a computer—but it tries to look like it.
Finding work the flows out of you
I remember observing something interesting when I started interviewing people for the support team in my first manager role back at Trello. At the time, I was trying to create a writing exercise for customer support that mimicked Joel Spolsky’s Guerrilla Guide to Interviewing for software developers. In the method I ended up using, I would come up with a handful of fake support questions of increasing difficultly and then ask candidates to write responses in real time.
For candidates who hadn’t worked in customer support before—even those with impressive resumes1—they would often struggle to get through the exercise. They could usually get through to the end, but it wasn’t effortless, and the work wasn’t always great. It needed work.
On the other hand, when I encountered people who had spent significant time in a support queue, they would positively fly through the exercise. They would finish in half the time, produce a near perfect response, and say “that was fun!”. For them, the work was easy. Sometimes they would ask if the exercise was even helpful because of how easy it was. (It was. Very).
What was surprising to me was how frequently people didn’t realize they had acquired this skill and how valuable it was. It turns out being able to do valuable things effortless is quite valuable indeed.
Be Your Own LLM
Sometimes I wonder about the work that I do and what aspects of it will eventually be replaced by an LLM. The other day I wrote an email that communicated an update on various projects I had been working on and new ones I was starting. Unlike my usual weekly updates, it was surprisingly challenging work. I had to prioritize work streams and justify complex decisions. It was slow getting started, but I eventually got into the flow and was able to produce something that could communicate a detailed plan that was both succinct and clear.
When I think about Augustine writing The Confessions, I’m amazed at how he was able to pull in different passages of scripture and tie them into a coherent train of thought. He didn’t have a computer or even printed books (though he might have occasionally had access to written manuscripts). It just flowed out of him because it was in his memory.
LLM tools are going to continue to improve and become more useful. Work will flow effortlessly out of these tools, likely replacing humans in certain functional areas along the way. The question we need to ask ourselves is: what areas do I have experience and expertise—even if I don’t realize it yet—where work flows out from me effortlessly? Where am I “becoming my own LLM”?
This is worth a footnote. Customer support was often seen as a way to “get in” to a small but promising startup. The idea was that you would answer tickets for a while and then figure out how to jockey your way into a product management position later on.
Although I worked with several customer support folks who went on to be excellent PMs, I never cared for people interviewing for customer support roles with an explicit intention to “get out of” support in the near-ish future.