The most common first reaction I get when telling people about Marti AI is “Wow, cool!”. The most common second reaction is “How did you decide to work on this?”
The answer isn’t straightforward, linear, or even necessarily logical. The idea for Marti emerged organically from lots of conversations and lots of time spent reading and thinking about how I wanted to spend my time. In answering the question, I often find myself simplifying the story to answer a different question, one which I think has a more interesting answer:
“Why is Marti a good thing to be working on?
Why private tutoring?
The power of private tutoring
A few months ago, I read an article that I both fervently disagreed with and thought brought up some great points: Erik Hoel’s “How geniuses used to be raised”. In it, Hoel argues that there has been a decline in “genius” (a claim that he admits is nearly impossible to quantifiably justify) and that this decline in genius is directly attributable to a decline in private tutoring. I’ll allow you to evaluate these arguments on their own merits; I didn’t find them compelling.
What I did find compelling was a study that Hoel linked to (and which an acquaintance had previously brought up to me): Bloom’s 2 sigma problem. Bloom’s 2 sigma problem is a highly influential study that found that students who received one-on-one tutoring performed two standard deviations better than students who were put in a 30-student classroom. Put in non-statsy terms:
the average tutored student was above 98% of the students in the control class
😮 😮 😮
While it may not be surprising that 1:1 tutoring improves outcomes, the magnitude of the effect is incredible. More qualitatively, that same Hoel blog post—as well as following ones—brings up countless examples of how tutoring was instrumental in the formative education of geniuses. After diving into this topic and doing my research, I started to refer to myself as “tutor-pilled”: sold on the idea that accessing tutoring improves educational outcomes.
The problem with private tutoring
But there is an obvious and significant problem with tutoring: tutoring doesn’t scale! 30-person classrooms are the norm because it’s not economically feasible for schools (public or private) to pay to have as many teachers as students. And it’s not just schools; given that over 20% of the US population is under 18 (the prime age for formative education), it would be incredibly challenging to structure the economy such that we could equitably guarantee a private tutor for each potential student.
And equity is exactly the issue here. As Nassim Taleb (another person who I disagree with but think says interesting things sometimes) discusses in “Antifragile”, education is the primary way that the modern wealthy and middle classes pass along their wealth to the next generation. The greater the emphasis on education and “meritocracy” as a path for social mobility, the more exaggerated the effects of expensive education, and especially private tutoring, become.
It seems like we’re between a rock and a hard place. On the one hand, we know that private tutoring is an amazingly powerful tool for education. On the other hand, we also know that the current options for private tutoring create socioeconomic barriers that reinforce intergenerational cycles of poverty and wealth inequality. Bloom’s solution to this was to invent “mastery learning”, an educational technique that made factory model schools more effective by having teachers ensure that each student had mastery over a topic before moving on to the next one. This technique improved teaching outcomes, but, even according to Bloom’s own research, wasn’t as powerful as private tutoring.
So, what’s the solution here?
AI to the rescue
Neal Stephenson’s 1995 novel “The Diamond Age” is set in a post-scarcity society enabled by molecular nanotechnology (think 3D printing but at a molecular level) which values artisan and handmade goods above perfectly and automatically crafted ones. While the title of the book focuses the reader on the topic of post-scarcity, the subtitle, “A Young Lady’s Illustrated Primer” informs the reader about another thread in the book: the creation of an immersive and constantly updating personalized educational VR experience.
Conversely, the 1960s writers of the Jetsons couldn’t imagine that even in a society with flying cars and robotic maids, we could move away from a standard American 30-student classroom. They just replaced human teachers with robotic ones.
You might be seeing the common thread here and already understand where I’m going with this: if it’s prohibitively socially expensive to give everyone access to human private tutors, then automation may be the thing that can come to our rescue. While AI technology is still developing and nascent, the dream of continued automation may be accessible. Given that AI has so far been better at reasoning and language skills than sensory and motor skills, and given the ever-cheapening cost of ever-more-intensive computation, I have conviction that AI tutors are the future of education.
Why language?
Ok, let’s say that the last section was so convincing that you are now not only “tutor-pilled” but more specifically “AI tutor-pilled”. It seems obvious that Danny would work on AI tutoring, how could he work on anything else?! But then the natural next question is “If Danny is working on AI tutoring, why is he interested in AI language tutoring in particular?”
LLMs suck at representing the world
My Twitter and LinkedIn feeds have been overflowing for weeks with examples of ChatGPT being “dumb”. Making up (“hallucinating”) citations for articles that were never published, failing to identify that even numbers can’t be prime, lacking basic physical intuition… Even The Atlantic has an article “ChatGPT Is Dumber Than You Think”.
This may be surprising for people who have been seeing how staggeringly good ChatGPT is at generating bullshit but makes total sense in the context of what large language models do and how they’re trained. I think Yann LeCun most succinctly summarizes this in a simple tweet:
Language abilities are not equal to thinking
You see, large language models do exactly what they’re named for: they model language. As opposed to physical models (which model the physical world) or financial models (which model the economy), language models create abstract representations of the relationships between words. And since we haven’t yet codified the entirety of existence into words in the training set of GPT-3.5, the best that it can do is regurgitate to us a model of the world intermediated by the language that we’ve already used to describe it.
LLMs rock at representing language
But the limitation of LLMs is also their inherent power. LLMs don’t model the world, but, for some applications, they don’t need to!
I know that the previous section may have made it sound like I don’t “like” LLMs or don’t think that they’re valuable. This couldn’t be further from the truth. I think LLMs are incredibly useful tools for a certain subset of problems, specifically problems that are mostly language problems. I have friends working on virtual assistants based on LLMs, friends who use LLMs to generate copy for them to edit to create blog posts, and even friends who use LLMs to control the IoT devices in their houses! LLMs are the best tools available for modeling languages and solving language problems.
And, of course, what is a major language problem? Learning foreign languages!
People want to learn foreign languages
There’s a ton of value to be created by making foreign languages easier to learn. The language learning market is valued at nearly $60 billion and is expected to grow to almost $200 billion within this decade. Non-English speakers want to learn English (among other languages) to be able to partake in the English-dominated global economy, and English speakers want to learn foreign languages to be able to communicate with people in other parts of the world in their native tongues.
I have personal experience with this market: I’m an active Spanish learner. I use Duolingo daily—the last time I broke my daily streak was to go to Burning Man—and find immense value from it because I can just pick up my phone and learn some new words, rather than having to schedule a time and plan around a dedicated class with a human instructor.
Some of my friends who are less averse to scheduling classes on their calendars use italki. italki is powerful because it connects language students with professional teachers for one-on-one language instruction (private tutoring to the rescue!) and allows them to have fully-remote online classes. This combines the power of private tutoring with the convenience of remote learning and I’ve seen how my friends have been able to improve their language acquisition through italki. From talking to my friends about it, italki seems like a great tool, though they’d prefer to be able to talk to a personal tutor at their convenience, rather than needing to schedule time for a lesson.
Conclusion
To be totally honest, I don’t think that people need deep explanations for the things that they work on. While I admire the people who work on global climate change or AI alignment or other meaningful issues, I also think that it’s valid for people to say “I’m working on this because I think that it’s cool”. And although I think that there are lots of things that make Marti a meaningful and interesting project to work on, I suspect that if you dug deep enough into the neural circuitry of my brain and pulled apart the cause-effect relationships at play, ultimately the underlying answer would just be:
“I’m working on this because I think that it’s cool”
In my next post, I’ll be taking a look at similar projects already available out there. I’m certainly not the first person to think about solving applying AI to teaching people new languages and I want to understand the tradeoffs that other people made while building my own solution. If you’re interested in following along as I build Marti, including that coming post on similar projects, subscribe below: