Learning a language can’t be that hard — every baby in the world manages to do it in a few years. Figuring out how the process works is another story. Linguists have devised elaborate theories to explain it, but recent advances in machine learning have added a new wrinkle. When computer scientists began building the language models that power modern chatbots like ChatGPT, they set aside decades of research in linguistics, and their gamble seemed to pay off. But are their creations really learning?
“Even if they do something that looks like what a human does, they might be doing it for very different reasons,” said Tal Linzen (opens a new tab), a computational linguist at New York University.
It’s not just a matter of quibbling about definitions. If language models really are learning language, researchers may need new theories to explain how they do it. But if the models are doing something more superficial, then perhaps machine learning has no insights to offer linguistics.
Noam Chomsky (opens a new tab), a titan of the field of linguistics, has publicly argued for the latter view. In a scathing 2023 New York Times opinion piece (opens a new tab), he and two co-authors laid out many arguments against language models, including one that at first sounds contradictory: Language models are irrelevant to linguistics because they learn too well. Specifically, the authors claimed that models can master “impossible” languages — ones governed by rules unlike those of any known human language — just as easily as possible ones.
Recently, five computational linguists put Chomsky’s claim to the test. They modified an English text database to generate a dozen impossible languages and found that language models had more difficulty learning these languages than ordinary English. Their paper, titled “Mission: Impossible Language Models (opens a new tab),” was awarded a best paper prize at the 2024 Association of Computational Linguistics conference.
“It’s a great paper,” said Adele Goldberg (opens a new tab), a linguist at Princeton University. “It’s absolutely timely and important.” The results suggest that language models might be useful tools after all for researchers seeking to understand the babbles of babies.
To read more, click here.