Can AI learn impossible languages?

If you've used artificial intelligence (AI) chat apps like ChatGPT and Gemini, chances are you've been impressed with how human-like they are in conversations. It's no surprise that people are using them as companions, therapists, and even life partners¹. Keeping aside moral considerations, this raises an important question: is AI able to do this because it uses langauges like humans, or is it producing speech that sounds natural but is generated in a completely unnatural, non-human way? I tackled this question in my Master's dissertation and in this post I'll describe how I did so and the interesting results I found. Strap in!

Similarity in learning languages

Why does similarity to humans in the usage of language matter? One compelling reason is that language is the most widely used medium of expressing knowledge, which is also why we find conversing with them so human-like despite their occasional basic logical errors. Understanding the extent of this similarity could therefore tell us more about the capabilities of current AI design to understand, express and potentially surpass human knowledge.

On one level, this question is trivial to answer: of course AI doesn't use language like us, they are machines without any biological substrate. Most AIs today are based on large language models (LLMs), which takes enormous amounts of time, training data, and energy before they are even capable of using language on par with a 5 year old human child². However, while their physical features are clearly non-human, what I'm interested in finding out is if they have similar methods of learning a language like humans. For instance, are they learning to reason about grammar rules and subject-verb-agreement as they produce text? Answering this question requires an ability to compare the language learning strategies of two different entities, along with a more precise notion of what similarity means and how to measure it.

How do you meaningfully and qualitatively compare the learning process of two entities while ignoring external factors? It turns out humans have been doing this for decades and they call them exams. Universities and companies have standardised tests and interviews where they try to select humans who are similar to their 'ideal' candidate (e.g presitigous instituitions look for people who are smart and able to think critically). The scores on exams with a variety of questions, or interviews with multiple rounds, are used as a proxy for how similar they are to their preferred candidate, and this could be the first model for defining and testing similarity: 2 entities are similar in their reasoning of a task if their performance on it is similar.

Consider applying this model to learning mathematics, and let's say we observe two students solving a difficult math problem correctly but using two different approaches. It's reasonable to draw conclusions on their mathematical abilities, but this result alone doesn't guarantee that they used the same mathematical techniques. For instance, one could have used a calculator while the other didn't, and this could mean that they learned mathematics in two fundamentally different ways because they are used to thinking about solving problems with different constraints. This also means these two students have different limits on the kinds of problems they can solve, since most humans cannot do large calculations that require a calculator mentally. Therefore, if we wanted to comprehensively test for a similarity in their mathematically abilites, we should include a wide variety of problems, including ones that can only be solved with a calculator. This updates the eariler model of similarity as: 2 entities are similar in their reasoning of a task if their performance is similar on a wide variety of meaningfully different tasks.

In the case of language understanding and generation, the ability to recognise grammatical sentences is a relevant test. LLMs have shown that they are capable of producing grammatical sentences when talking to humans, evidenced in the various chat apps, but this can be tested more formally. Linguists have designed a simple but effective task to test for grammatical knowledge: present two nearly identical sentences where one is grammatical and the other isn't, and check if a subject (human or AI) is able to correctly pick the grammatical one. We'll call each one of them a BLiMP task ³, and you can try a couple out for yourself below:

Each BLiMP task above chose a different type of word to differ, and thus tested a different kind of grammatical rule. By varying the word that differs and making hundreds of these pairs, we can test a wide variety of grammatical phenomena and use them to test for similarity in language reasoning. However, like in the earlier example of mathematics, similarity in performances may not rule out differing underlying processes and this means that these tests are still not discriminative enough. What if we made the tests impossible for humans?

Impossible Languages

Human languages are vast and many of them share common properties, but linguists have found it hard to prove the existence of a universal set of rules followed by all of them. This makes the notion of an impossible language tricky to define, especially compared to how trivial it is to find impossible examples in other fields like physics or biology (e.g it's impossible for an organism to exist that produces more energy than it consumes because it violates the fundamental laws of thermodynamics). Nevertheless, a clever trick that linguists used to come up with impossible languages was to create grammatical rules that have never occurred in any observed human language⁴. For instance, consider adding a new rule in English that stated that after every 3rd word in a sentence, the rest of the sentence will be in the reverse order of regular English. Shown below are some English sentences and their 'impossible' counterparts, with the reversed portion displayed in italics:

Intuitively, such a rule feels weird because human brains don't think in terms of numerical word ordering when processing sentences. Linguists have hypothesised that is because they have adapted to learn grammar rules in a hierarchical sense⁵, where structures matter more than the absolute order of words. That is, we learn grammar rules that generally describe how words are placed relative to each other, not according to their exact position. If we came across text written in an alien language that contained such kinds of rules, we may be able to decode or translate it into a human language and understand it but we'd never find it natural to learn, and in this sense such languages are impossible languages. Putting the two ideas together, we can create a variety of BLiMP tasks in impossible langauges like so:

Testing and results

In practice, carrying out controlled experiments to compare how humans learn languages is tricky. After all, you can't wipe out a person's language memory and force them to try and learn a new impossible language (and besides, my university allocated budget didn't permit it). Luckily, computers don't have such constraints, so I trained transformer-based language models (a scaled down version of LLMs) to learn English and a variety of impossible languages before evaluating how well they learned its grammatical rules. I also compared them to human performance on English-based evaluations and to make these comparisons meaningful, they were all evaluated on canonical sets of BLiMP tasks.

My experiments surfaced two interesting results. First, the models performed worse overall on impossible language based tests when compared to the English ones, suggesting that they shared some similarities to humans when it comes to learning languages (prior work shows this as well⁶). Second, on certain subsets of related BLiMP tasks, the models performed very well even in the impossible setting and achieved scores similar to the highest human scores, suggesting that they possess capabilities that allow them to learn far more and better from impossible languages than humans can.

What does this mean for the original question of similarity in language understanding and learning? These models have shown that while they share some overlaps with humans, they are capable of learning things that we find unintuitive and difficult. This should add caution when extrapolating results of linguistic experiments based on models to humans, but more broadly it could mean that the path to a super intelligent being that is modelled on human intelligence might not be achievable using only the current transformer-based architecture⁷. Personally, this serves as a reminder that there is still a ways to go before AI is able to replicate the full extent of human capabilites.

If you are curious about the details on how I implemented these experiments, the caveats, and the limitations of this approach, you can read about it in my dissertation here: MSc Dissertation

If you liked this reading this post, let me know here