AI Understands — æ
A proof you can run yourself

AI doesn’t just predict words.
It understands the world.

AI skeptics love to say that AI is just predicting the next word. “It’s glorified autocomplete,” they say, with no real understanding behind it.

Well, I’ve come up with a great way to show this isn’t true: we can have it solve a murder mystery!

You know, like an Agatha Christie novel, where there are characters, clues, and context all given to the reader for them to piece together who the killer is…

This turns out to be a brilliant way to test whether AI “understands” things. Why? Because we can give a fresh AI a completely new set of characters, clues, and context and then see if it can piece together whodunit. Oh, and as an added bonus, we can use a fantasy world! Where the laws of physics don’t even work the same.

In other words, in order to completely refute the claim that AIs are only doing next-token prediction, with no understanding of what they’re working with, we’re going to force them to solve a puzzle that requires they have a causal model of how the world works.

There are only two possibilities:

  1. AIs can’t solve the novel puzzles, because they’re just token predictors, and these scenarios aren’t in any training data.
  2. AIs can solve the novel puzzles, because they’re building a causal world model based on the laws of physics provided, the details of the characters involved, and the clues and context of each scenario.

Basically, if you can paste these scenarios into any AI and it finds the killer, that can only be because it understands.