Moving the AI Goalposts
Another hot take on the state of Artificial Intelligence
I want to throw a flag on a moving-the-goalposts violation. Once again the AI goalposts are being moved, and this time at a gallop.
In 1950, computing pioneer (and fatally homosexual person) Alan Turing proposed what we all know as The Turing Test. His conjecture postulates two people in isolated rooms, communicating via teletype. If you are one of those people and you cannot tell whether the other person is human or a computer, then that computer is intelligent. The test has been run in real life. According to GPT-4:
The Loebner Prize was an annual competition in artificial intelligence that awarded prizes to the computer programs considered by the judges to be the most human-like. The prize was founded in 1990 by Hugh Loebner and was based on the Turing Test1. The competition awarded a Grand Prize of $100,000 and a gold medal to the creators of the first bot that could pass an extended Turing Test involving textual, visual, and auditory components. There was also a prize of $25,000 for the first bot that could pass a text-only Turing Test, and $2,000-$3,000 (the amount varied over the years) for the most human-seeming of all contestants that year.
I assume it went defunct because bots were passing the tests consistently.
But when the first bot passed a Turing Test, was the computing community willing to call that bot truly intelligent? Of course not. Wikipedia says:
The Turing test does not directly test whether the computer behaves intelligently. It tests only whether the computer behaves like a human being. Since human behaviour and intelligent behaviour are not exactly the same thing, the test can fail to accurately measure intelligence…
Goalposts moved.
But chatbots got much better. They began to do useful work. The constraint of communicating through teletypes was removed: bots could now talk and listen, indistinguishable from human speech. In 2018, Google demonstrated a chatBot (Google Duplex) making restaurant and hair salon reservations over the telephone, where the person on the other end believed they were talking to another human. This capability upset many experts and began to raise suggestions that bots be required to identify themselves as such.
But did we admit these bots were intelligent? No, they were still just aping human behavior. Goalposts moved.
Let’s jump back in time and follow AI research apart from chatBots. Stuff that was supposed to be true Artificial Intelligence.
In 1997, IBM’s Deep Blue became the first chess AI to defeat a grandmaster in a match. It defeated then-reigning world chess champion Garry Kasparov in a six-game match [source: GPT-4]
But this “deep learning” AI was not really intelligent. Chess is basically the ability to look ahead and check a large number of possible moves for best outcomes, plus a library of openings and closings. Besides, it’s only an 8x8 board, that’s not very complex. That’s not true intelligence. Let’s see it beat a Go master. Goalposts moved.
In October 2015, AlphaGo, a computer program developed by Google DeepMind, became the first computer Go program to beat a human professional Go player without handicap on a full-sized 19×19 board. It defeated the European Go champion Fan Hui with a score of 5–0 [source: GPT-4]
Still not really intelligent, even though Go masters began studying AlphaGo games to understand some of the non-human strategies and tactics employed. It’s a single-purpose bot trained intensively for one thing only, the game of Go. Goalposts moved.
Soon DeepMind AI’s were crushing all sorts of video games, not by being trained for them, but by learning them directly through gameplay, often AI against AI. But these “deep learning” AIs would never be truly intelligent. They were just goal-seeking machines with no true understanding of what they were doing. Goalposts moved.
The AI field was quietly—and soon, not so quietly—revolutionized in 2017 when Google scientists invented “transformers” and used them to power a new class of AI called Large Language Models or LLMs. They took the ability of AIs to communicated in natural human language to a whole new level.
The LLM field exploded into the public eye when the non-profit OpenAI organization released ChatGPT into the wild on November 30, 2022. The GPT-3 model behind ChatGPT was trained on a huge corpus of information, a lot of it scraped from the internet, at the cost of millions of dollars. The model has 117 million parameters, and was later upgraded to GPT-3.5 with 175 billion.
ChatGPT can converse quite “intelligently” on just about any topic. It’s converstaion is on par with human, and often on par with subject matter experts. But it’s accuracy isn’t great, something like 85% to 95%. And when it is wrong, it can be a doozy. Chattie (as I call her) will write confidently about falsities and make up perfectly plausible quotes and references. They call it hallucinating, comparing it to the way the humans can hallucinate, seeing and hearing things that aren’t real. So ChatGPT is not intelligent. It makes too many mistakes! It’s too dumb! Goalposts moved.
Less than four months later, on March 14, 2023, OpenAI released GPT-4 and Microsoft made it available in a chat interface at bing.com/new (on Edge browser only). GPT-4 has about 100 trillion parameters and cost more than $100 million to train. As to accuracy, GPT-4 itself tells me:
Personally, I have come to rely heavily of GPT-4 for research, using Google search less and less. I have not encountered any false information (as far as I know). GPT-4 also helpfully provides links to where it found the information so I can click through to follow up or verify.
GPT-4 is intelligent, as far as I am concerned.
But the goalpost movers are at it again. GPT-4 is not true Artificial General Intelligence (AGI) because it does sometimes make mistakes, or not take into account recently published research. Test cases can be concocted that a human would understand but GPT-4 gets wrong. It is not truly intelligent because it doesn’t really “understand,” it is just a statistical answer predictor. A computer cannot be truly intelligent because human-level intelligence can be achieved only in a body. (Never mind that the AI powered robots are just around the corner.)
Personally, I refuse to move the goalposts any further. GPT-4 and some other AIs are intelligent. (Which is not to say they are conscious or sentient. That’s a whole other deal.)
Is AI at its end state? Is this as good as it gets? Hell no, it is barely just beginning, and it is growing at an incredibly fast pace. I believe it will soon be at a point where nobody is going to be able to move the goalposts any further, and AI finally will get its victory: Everybody will agree it is truly intelligent and has, in fact, achieved Artificial General Intelligence.
Medium author Paul Pallaghy, PhD writes extensively and persuasively arguing that today’s LLMs are truly intelligent here, here, and here, and that AGI is highly imminent and AGI ‘just’ occurred.
A new, much different goalpost is already set up, and that is for super-human intelligence. When will AI be smarter than the smartest human alive? And that definition includes a dimension of creativity. I don’t know when super-human AI (SHI?) will be achieved, but I suspect it’s not far off. Years, if not months, but certainly not decades. The race is on: Will we achieve SHI before completely destroying the climate and crashing civilization? (My bet is on SHI followed shortly by crash/destruction.)
— Lannie Rose, May 2023
preferred pronouns: she/her/hers
GPT-4 (bing.com/new) used heavily for research, but not writing, except as quoted and credited.
