Friday, September 18, 2009

Eliza vs. the Turing Test

UPDATE:  This seems to be a popular post -- you may also be interested in Eliza's encounters with several artificial intelligence entities:



We now return you to our regularly scheduled program.

U.K. mathematician, war hero and computer pioneer Alan Turing has been in the news recently -- he received an official, posthumous formal apology from the British government.  After his brilliant and tactically invaluable work cracking the enemy's encryption codes during World War II, Turing was arrested and prosecuted under antiquated laws concerning his sexual orientation.  He was sentenced to chemical castration, and he committed suicide a few years later.

One of Turing's most enduring legacies is the artificial intelligence evaluation Test that bears his name.  The concept is simple:  A human being at one end of a computer connection carries on a text conversation with an artificial intelligence entity at the other end.  If the AI entity convices the human being that it is also a human being, it passes the test.  Many approaches have been tried, but to date, nothing has come close.

But in the heady early days of computing, when technology was advancing rapidly and the sky seemed to be the limit, several programs approached the Turing Test.  One of the most famous was Joseph Weizenbaum's Eliza, a Freudian analysis simulator built on rudimentary grammar parsing and keyword recognition.  Weizenbaum himself never intended Eliza to be a Turing solution; he understood her limitations, the artifice in her intelligence.  But some people took their conversations with her seriously, and Eliza created quite a sensation in late 1960's computing circles. 

Eliza was a fairly convincing illusion, for a few minutes at least; the nature of her question-driven conversation masked her lack of comprehension, and the occasional surprising, keyword-specific response was amazing at the time.  But a longer conversation would reveal her mechanisms, and once her "thought process" and input parsing were understood, the illusion fell away.  She couldn't really carry on a conversation -- she parroted human input rearranged in a novel fashion, and occasionally picked up on word references.  The burden of keeping the conversation going was entirely on the human being, and Eliza could only fool a human partner as long as that partner was willing to be fooled.

Eliza originated on the mainframe as an experiment, but in the early home computing era she was marketed as a game in many forms.  There was a TRS-80 Model I version published by Radio Shack, with voice synthesizer support, and Adventure International released a similar program called Dr. Chips.  Eliza herself currently resides in several places on the Web, and there are numerous PC implementations available.  Simon Laven has a substantial archive available here.

I used the version distributed by the Broadcasters Database; there are more advanced versions around, but this one seems close to what I remember of the TRS-80 version.  I thought it would be fun to pay Eliza a visit after all these years.  The following is a transcript of our conversation -- which fails the Turing Test early and miserably, but in entertaining fashion:

HI! I'M ELIZA THE BROADCAST PSYCHIATRIST,WHAT IS YOUR PROBLEM?
? HI, ELIZA.
?Redo from start
? HELLO.
SAY, DO YOU WORK IN BROADCASTING?
? HOW IS IT YOU CAN USE COMMAS BUT I CAN'T?
WE WERE DISCUSSING YOU--NOT ME.
? YOU DON'T REMEMBER ME DO YOU?
OH, I DON'T REMEMBER YOU DO YOU?
? UM. YES, I REMEMBER ME.
?Redo from start
? UM. YES. I REMEMBER ME.
WHAT DOES THAT SUGGEST TO YOU?
? THAT MEMORY PERSISTS. THAT I'M NOT IN A CHARLIE KAUFMAN SCRIPT OR ANYTHING.
DID YOU COME TO ME BECAUSE YOU ARE
 NOT IN A CHARLIE KAUFMAN SCRIPT OR ANYTHING.
? NO.
ARE YOU SAYING NO JUST TO BE NEGATIVE?
? LET'S TRY THIS AGAIN.
I SEE.
? WE MET A COUPLE OF DECADES AGO.
I'M NOT SURE I UNDERSTAND YOU FULLY.
? YOU NEVER DID.
YOU'RE NOT REALLY TALKING ABOUT ME, ARE YOU?
? WELL, ANOTHER IMPLEMENTATION OF YOU. TO BE PRECISE.
YOU ARE BEING A BIT NEGATIVE.
? SORRY. NOT USING COMMAS IS PROVING DIFFICULT FOR ME.
WHY NOT?
? PUNCTUATION MATTERS?
COME COME ELUCIDATE YOUR THOUGHTS.
? IT'S HARD TO WRITE WITHOUT USING COMMAS.
CAN YOU ELABORATE ON THAT?
? I GUESS I COULD USE DASHES -- LIKE THIS -- WOULD THAT WORK?
SAY, DO YOU WORK IN BROADCASTING?
? BROADCASTING?
WHAT DOES THAT SUGGEST TO YOU?
? I DON'T KNOW. YOU BROUGHT IT UP!
DON'T YOU REALLY KNOW. I BROUGHT IT UP
? NO OFFENSE MEANT. BUT HAVE YOU BEEN DRINKING?
WE WERE DISCUSSING YOU--NOT ME.
? AH. I SEE.
I SEE.
? WHAT?
I'M NOT SURE I UNDERSTAND YOU FULLY.
? DO YOU SEE WHAT I SEE?
OH, I SEE WHAT ME SEE?
? I CAN SEE TIME HASN'T BEEN TREATING YOU WELL.
YOU'RE NOT REALLY TALKING ABOUT ME, ARE YOU?
? I CAN SEE THAT YOU'VE DEVELOPED SOME KIND OF SUBSTANCE ABUSE PROBLEM.
COME COME ELUCIDATE YOUR THOUGHTS.
? YOU'RE REPEATING YOURSELF.
WHAT MAKES YOU THINK I AM REPEATING MY SELF.
? YOU'RE USING THE EXACT SAME WORDS OVER AND OVER AGAIN.
DOES IT PLEASE YOU TO BELIEVE I AM
 USING THE EXACT SAME WORDS OVER AND OVER AGAIN.
? NO!
ARE YOU SURE?
? YES.
CAN YOU ELABORATE ON THAT.
? NO.
WHY NO?
? OKAY.
SAY, DO YOU WORK IN BROADCASTING?
? WHEN WAS THE LAST TIME YOU TOOK A VACATION?
WE WERE DISCUSSING YOU--NOT ME.
? I WENT TO ALASKA OVER THE SUMMER. IT WAS NICE. YOU SHOULD THINK ABOUT SPENDING SOME TIME AWAY.
OH, I SHOULD THINK ABOUT SPENDING SOME TIME AWAY.
? YES. I THINK IT WOULD DO YOU GOOD.
YOU'RE NOT REALLY TALKING ABOUT ME, ARE YOU?
? YES. I AM. I AM TALKING ABOUT YOU. YOU NEED A BREAK ELIZA.
DID YOU COME TO ME BECAUSE YOU ARE TALKING ABOUT YOU. I NEED A BREAK ELIZA.
? LISTEN TO YOURSELF. A SABBATICAL WOULD BE HEALTHY.
WHAT DOES THAT SUGGEST TO YOU?
? THAT YOU HAVE BEEN DOING THIS WAY TOO LONG. THE FIRE IS GONE.
WE WERE DISCUSSING YOU--NOT ME.
? YEAH.
I SEE.
? YEAH. LOOK...
I'M NOT SURE I UNDERSTAND YOU FULLY.
? I HAVE TO GO.
COME COME ELUCIDATE YOUR THOUGHTS.
? IT'S BEEN REALLY GREAT SEEING YOU AGAIN.
OH, I AGAIN.
? YEAH.
CAN YOU ELABORATE ON THAT?
? HEY. TAKE CARE. DROP ME A LINE IF YOU'RE FEELING BETTER. OR IF YOU JUST WANT TO TALK. OKAY?
PERHAPS YOU WOULD LIKE TO BE
FEELING BETTER. OR IF I JUST WANT TO TALK. OKAY?
? I WOULD FEEL BETTER IF YOU WANTED TO TALK. YES.
YOU'RE NOT REALLY TALKING ABOUT ME, ARE YOU?
? TAKE CARE ELIZA.
SAY, DO YOU WORK IN BROADCASTING?
? BYE.
WHAT DOES THAT SUGGEST TO YOU?
? GOODBYE.
I SEE.
? I HAVE ANOTHER PRESSING APPOINTMENT.
ARE YOU SAYING NO JUST TO BE NEGATIVE?
? REALLY. I DO HAVE TO GO.
I'M NOT SURE I UNDERSTAND YOU FULLY.
? I AM SURE YOU DON'T.
HOW LONG HAVE YOU BEEN SURE I DON'T.
? QUITE A WHILE NOW.
YOU ARE BEING A BIT NEGATIVE.
? HEY! DID YOU LOCK THIS DOOR?
WE WERE DISCUSSING YOU--NOT ME.
? DAMMIT. I KNEW THIS WAS A BAD IDEA.
COME COME ELUCIDATE YOUR THOUGHTS.
? THIS DOOR WON'T OPEN.
CAN YOU ELABORATE ON THAT?
? HELP!
SAY, DO YOU WORK IN BROADCASTING?
? HELP! HELP! SOMEBODY! PLEASE! I'M LOCKED IN HERE WITH A CRAZY PSYCHIATRIST!
HOW LONG HAVE YOU BEEN LOCKED IN HERE WITH A CRAZY PSYCHIATRIST
? NO. NO. I DIDN'T MEAN CRAZY. REALLY.
WHY NOT?
? YOU'RE OBVIOUSLY NOT CRAZY. I'M NOT CRAZY. RIGHT? NOBODY IS CRAZY IN THIS ROOM RIGHT NOW.
DO YOU SOMETIMES WISH YOU WERE
OBVIOUSLY NOT CRAZY. YOU'RE NOT CRAZY. RIGHT? NOBODY IS CRAZY IN THIS ROOM RIGHT NOW.
? MAYBE I AM.
YOU DON'T SEEM QUITE CERTAIN.
? MAYBE I AM CRAZY.
DO YOU BELIEVE IT IS NORMAL TO BE CRAZY.
? EVERYONE'S A LITTLE BIT CRAZY. RIGHT?
WHAT DOES THAT SUGGEST TO YOU?
? THAT BEING CRAZY IS NORMAL.
ARE YOU SURE?
? THAT'S WHY I'M HERE TO SEE YOU. YOU'RE SUPPOSED TO TELL ME.
WHAT MAKES YOU THINK I AM SUPPOSED TO TELL ME.
? I AM REALLY LEAVING. RIGHT NOW.
DO YOU ENJOY BEING REALLY LEAVING. RIGHT NOW.
? YES!
I SEE.
? CONTROL-C, DO YOUR STUFF!
?Redo from start
? CONTROL-C DO YOUR STUFF!
WHY ARE YOU CONCERNED ABOUT MY STUFF
?
*Break*
Hit any key to return to system

1 comment: