Eugene Goostman is a thirteen year old boy from Ukraine. Or so he would have you believe; he is actually a chatbot who made history this week by reportedly passing the Turing test. The procedure, conceived over sixty years ago by mathematician and wartime code-breaker Alan Turing, is designed to determine whether responses in a conversation are coming from a human being or a machine.
The judging panel was eclectic, to say the least, including Lord Sharkey, who led the campaign to grant Alan Turing a posthumous pardon over his conviction for homosexuality in the 1940s, and Robert Llewellyn, the actor best known for playing the robot Kryten in cult sci-fi comedy Red Dwarf.
Eugene, the creation of Vladimir Veselov, convinced judges approximately 33% of the time that he was human during a series of five minute keyboard exchanges. The widely accepted pass mark for the Turing test is 30%, although Turing himself never set this classification. This is just one example of the organisers adopting a liberal interpretation of the experiment that has led to widespread scepticism regarding this supposed breakthrough, which has been described as “historic.”
Kelly Oakes says “there’s no need to welcome our robot overlords (yet)”, as it is unclear whether the Eugene Goostman program did legitimately pass the test. According to Oakes, “the test relies on trickery rather than genuine intelligence and is also reliant on the judge.” The full BuzzFeed article includes commentary from Professor Murray Shanahan at Imperial College London, who believes that “the small number of judges is not enough to be representative of the average interrogator” and that calling this particular trial a success “trivialises Turing’s thought experiment (which is fraught with problems anyway).”
Matthew Sparkes at The Telegraph is similarly critical, stating: “Using the limited life experience, vocabulary and sophistication of language of a child is a slight smokescreen that could, perhaps, be used to mask a range of flaws… Secondly, it pretends to be a 13 year old child from Ukraine, where the first language is not English. But the contest was performed in England with largely English-speaking judges.”
So we might be a little on the premature side to be saying that a computer can convincingly think and respond in the same way as a person. And you may find the title of this article a tad misleading. But what can I say? I’ve dreamed of writing a headline like ‘Robot passes for human’ my entire life; I hope you’ll forgive the indulgence?