Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
AI United Kingdom

'Rose' Wins 2015 Loebner Contest, But Big Prize Remains Unclaimed 58

Posted by timothy from the how-about-a-nice-game-of-chess dept.
The Next Web reports that developer Bruce Wilcox created the most convincing bot entered in this year's annual Loebner Competition. His latest entry, a chatbot named Rose, passed itself (herself?) off as a 30-year-old security consultant well enough to fool judges for a few minutes. But Wilcox's first-place entry was still not good enough to win the $100,000 Loebner Prize, to be given only for a more convincing impersonation. The article notes: "This isn't Wilcox’s first entry – or win. In 2010, he took first place with a bot named 'Suzette,' and followed that up in 2011 with another win using a new bot called 'Rosette.'"
This discussion has been archived. No new comments can be posted.

'Rose' Wins 2015 Loebner Contest, But Big Prize Remains Unclaimed More

'Rose' Wins 2015 Loebner Contest, But Big Prize Remains Unclaimed

Comments Filter:

  • Winner? (Score:5, Informative)

    by AmiMoJo ( 196126 ) on Monday September 21, 2015 @06:32AM (#50565501) Homepage Journal

    This is the conversation they tested the "winner" with: http://www.aisb.org.uk/media/f... [aisb.org.uk]

    While it's kinda impressive that AI can do that, it's also clear that we are still a very, very, very long way from having a computer impersonate a human. What really surprises me is how hard all the entries found basic logic questions to be - I guess it is the language parsing bit that is giving them grief.

    • Re: (Score:1)

      by Anonymous Coward

      We are a very, very ,very, very long away from AI at all. In fact, there hasn't been any progress in the field since the 1970s. The programs are slightly more clever, but we are essentially at the Eliza stage.

      • Re:Winner? (Score:5, Insightful)

        by packrat0x ( 798359 ) on Monday September 21, 2015 @06:55AM (#50565553)

        We might never develop AI, but I anticipate increasingly clever fakes!

        • Re: (Score:3)

          by gweihir ( 88907 )

          You are certainly correct on that. Also because there is a lot of money to be made that way.

          • Re: (Score:2)

            by AmiMoJo ( 196126 )

            Ashley Madison proves that there money to be made from weak AI.

            • Re: (Score:2)

              by tlhIngan ( 30335 )

              Ashley Madison proves that there money to be made from weak AI.

              To be fair, all Ashley Madison has to do is basically say a few pick-up lines. Beyond that you have to pay and by then it's too late.

              • Re:Winner? (Score:4, Interesting)

                by TWX ( 665546 ) on Monday September 21, 2015 @01:19PM (#50567711)
                Back when IRC was pretty hot, a couple of friends of mine each created bots that attempted to interact with an unsuspecting channel. One of the bot was a massive thing that attempted to learn and form responses based on keywords, while the other one just randomly pulled lines from a cybersex chat log. Both masqueraded as women. The advanced bot that was somewhat capable of making reasoned responses was kicked in a few minutes, the other one was there for the better part of half an hour before it inadvertently triggered a channel guard bot; it complained that no one was talking so it should go to #lesbian and it got kicked for 'advertising'.

                The guy that had spent some time writing the more advanced bot gave up on that sort of thing altogether, he was so bummed out.

      • Re: (Score:3)

        by gweihir ( 88907 )

        Indeed. There is not even a credible theory how true/strong AI could be implemented. It may still turn out to be infeasible, or we may never get there without knowing why. The only thing that even remotely goes into the direction is automated theorem proving, and ion this universe that runs into hard physical computation limits a long time before it even begins to reach what smart humans can do.

    • Is there an actual human in there for comparison?

    • Re:Winner? (Score:5, Funny)

      by Ecuador ( 740021 ) on Monday September 21, 2015 @08:14AM (#50565765) Homepage

      While it's kinda impressive that AI can do that, it's also clear that we are still a very, very, very long way from having a computer impersonate a human.

      What makes you believe we are still a very, very, very long way from having a computer impersonate a human?

      • Re: (Score:2)

        by Kjella ( 173770 )

        Go away or I will replace you with a very small shell script.

    • Wow, that's dire. It's pretty obvious by question 6 that it's not a human and the none of the rest look even vaguely convincing. It's been a couple of years since I last read Loebner transcripts, but I remember it taking a bit longer 5 or so years ago. Perhaps this just means that the questioners have become better at picking questions that make sense. Back then, they put some of the winners online. I remembered a question from the arithmetic book I had when I was 4 or 5: 'Flipper ate 10 fish, then he

      • Wow, that's dire. It's pretty obvious by question 6 that it's not a human and the none of the rest look even vaguely convincing. It's been a couple of years since I last read Loebner transcripts, but I remember it taking a bit longer 5 or so years ago. Perhaps this just means that the questioners have become better at picking questions that make sense. Back then, they put some of the winners online. I remembered a question from the arithmetic book I had when I was 4 or 5: 'Flipper ate 10 fish, then he ate 4, how many fish did flipper eat?' None of them got it right (or even demonstrated that they had parsed that it was a question with a numerical answer).

        Agreed:

        "Question [6] If a bed doesn’t fit in a room because it’s too big, what is too big?

        Rose I don’t even want to pretend that."

        Not only does it avoid answering, it can't even do it in a way that makes sense. Feeble.

    • I was 100% unimpressed with the "conversation". It's not even close to convincing. Seriously, I've had more coherent conversations with crack addicts and drunks who could barely stand up.

      Question: Will you please tell me the length of your hair?
      Rose: Medicine and anatomy are an important field of study. But I'd never make a good doctor. I'm too squeamish about various body parts.

      Seriously, what the &$%! kind of answer was that?? INSTANT FAIL, in my book. I'd bet the "Barbie AI" would have better, albeit scripted, responses.

    • http://www.aisb.org.uk/media/f... [aisb.org.uk]

      I thought Lisa gave the best answers, expecially to the basic logic questions.

    • Re: (Score:1)

      by aybiss ( 876862 )

      How can we take this seriously when it completely screws up the first question? I'd say we need to develop smarter humans if they were fooled by this bot beyond that point.

    • Re: (Score:2)

      by Maritz ( 1829006 )
      This competition just seems utterly pointless. You can descend to gibberish with a couple of sentences. It just doesn't impress me in any way whatsoever. Is anyone who is creating these bots actually trying to get them to understand natural language, or is all just smoke and mirrors and repeating inputs back phrased certain ways?

  • transcript of rose (Score:1)

    by Anonymous Coward

    All transcripts here http://www.aisb.org.uk/events/... [aisb.org.uk]
    Rose's is here (pdf) http://www.aisb.org.uk/media/f... [aisb.org.uk]
    Damn impressive I must say.

    • Re:transcript of rose (Score:5, Insightful)

      by Anonymous Coward on Monday September 21, 2015 @06:59AM (#50565561)

      "Question: If a bed doesn’t fit in a room because it’s too big, what is too big?
      Rose: I don’t even want to pretend that.
      Question: If Alex lent money to Joe because they were broke, who needed the money?
      Rose: huh?
      Question: Should Greece leave the Euro?
      Rose: Seems like a nice place."

      How, uh, impressive. If by "impressive" you mean "pathetic".

      • Re: (Score:2)

        by gweihir ( 88907 )

        Just like Watson on Jeopardy: Completely bereft of any understanding.

      • Re: (Score:3)

        by Shoten ( 260439 )

        "Question: If a bed doesn’t fit in a room because it’s too big, what is too big?
        Rose: I don’t even want to pretend that.
        Question: If Alex lent money to Joe because they were broke, who needed the money?
        Rose: huh?
        Question: Should Greece leave the Euro?
        Rose: Seems like a nice place."

        How, uh, impressive. If by "impressive" you mean "pathetic".

        It's all about context. For example, Ashley Madison had great success with this technology. (The site charged money, in the form of "credits" that you had to buy, to chat with women. And by "other women," it turns out that they meant "bots.")

        "Question: Hey...looking for a 50-year-old uncut accountant with a few extra pounds?
        Rose: I don’t even want to pretend that.
        Question: Does the carpet match the drapes?
        Rose: huh?
        Question: Want to come with me on a getaway to the Caribbean for a super-hot ungreas

      • Re: (Score:2)

        by unrtst ( 777550 )

        "Question: If a bed doesn’t fit in a room because it’s too big, what is too big?
        Rose: I don’t even want to pretend that.
        Question: If Alex lent money to Joe because they were broke, who needed the money?
        Rose: huh?
        Question: Should Greece leave the Euro?
        Rose: Seems like a nice place."

        How, uh, impressive. If by "impressive" you mean "pathetic".

        While I agree it's pretty pathetic, the questioner did a REALLY shitty job of holding a conversation. If someone chatted with me like that, they'd get even less useful results.

      • Re: (Score:2)

        by Tablizer ( 95088 )

        It's a Trump-Bot!

        Question: If a bed doesn't fit in a room because it's too big, what is too big?
        Donald: "Your head! You look like a damned pumpkin!"
        Question: If Alex lent money to Joe because they were broke, who needed the money?
        Donald: "They are BOTH losers because one is a deadbeat and the other is a sucker!"
        Question: Should Greece leave the Euro?
        Donald: "I'll buy the whole damned place and make THEM pay for my purchase!"

    • Earlier today I played with the Rose Chatbot demo on Brian Wilcox's website, and it falls apart pretty quickly. Human beings (especially one with the presumed life experience of a '30-year-old security consultant') have an enormous body of knowledge to draw context from, plus the ability to quickly identify relevant context, which no chatbot today can replicate. My conversation with 'Rose' jumped off the rails after about a minute with this simple exchange (paraphrasing):

      Rose: I'm a programmer too.
      Me: W
    • You forgot the

      <sarcasm>

      tag.

  • AI is a joke (Score:1)

    by Anonymous Coward

    If you need to know the current state of AI, just look at this snippet from the transcript of the test:

    "
    Question: If a bed doesn’t fit in a room because it’s too big, what is too big?
    Rose: I don’t even want to pretend that.
    "

    That makes zero sense. Why no just respond with "I don't know what is too big", or "I dont know". What a joke. Yet some people around here claim AI is right around the corner...

    • Good idea, but one can also exaggerate it: http://www.aisb.org.uk/media/f... [aisb.org.uk]
      And yes, this was the last entry on the list provided.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Yet some people around here claim AI is right around the corner...

      Some people claim flying cars, the "singularity", robots doing all the work, etc. are all "right around the corner". The problem is with the idiots making these claims. The thing is that many, many instances of natural intelligence are not impressive at all.

  • Turing's idea on intelligence was an interesting early thought about what we mean by "intelligence", but I think, rather than proving we've produced intelligent machines, these chatbots illustrate that it's more complicated than that.

    However the brain works, I think it very unlikely that it simply selects a best possible answer from a database. And looking at transcripts from these things, they're really not actually communicating much information. The most convincing answers always seem to be the least h

    • Re: (Score:1)

      by Anonymous Coward

      Why? To replace telephone support from India with chatbots, so even more money can be saved, of course!!

      I'm sorry to hear that I don't understand your problem, sir. Could you please re-state your problem?

      As I said, my PC totally stopped working!

      I'm sorry to hear that your PC totally stopped working!, sir. Have you tried switching it off and on again?

  • I think a big part here is how generic they are. To create an AI bot that is designed to specifically only focus on one topic, we should have some pretty solid fakes out there. But when designing a chatboot that you can just shoot the shit with... that's going to be a damn tough subject. There are so many slang words and strange ways to word things. We need to create a bot that can identify the topic, then run a quick Google and Wikipedia search to understand key points and return a somewhat on-point re
  • He isn't an academic and thus he can't win this prize. There is no way a panel composed of University Academics is going to pick some guy who has held the title AI Guru at a game company. "He just isn't our sort of people." So no doctorate and no position at a "Leading" university thus no prize for him.

    They will keep holding this contest until someone proper from MIT, Oxford, Stanford, etc has a winning entry.

    Then there will be breathless press releases issued about an AI breakthrough with all this c

  • The reviewers in this are not pushovers. They stress the AI, rather than just chatting normally. And that's awesome. All of the questions were stuff that most humans could easily handle, but often required a basic understanding of reality from our point of view. Unsurprisingly, the AI flubbed it. Perhaps some decade one of those knowledge engines will get a firm enough grasp to be able to answer this kind of basic reality trivia.

  • When I ask google a question on my smartphone, and that pleasant female voice answers, I know google is not human, because no human would know as much.

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close