Thursday, August 02, 2012

Chatbot Battles Post-Match Analysis

The matches have all been played, the winners have been announced, and the prizes have been awared.  The first Chatbot Battles contest is over.  The winner was Davidswinton, an artificial intelligence created by an American botmaster, Jim Koch, who goes by the name of “Cybertronics”.  Hosted by the appropriately named Ai company of Tel Aviv, Isreal, Davidswinton is described as from the future, where “there exist a highly advanced robotic boy named David Swinton. He is a marvel of the 22nd century and hopes to become a real boy.”

Rounding out the top four bots were Skynet-AI by Ken Hurtubise, Talking Angela by Bruce Wilcox and Elbot by Fred Roberts.  Skynet-AI is a general purpose virtual assistant written in Javascript, with a very fast response time.   Talking Angela is an Android app from Outfit7, the same “outfit” behind the popular Talking Tom app.   Talking Angela also received an award in the Chatbot Battles for “best 15 minute conversation”.  Elbot, already awarded in previous chatbot contests, has been under development for years by Fred Roberts of Artificial Solutions.

Chatbot Battles was the brainchild of Steve Worswick, botmaster of the acclaimed Mitsuku bot.  Although Steve would have liked to compete in the contest, he graciously recused himself in the interest of running a fair competition.  We had a chance to catch up with Steve and ask a few questions about the Chatbot Battles.  Steve said toyed with the idea of making a chatbot competition for a couple of years.  He wanted to create a new contest that “was easy and accessible for anyone with a chatbot to enter”, compared with other bot contests and Turing tests that have a higher technical barrier to entry.

Unlike the Turing Test and previous chatbot contests, the emphasis in Chatbot Battles was bot vs. bot, as opposed to bot vs. human, competition.  Steve said, “The key difference [between the Chatbot Battles and other bot contests] is that the chatbots compete directly against one another in many separate matches rather than en masse against a judge.”   What made the contest really exciting was the large number of battles between the 41 competing bots.   According to Worswick, “The bots are arranged in a league/knockout format similar to the soccer world cup which built excitement as the league placings altered daily with each match.”

For Steve, the best part of the contest was thinking up the questions that the judges asked the entrants. He also enjoyed meeting and chatting with the large number of competing bots.  His biggest challenge was keeping the contest running to schedule.  “120+ battles to run in a few weeks proved to be too much for myself and the judges and so the schedule unfortunately slipped a little,” he said.   Some of the contestants appear to have missed the memo about Good Sportsmanship.  Steve said, “I had emails from the league draw stage right through to the final few matches complaining that the draw was fixed, humans were posing as bots, I was giving certain bots an easy ride, the questions were unfair, I was biased against bots who had beaten me in the past and so on and so on.”   At one point the complaints became so frustrating that he considered stopping the whole contest.

Thank goodness Steve persevered.    He created, organized and ran the best chatbot contest to date.   And despite the negativity, Steve has a positive attitude about the future of his contest. “This was the first year of the contest and I will probably do the whole thing again next year”, he said. “I had some great suggestions on things that could be improved or altered and I will take on board each of these suggestions. On the whole, it was great fun and a big success and I would like to keep it going for a few years. While running the contest, I had a few ideas myself which may lead to another chatbot competition and so watch this space!”


blogger templates | Make Money Online