A team of six computer science students from Emory qualified for the final round of Amazon’s Alexa Prize Socialbot Grand Challenge, a global competition between universities to create a chatbot that advances the field of artificial intelligence. The 2021 Alexa Prize winner will be announced mid-August. The stake is a first prize of $ 500,000. In addition, $ 1 million in research funding will be awarded to the winning team if they meet the criteria for the “big challenge,” including their chatbot’s ability to engage in conversation with the judges for at least 20 minutes.
In addition to Emory, the finalists are the Czech Technical University in Prague; SUNY in Buffalo, New York; Stanford University; and the University of California at Santa Cruz.
The Emory team is led by graduate students Sarah Finch and James Finch, as well as academic advisor Jinho Choi, assistant professor in the Department of Computer Science. Last year, the trio led a team of 14 Emory students who took first place, winning $ 500,000 for their chatbot named Emora. They chose the name because it sounds like a female version of “Emory” and is similar to a Hebrew word for an eloquent sage.
This year, they’re stepping up the heat with an even more advanced version of Emora and new team members including graduate student Han He and undergraduates Sophy Huang, Daniil Huryn, and Mack Hutsell. All students are members of the Choi Natural Language Processing Research Laboratory.
“I am extremely proud to have such a talented student team,” says Choi. “It’s a group of highly motivated people with the right mix of diverse skills who come together at the right time. They are working to change the paradigm of conversational artificial intelligence.
“We use established technology, but take a revolutionary approach in the way we combine and execute dialogue management so that a computer can make logical inferences while conversing with a human,” adds Sarah Finch. “Ultimately, we make Emora even more flexible in the way she can interact with people.”
The annual Alexa Award, launched in 2016, challenges college students to make inroads into the design of chatbots, also known as socialbots, software applications that simplify interactions between humans and computers by allowing them to talk to each other.
As the finals approached, users of Amazon’s chatbot, known as Alexa, volunteered to test competing chatbots, which were not identified by name or university. The success of a chatbot was evaluated by user ratings. Teams used this feedback to continue improving their chatbots.
“The competition has pushed us to create something that works, based on immediate feedback from real users,” says James Finch. “It forces us to constantly focus on the right problems and find solutions. “
Sarah and James Finch, who tied the knot in 2019, are the ultimate IT couple. They met when they were 10 years old in a math class in their hometown of Grand Blanc, Michigan. They dated in high school, bonding around a common love for computer programming. As undergraduates at the University of Michigan, they worked together on a shared passion for programming robots to speak more naturally with humans.
They chose to come to Emory for their graduate studies because of Choi, an expert in natural language processing, and Eugene Agichtein, professor in the Department of Computer Science and expert in information retrieval.
At the start of the Alexa Prize competition in September, each team received a $ 250,000 research grant, Alexa-enabled devices, and other tools, data and support from Amazon. The Emory 2021 team had a head start with the winning chatbot Emora created in 2020, by a team of 14 students from the Choi and Agichtein laboratories. They deployed the existing Emora, which had already proven to be a good grade-scoring tool, to save them time to develop a new framework from scratch.
Emora was designed not only to answer questions, but as a “social companion”. At the height of the COVID-19 pandemic, the chatbot provided comfort and warmth for people interacting with Amazon’s Alexa-enabled devices, whether they wanted to discuss movies, sports and their pets or their concerns. for themselves and their families.
The strategy paid off when the Emory 2020 team achieved an average user rating of 3.81, beating second place Stanford University (with a score of 3.17) to win the top Alexa award of $ 500,000. Ultimately, however, none of the teams competing last year achieved a composite score of 4.0 from the final judges to win the $ 1 million Grand Challenge prize. A key obstacle to the big challenge is the requirement that the chatbot engage the judges for at least 20 minutes in most conversations.
Such a long, logical and free conversation between a computer and a human is a major challenge remaining in the field of artificial intelligence.
Computers learn to respond to a human’s questions or comments by receiving massive amounts of data about possible answers. The nuances, however, are often lost on machines. A logical response to someone telling you they bought a carton of milk, for example, would be very different from a logical response to someone telling you they just bought a house.
“Changing a few words in a sentence can give it a very different meaning, requiring a completely different reaction from the listener,” explains Choi. “The human brain is wonderful because it can explore all possible nuances of meaning in an instant and come up with an appropriate response. It’s a huge problem to design an algorithm that can do these kind of subtle sociolinguistic calculations in seconds.
While the original Emora was at the cutting edge of technology in 2020, it was still too limited to meet the big challenge. It was built on a platform the team calls Emora State Transition Dialogue Manager, a behavioral mathematical model similar to a flowchart and equipped with several natural language processing models. Based on what people say to the chatbot, the machine chooses which conversation path to take and continues to do so based on the odds. While the system is good for gossip, the longer a conversation lasts, the more likely it is that the system will miss a sociolinguistic nuance and the conversation will derail, diverting the logical thread.
The Emory 2021 team has made the bold decision to develop a whole new system for Emora, leveraging the areas of expertise of the team members. They based this year’s Emora on three types of frameworks to advance basic natural language processing technology, computational predicate logic structures, and probabilistic reasoning for dialogue management.
“To our knowledge, our approach has never been taken the way we try it,” says Sarah Finch.
In a race against time, Sarah and James Finch worked on the engineering of the new Emora system, as well as the design of logical structures and the implementation of associated algorithms. Undergraduates Huang, Huryn, and Hutsell focused on developing dialogue content and conversational scripts for integration into the chatbot. Graduate student He focused on computational analysis, including recent technological advancements, which is essential for translating unstructured natural language into computational structures that can infer logical outputs.
As the competition nears the final round, the team members continue to work nonstop, continually tweaking and improving the system.
“Everyone on the team is extremely dedicated,” says Choi. “We believe Emora represents a revolutionary moment for conversational artificial intelligence.”