Despite the death of its personal AI assistant M, Facebook hasn’t given up on chatbots just yet. Over the past couple of years, it’s slowly improved what its artificial agents can do, but their latest challenge is something that can confound even the smartest human: making small talk.
You’d be forgiven for thinking otherwise because of their name, but chatbots can’t really chat. As researchers from Facebook’s FAIR lab explain in a pre-print paper published this week, they fail at this task on a number of levels. First, they don’t display a “consistent personality,” sticking to the same set of facts about themselves throughout a conversation; second, they don’t remember what they or their conversational partners have said in the past; and third, when faced with a question they don’t understand, they tend to fall back on diversionary or preprogrammed responses, like “I don’t know.”
Even with these constraints, chatbots can be engaging. (See, for example, the famous ELIZA bot from the 1960s, which acted as a rudimentary therapist by relying on stock phrases like “How do you feel right now?”) But the goal now is not just interrogation, but conversation; to try to recreate this attribute, researchers have turned to deep learning. This means that instead of mapping out preprogrammed questions and answers, chatbots are taught by looking for patterns in large datasets.
So far this has got some good results, but one challenge now, say Facebook’s researchers, is getting the right data to begin with. Many contemporary chatbots are trained on dialogue taken from movie scripts, for example. And this means that when they’re having meandering conversations (rather than directed dialogue for tasks like customer service) they tend to come up with some odd non-sequiturs.
The clever thing about Persona-Chat is that the idle talk it contains isn’t just random. To give some structure to the data, and to address the challenge of making chatbots with personality, the Mechanical Turk workers were asked to design their own character to guide their dialogue. This meant coming up with five basic biographical statements and using them as topics of conversation. So, for example, one persona was based around the following facts: “I am an artist. I have four children. I recently got a cat. I enjoy walking for exercise. I love watching Game of Thrones.”
It’s not exactly Tolstoy, but it’s a start.
This data was used to train neural networks used for existing chatbots, with the results then assessed by another group of Mechanical Turkers. In each case, they were asked to conduct a conversation with the persona-driven bot, and compare it with both other chatbots and humans.
Interestingly, though, while the persona chatbot scored well on fluency and consistency, test subjects said they found it less engaging than chatbots trained on movie dialogue. Facebook’s researchers offer no explanation for this, but perhaps because of the constrained nature of the constructed personas (each one defined by just five biographical statements), the bots soon ran out of topics to talk about.
Still, the research points to something that’s almost common sense. If you want to have an interesting conversation with someone — even a virtual someone — then it helps if they have a personality, including likes and interests. As virtual assistants like Siri and Alexa become more integrated into our lives, we can expect their creators to build out their characters. The problem then is will we get along?