Slots and Bots

I’ve been consulting with quite a few companies lately and helping them with their “bot” and “messaging” strategy. These forward-thinking companies recognize that there is a titanic shift coming — some say a once-in-a-decade paradigm shift — from websites and mobile apps to messaging-based collaboration.

The evidence for this shift is overwhelming. US Adults spend 23 hours a week in messaging services. Facebook Messenger has more than 1.2 billion users worldwide. 9 out of 10 of consumers report that they would prefer to message with a business rather than have a phone call. So, businesses of all sizes are flocking to messaging-based solutions in an effort to please customers, convert more sales, capture market share, and reduce costs. Companies using Chatbox® for messaging have driven 30%+ increase in bookings. In short: messaging works.

One of the keys to offering messaging solutions at scale is automation. For a business of any size, automation is a must: the costs of manually composing and replying to every in-bound message would be just too overwhelming. So, I’ve been talking to customers about automation through Artificial Intelligence (AI) and Natural Language Processing (NLP).

The mismatch between expectations and current capabilities for AI/NLP are immense. Let’s take a simple, illustrative exercise. Your company convinces a customer to text you to “Book an Appointment”. The AI is supposed to parse the inbound message from the customer (“I’d like to book an appointment, please”) to determine the customer’s intent: “Schedule Appointment.” With ample effort and training, the NLP will be quite good at Intent Discovery — expect about 80–90% accuracy.

After Intent Discovery, the bot will need to discover the date and time for the appointment through a series of questions: “What time would you like to schedule your appointment?” In NLP parlance, this is known as “slot filling.” Notice that while the bot is looking for a specific answer, ideally in a specific format, the customer answers in unconstrained, free-form text.

This is where things get dicey, chatbots fail, and customers get frustrated.

Let’s take a took a look at two of the “best” NLP engines available: Google’s API.AI and Amazon’s Alexa. Here’s how they do at filling a simple “time-slot”:

Time-Slot-Filling for Leading NLP Engines

Every human would be able to look at the test phrases and accurately determine the time the customer wanted to be called. While both AI engines were able to understand simple phrases (“noon”, “1030am”, and “in 3 hours”), they faltered when the answer was more complicated (“in a few minutes”, “immediately”). Phrases that are easily interpreted by humans like “after lunch” were missed, as expected.

The situation for chatbots is even more dire due to the laws of mathematics. Once you string a few of these slot-filling exercises together serially, your chatbot will quickly encounter the scourge of combined probability:

Chatbot Slot-Filling Funnel Analysis

As you can see, as the number of slots increases, the combined chance of success falls dramatically. Combined probability for slot-filling — the “chatbot success funnel” — is a beast. When I show this chart to executives looking to deploy chatbots to solve some sort of problem, they immediately grasp the gravity of the situation.

Even dramatic improvements in NLP over the coming years — say from a 70% success rate for slot-filling to a 90% success rate actually won’t help much. At a 90% success rate, the chance that NLP would succeed filling four slots is around 65% — a third of the time these mythical future bots will just fail with “Sorry, I didn’t understand.” This problem isn’t going away anytime soon.

Here’s the bottom line: using natural language processing for slot-filling is destined to create sub-optimal experiences now and in the foreseeable future.

At Chatbox, we’ve developed a solution to this slot-filling conundrum that we call Instant Apps. We allow bots to discover intent through NLP, but then transition to structured data capture (forms, file exchange, checklists, wizards, surveys) for slot-filling. Instant Apps can hand that data back to the bot inline within the conversation with near 100% accuracy. Bots can function more as intent-discovery mechanisms, leaving the hard slot-filling to the Instant Apps.

Why do Instant Apps work where natural-language based slot-filling can’t? Structured data, at its essence, is data in the form you specify. So instead of asking a customer when they’d like their next appointment in a free-flow messaging format (opening up the floodgates to any kind of answer a customer can dream up), Instant Apps limit your customers’ answers to a format that can be accurately interpreted. Whether it’s serving up a few choices, asking the customer to specify the month, day and time in different fields, using a graphical interface like a calendar with time slots, or having the customer choose from a few pre-determined buttons, Instant Apps collect the data you need in structured form so your chatbots can get the data they need in a form computers are great at using. Problem solved.


Phil Gordon
Founder, CEO

More News


Iowa Caucus App Reimagined

Mobile Transformation Image

Insurance: Mobile Customer Engagement Could be More…Mobile

Enterprise Messaging Brad Epker Interview Image

Enterprise Messaging: What’s the Chatbox Approach?

Don't miss a thing. Stay up to date on the latest tips, hacks, and news.

“The impact to our admissions process was instant and measurable. We saw an increase in conversion in less than a week, and the impact was so indisputable we cancelled testing and rolled it out globally.”

Edward Nevraumont, CMO/CRO, General Assembly