Start building your own chatbot now

We expect a lot from Artificial Intelligence: it needs infinite knowledge and  the ability to perform infinite tasks, in real time. Real time doesn’t necessarily means instant, like x milliseconds as we often want in systems architecture.

In fact in a normal conversation it takes on average 1 second for people to respond. When we are talking, we have to understand the other person’s sentence, search for the relevant data  in our brain, and generate an answer. This takes time. According to Pr Paul Reber,

“Yet neurons combine so that each one helps with many memories at a time, exponentially increasing the brain’s memory storage capacity to something closer to around 2.5 petabytes” (1 pb = 1 million gb).

And we want AI to have the knowledge of 7 billion humans !

That’s why we should talk talk about real time rather than instant, and in order to process such a huge amount of data in no more than a second, we can use a microservices architecture. But we need more. We also need to identify “request intents”, get data ready, and even anticipate user requests.

A microservices architecture for real time

When one or more of our services are running out of resources (I/O, memory, disk space…), we need to add more power to this specific service without being forced to scale the whole application. That means each of our services needs to be independent and gives rise to the need for microservices architecture.

A microservices architecture enables us to scale our architecture in the minimum amount of time. Each service has different resource needs, for example data querying will take more power than grammatical analysis. If we put all the application in a monolithic architecture, booting a new instance of the app when traffic suddenly rises will take longer than booting new smaller instances of services which need more power. Also, if an instance of a service goes down (intentionally or unintentionally), it doesn’t cause the whole application to go down.

Monolithic vs Microservices Architecture for Real Time

Monolithic vs Microservices Architecture for Real Time

A real time architecture is what we need. It’s quite common to use such architecture, but we need AI-specific optimisations.

What else?

Identifying request intent

We need to know what the user is talking about as fast as possible. An idea is to use a sift system: we analyse the sentence deeper and deeper until we have identified all the details of it.

For instance, let’s take the sentence “Who is the president of the United States?”

To identify precisely the intent(s) of the request, we can use Xin Li and Dan Roth’s work (check out page 5). Let’s apply this to our sentence:

  • We detect a question, so the user expects an answer.
    • “Who” is the subject. It indicates that we are looking for a HUMAN, an individual.
      • “the president” is an attribute of “Who”. It’s a job.
        • “of the United Stated” is a second attribute of “Who”. It’s a country.

Now we know that we aren’t talking about the weather nor or sports. We will find the key in the presidents of the United States.

Now, a structured database is essential to determine an answer fast.

Preparing data

To be able to answer this question we need to have anticipated it and compiled/structured a list of country presidents.Thus, once the sentence is filtered and interpreted, we can access the  to data that matters directly. We can access the presidents of the USA, then the one that matches the current date. That will lead us to Barak Obama.

This process helps us lower the search time through the database, and therefore the response time.

We can do even more by anticipating user requests.

Anticipating requests

The goal is to reduce the range of possibilities according to the context in which the user in interacting. In the near future, when you will be in a meeting, your smart agent will select and prioritize data that matches the topic of the meeting. This way, it will be ready to process a request faster, and also be more accurate, because the dataset is relevant. If your request doesn’t match the topic of the meeting, your smart agent will resume the normal process (request targeting and searching through structured data).

With requests targeting and anticipation, Artificial Intelligence can understand a request very quickly. With a structured database, it finds the data it needs efficiently and can then start to answer or determine the action to take. That’s  how a microservices architecture enables the user to get an answer in real time !

Want to build your own conversational bot? Get started with SAP Conversational AI !

Follow us on
There are currently no comments.

This site uses Akismet to reduce spam. Learn how your comment data is processed.