Start building your own chatbot now

Many people, especially those without a background in computer science, might think that building a chatbot using AI to recognize objects in images is rocket science. This tutorial will show you that making this type of bot is a lot easier than you may believe – and that you don’t have to be a developer to build an awesome bot.

I created the MovieStars.A chatbot in July 2019, using SAP Conversational AI bot building platform in conjunction with Python. The bot allows users to input an image URL (for example, http://aaa.com/example.jpg) to its chat window and responds with a message containing the names of the movie stars in the image. To achieve this AI magic, the bot runs a Python program on a simulated local server (ngrok).
What’s more, the program uses the Amazon Rekognition API to analyze the image and output a reply to the chat window. To see how this bot works, check out my demo video on YouTube by clicking the image below:

This tutorial is intended to help you recreate the MovieStars.AI bot using your own SAP Conversational AI account. In addition, it will show you how to set up and run the bot correctly on your local machine so that you can input an image URL and have the bot reply with a message telling you which movie celebrities the image contains. Let’s get started.

This tutorial consists of six main steps:

  • Step 1: Creating the MovieStars.AI bot on SAP Conversational AI
  • Step 2: Getting access to AWS and Amazon Rekognition
  • Step 3: Preparing the Python program for your bot
  • Step 4: Installing ngrok on your local machine
  • Step 5: Powering the MovieStars.AI bot with bot.py and ngrok API
  • Step 6: Using the MovieStars.AI Bot webchat

Step 1: Creating the MovieStars.AI bot on SAP Conversational AI

If you don’t want to set up the MovieStars.AI chatbot on SAP Conversational AI from scratch, simply fork the moviestars-ai bot I created. First, you sign in to your account at https://cai.tools.sap, visit https://cai.tools.sap/dennis2018, and then click the fork icon next to moviestars-ai to fork the bot to your SAP Conversational AI account. But you don’t want to miss out on all the fun of creating the MovieStars.AI bot from the ground up, so I recommend that you continue with Step 1.

Now let’s begin the tutorial for this step:

1.1 Create the bot on SAP Conversational AI

1.1.a Sign in to your account at https://cai.tools.sap

1.1.b Click + New Bot

1.1.c Create your bot by completing the following substeps:

  • Under Select predefined skills for your bot, select Greetings
  • Under Create your bot, enter “MovieStars-AI-<yourname>” in the Your bot name field and include a description. (Note: For “yourname” enter your SAP Conversational AI username. For example, if your username is david3, use that here.).
  • Under Data Policy, select the Non-personal (type of data) and Non-vulnerable (end users) radio buttons.
  • Under Bot visibility, select Public.
  • Click CREATE A BOT at the bottom of the screen to create your bot

1.2 The bot is now on SAP Conversational AI at the link https://cai.tools.sap/<yourname>/moviestarts-ai-<yourname>. (As above, the link should include your SAP Conversational AI username.)

1.2.1 You see that your bot has the @greetings intent. Create four more new intents. In each case, do this by clicking + CREATE and entering the relevant text:

@imageurl
@no
@yes
@goodbye

1.2.2 Click the @no intent link and enter expressions like “No” or “not now” in the Add an expression text box. After you’ve typed in each new expression, hit the Enter

1.2.3 Click the Intents link to return to the Intents page: (Please note that I altered this and the following screenshots in Snagit by adding “yourname”.)

intent-sap-conversational-ai

1.2.4 On the Intents page, click the @yes intent link and enter expressions like “Yes” or “Yes do it” in the Add an expression text box. After you’ve typed in each expression, hit the Enter To maximize SAP Conversational AI’s recognition of a given expression (user input to bot) as an @yes intent, try to enter at least 20 expressions.

1.2.5 Click the Intents link to return to the Intents Click the @imageurl intent link and enter URL expressions like “https://images5.alphacoders.com/409/409184.jpg” in the Add an expression text box. To maximize SAP Conversational AI’s recognition of your input as an @imageurl intent, try to enter at least 20 expressions.

1.2.6 Note that every time you enter a URL expression for the @imageurl intent, SAP Conversational AI will automatically assign the URL expression as a #URL entity: (screenshot altered in Snagit)

chatbot-sap-platform-intent

1.2.7 Click the Intents link to return to the Intents Click the @goodbye intent link and enter expressions like “goodbye” or “see you later” in the Add an expression text box. Try to enter at least 20 expressions for this intent.

1.3 Click the Build tab to build skills for this bot. You see the greetings and fallback skills:

bot-builder-sap-conversational-ai

1.4 Click the fallback link on the fallback skill tile and click the Actions tab on the skill page. Click the pencil icon, type in the fallback response message, as shown below, and click SAVE:

fallback-skill-sap-conversational-ai

1.5 Click the Build tab and click + Create skill. In the Create a new skill dialog, type “detect_celebrities” in the text box and click CREATE SKILL:

create-skill-sap-conversational-ai

1.6 Now you see a detect_celebrities skill tile on the Build tab page.

1.7 On the detect_celebrities tile, click the detect_celebrities link to open the skill page. Click the Triggers tab and fill in the trigger condition (“@imageurl is-present”), as shown below:

triggers-sap-conversational-ai

Entering the trigger condition ensures that the detect_celebrities skill is triggered if the user’s input to the bot is an @imageurl intent. In other words, this skill will be triggered to perform actions if you input an image URL to the bot.

1.8 Click the Requirements tab and enter the requirement (“#url as url”), as shown below:

1.9 Click the Actions tab and click ADD NEW MESSAGE GROUP.

1.10 Click the ADD CONDITION to trigger messages link.

1.11 Enter the trigger conditions, as shown below:

1.12 Click SEND MESSAGE and select Card. Make the entry shown below:

1.13 Click ADD A BUTTON, select Link, and make the entries shown below. (The button will have the name Original Image, and clicking the button will lead to the {{memory.url.raw}}) Click SAVE:

1.14 Click ADD NEW MESSAGE GROUP and then click CALL WEBHOOK. In the text box, you can now enter the API link that connects to a Python program that identifies movie celebrities in an image (pointed to by the URL that you input into the bot). For now, just enter the fake link “https://example.ngrok.io” in the text box as a placeholder and click SAVE.

You will generate this API in Step 5 of this tutorial, so hold your horses for now.

1.15 Click ADD NEW MESSAGE GROUP at the bottom of the screen, then click UPDATE CONVERSATION, and select EDIT MEMORY. Now select the Reset all memory checkbox and click SAVE.

1.16 You’re all set. Now you want to ensure that you always get a friendly reply from your bot when you input text like “I love music” or other non-URL text. To do so, go to the Actions tab on the greetings skill page and enter messages like those shown below (one for “@greetings is-present”, another for “@goodbye is-present”):

Access the greetings skill page by clicking the Build tab and clicking the greetings link on the skill tile.

Step 2: Getting access to AWS and Amazon Rekognition

2.1 Visit https://aws.amazon.com/ and create your AWS account. On your AWS Management Console page, click your username and then select My Security Credentials.

aws-management-console

2.2 Now, on the Your Security Credentials page, click Access keys (access key ID and secret access key) and, in the expanded section, click Create New Access Key. Next, click Download Key File in the popup, and you’ll get an Excel file containing the AWSAccessKeyId and AWSSecretKey information.

2.3 Always keep your AWSAccessKeyId and AWSSecretKey safe and save them to a text file. You will need these two keys to create the MovieStars.AI bot in this tutorial. Don’t let anyone see these two keys, or your AWS account could be hacked.

2.4 To access AWS Rekognition for the Python program of your MovieStars.AI bot, install the boto3Python library on your computer (boto3 has been created by Amazon to allow you to create Python programs using AWS Rekognition). To install the library on your computer, open your command prompt and use the command “pip install boto3”.

Step 3: Preparing the Python program for the bot

Please visit https://github.com/dennisypan/MovieStars-AI and download the bot.py Python file to the directory C:\CHATBOTS\MovieStars-AI on your local C drive.

The diagram below explains how the bot.py Python program works when using Amazon Rekognition API to analyze an image from a URL and output a response (about which movie stars have been identified) to your SAP Conversational AI bot.

In AWS, copy your AWS access key (AWSAccessKeyId) and AWS secret (AWSSecretKey). Next, open your bot.py file in a text editor, paste your AWS access key and AWS secret into the blanks in line 20 and line 21 of the Python code. Finally, save bot.py. You now have a working bot.py Python program in your C:\CHATBOTS\MovieStars-AI directory.

 

Step 4: Installing ngrok on your local machine

You need ngrok to expose your local server to the Internet to generate an API, allowing your MovieStars.AI bot on SAP Conversational AI to access bot.py and do its magic.

You install ngrok on your computer as follows. If you’re using a Windows machine, use the Chocolatey package manager to install ngrok. First, visit https://chocolatey.org/install and install Chocolatey on your computer. After that, run the following command on the Chocolatey console to install ngrok:

choco install ngrok_portable

Once the installation is complete, Chocolatey will add ngrok to your computer’s PATH, allowing you to start ngrok via a terminal window (for example, PowerShell) simply by typing and entering the “ngrok” command. Note that to ensure smooth installation of Chocolatey and ngrok, you should temporarily turn off your firewall. But don’t forget to turn your firewall back on after successful installation.

To see if everything is working, let’s move to the next step of the tutorial.

Step 5: Powering the MovieStars.AI bot with bot.py and ngrok API

After setting up ngrok, you’re ready to run the bot.py program and to run ngrok on your local machine to obtain an API for your bot to use. Add this API to the designated skill action for the bot. (We’ll show you how to do this later.) You can now test the bot on the platform to see whether it recognizes movie star(s) from an image (via the image URL you input to the bot).

Here are the steps:

5.1 Open a PowerShell window as administrator. First, enter the command “cd C:\CHATBOTS\MovieStars-AI”, then enter the “python bot.py” command to run the program. The result is as follows:


5.2 Open another PowerShell window as administrator. Enter the “ngrok http 5000” command in the first command line. You will then see the following in the PowerShell window:

5.3 In the PowerShell window, see the second “Forwarding” API starting with https. Copy the API https://3a8a00db.ngrok.io (in this example), ready to be pasted into your bot on SAP Conversational AI.

5.4 Open your bot at https://cai.tools.sap/yourname/moviestarts-ai-yourname, click the Build tab, and click the detect_celebrities This opens the skill page. Now click the Actions tab under the name of the skill:

5.5 Scroll down to the second action group on the Actions tab page. Paste the API https://3a8a00db.ngrok.io into the POST text box and click SAVE:

5.6 Now test your MovieStars.AI bot on the platform. First, click CHAT WITH YOUR BOT in the bottom-right corner. This opens the chat window for you to test your bot:

5.7 Now type an image URL in the chat window. In this example, we’ll enter https://pixelz.cc/wp-content/uploads/2018/09/liu-yifei-portrait-uhd-4k-wallpaper.jpg

5.8 Now hit Enter, and after a few seconds the bot responds in the chat window, with a message telling you which movie star is in the image (from the input image URL). Success!

5.9 But we need to perform one more test. We want to ensure that, if a URL (input to bot) points to an image containing nonhuman subjects or people who are not famous in movies, the MovieStars.AI bot can tell you that no celebrities were detected. To perform this test, go to the bot chat window, type in the following URL of a burger image, and hit Enter: https://www.seriouseats.com/recipes/images/2015/07/20150728-homemade-whopper-food-lab-35-1500×1125.jpg

5.10 Once you have done this, you should see that the bot does a great job recognizing that the image contains no movie stars and responds with the correct answer. Excellent! Your bot is working perfectly.

5.11 Please keep the two PowerShell windows open and running so that you can complete Step 6 of the tutorial, which focuses on the MovieStars.AI bot webchat.

Step 6: Using MovieStars.AI Bot webchat

If you want to allow folks on the Internet to play with your MovieStars.AI bot, you may need to embed it in a website page (html) deployed on a cloud server like AWS (assuming you already deploy the bot.py Python program to the same cloud server instance hosting your website). To do so, you must first get webchat script snippets for your MovieStars.AI bot from SAP Conversational AI and embed the script in the html code of the deployed html page so that the page has a functioning webchat window, allowing users to chat with your MovieStars.AI bot.

We won’t show you how to deploy a website on a cloud server, but we will teach you how to embed the MovieStars.AI bot webchat in a simple html page. The end results should look something like the following:

moviestars-ai-chatbot-recognition

To embed the MovieStars.AI bot webchat in a simple html page, proceed as follows:

6.4 Under Color scheme, change Accent color to green. Under Header customization, enter “MovieStars.AI” as the header title.

6.5 Under Message settings, enter your custom Onboarding message, Input placeholder, and Welcome message. For this tutorial, type in the messages shown in the following screenshot for these three fields:

6.6 Under Global settings, you can specify the number of hours your conversation with the bot will be saved for. In the dropdown menu, select an option for Keep chat window open on the webpage and specify the maximum number of characters users can input to the bot (to avoid the risk of an “injection” attack by hackers).

6.7 Under Global settings, you see the Webchat script snippet. Click SAVE CHANGES and copy the Webchat script snippet to the clipboard. (The channelId and token are hidden here for security reasons.)

6.8 Open your index.html file, paste the script snippet into the html code from line 13 to 16 and then save the file.

6.9 Open index.html in a browser. You will see a webchat icon in the lower right corner. Click on it.

6.10 After you open the webchat window, you see a welcome message asking you to input an image URL.

6.11 In the chat window, type “who is in https://wallpaperaccess.com/full/459878.jpg” and hit Enter. You can see that the bot correctly identifies the actress Scarlett Johansson in the image (pointed to by the URL).

6.12 In the chat window, type “who is in https://pbs.twimg.com/media/EEvUTqjWsAEoLYt.jpg” and hit Enter. You see that the bot correctly replies that no movie stars are in the image (pointed to by the URL).

Conclusion

The ideal chatbot platform is one that allows everyone to create chatbots, regardless of their technical skills.
This tutorial demonstrated that SAP Conversational AI allows even non-programmers to create smart chatbots – which is extremely helpful for non-technical business professionals who want to create business chatbot prototypes. It also showcased the image recognition capabilities of the platform.
SAP Conversational AI is designed to remove most of the technical barriers that for years prevented people without a technical background from creating chatbots.

Of course, developing your programming skills will help you create more sophisticated chatbots on SAP Conversational AI. If that sounds scary, Python is a good way to begin learning programming. Not only is Python is very versatile; it’s also easy to learn. Mastering Python takes some time and effort, but playing with it for as little as 10 hours a week will get you there. That’s one reason I based this MovieStars.AI bot tutorial on Python. Plus, it’s easy to integrate a Python program into the SAP Conversational AI bot, enabling the bot to do amazing work.

If you have any questions about this tutorial, feel free to ask us in the comments section below or go to SAP Answers.

Want to build your own conversational bot? Get started with SAP Conversational AI !

Follow us on
There are currently no comments.

This site uses Akismet to reduce spam. Learn how your comment data is processed.