Building a Speech-enabled Bot with Cortana Skill


Cortana is the intelligent personal assistant for Windows Phone 8.1 and Windows 10, for both work and personal use; it’s used by millions of Windows users. Cortana uses natural language processing; its skill directory showcases the best of its abilities. The Cortana Skills Kit provides all the tools and docs for developers to promote services and engage users through the Cortana experience. The Cortana Skills Kit is designed to help developers driving engagement across platforms, such as Windows, Android, iOS, and Xbox.

Cortana Devices SDK

Microsoft has launched a private preview of Cortana Skills Kit, which has enabled developers to easily create new Cortana skills. Microsoft also has launched the Cortana Devices SDK, which is currently in private preview and will be available broadly by end of 2017. The Cortana SDK lets OEMs integrate Cortana directly into their devices. The Cortana Devices SDK looks like it will help deliver Cortana to all kinds of new devices.

The Cortana Skills Kit allows developers to create voice apps for the intelligent assistant. Cortana has rich knowledge and understanding about the user with the skills kit. Developers are now able to create Cortana skills with both the Cortana Skills Kit and by using a Microsoft Bot Framework. With the Cortana Skills Kit, developers now can access knowledge about the user and build highly-relevant, personalized experiences-based apps.

Cortana with Bot Framework

During Build 2017, Microsoft announced several updates on the Bot Framework. Cortana was introduced as a new UI channel for the Bot Framework during that event. Cortana skills can be is extended to the Bot framework and publish them. Using the Bot Framework, we can develop, debug, and release bots that users can access via the Cortana Channel Windows 10 devices.

Set Up Your Cortana Bot

Now, let’s build a Cortana Bot using the Microsoft Bot Framework. You need to have prior knowledge of Microsoft’s Bot Framework before building Cortana chatbots.

Step 1

Currently, Cortana Skills Kit is only available to use for the US only, so you can train your Bot in the US English language for now.

Open the Cortana developer portal and click ‘get started,’ or you can create skills in the Cortana Developer Dashboard and you can then be redirected to the Bot Framework dashboard to register a Bot.

Cortana Dev Center
Figure 1: Cortana Dev Center

Step 2

Sign in using your Microsoft account. After successful sign in, you will be redirected to the dashboard to add Cortana Skills.

Microsoft account
Figure 2: Microsoft account

Step 3

Accept the Cortana developer terms and conditions and click ‘Create a New Skill.’ You will be redirected to the Bot Framework.

Accept Terms and Conditions
Figure 3: Accept Terms and Conditions

Step 4

You can register your bot in the Bot Framework dashboard first and then enable the Cortana channel. In this example, I have registered a bot in the Bot Framework portal first; my demo bot is named ‘Cortana bot.’

Bot Framework Sign in
Figure 4: Bot Framework Sign in

Step 5

All fields marked with an asterisk (*) are mandatory, so make sure you fill up the all the details before Bot registration.

  • Skill Icon: Select an icon to represent your bot.
  • Display Name: The name of this skill as displayed to the user at the top of Cortana’s visual UI.
  • Invocation Name: The name that Cortana will recognize and use to invoke this skill when spoken aloud by the user.
  • Skill Description: Describe the skill mentioned.
  • Short Description: This summary will be used to describe the skill in Cortana’s Notebook.

Bot Profile Page
Figure 5: Bot Profile Page

Please refer to the detail documentation available on the Microsoft Web site. It describes each Bot registration field.

Bot Configuration
Figure 6: Bot Configuration

Step 6

After filling up all mandatory fields, click Register.

Publish Bot
Figure 7: Publish Bot

Step 7

Once you have registered your bot in the Bot Framework portal, you will notice that Cortana is now an available channel, along with the new Skype 4 Business, Bing, other channels.

Select Cortana Channel
Figure 8: Select Cortana Channel

Selecting the new channel will open a new window and take you to the Cortana Skills Dashboard, where you have to se tup your new skill that will be connected to your bot. Next, you can test your Bot by clicking the Test button; it will open a chat window. The user can ask the Bot a question.

Test your Bot
Figure 9: Test your Bot

Finally, you can run the Bot locally or deployed in the Cloud.

Adding Speech to Cortana Bot

Cortana Bot has both visual and audio components. Cortana provides an area of the canvas for visual display and for audio; you have to provide text or SSML input in your Bot’s messages. Cortana will read that message, giving voice to the Bot. Cortana can read both plain text and SSML; refer to the following JavaScript code.

  // Cortana reading plain text
  session.say('Sample text that Cortana will displays',
     'Sample Text spoken by Cortana.');

  // Cortana reading SSML
  session.say('Sample text that Cortana will displays',
     '<speak version="1.0"
     xml_lang="en-US">Sample Text spoken by Cortana.</speak>');

The following code snippet allows the user to input their voice by using the Cortana opening microphone.

  // Taking Voice Input from user
  session.say('Hello', 'Please enter your country name.', {
     inputHint: builder.InputHint.expectingInput

The following code block is an example of prompting the user, using the speak and retrySpeak options.

  builder.Prompts.text(session, 'text prompt', {
     speak: 'Please enter your country name.',
     retrySpeak: 'I am still waiting ...
        Please enter your country name.',
     inputHint: builder.InputHint.expectingInput


I hope this article has given you basic knowledge about building a Cortana Bot and connecting it to the Bot Framework. That’s all for today; stay tuned for more Cortana articles!

More by Author

Must Read