:::: MENU ::::
Apr 29, 2015    |      0 comments

Windows 10 & Cortana – Personal Assistance Everywhere

[This article was first published on gooroo.io]

Most people are familiar with Siri on iOS and Google Now on Android, but the much more personal and arguably more helpful Cortana on the Windows platform is not as well known. And that is a shame. In terms of contextual assistance and personalised service, Cortana is more advanced and more adaptive than her competitors. Yes, “her”.

Getting Up to Speed

Before I get into Cortana’s great promise, let’s have a quick look at how she currently helps out. Whereas Siri is a bit robotic and feels automated, Cortana understands almost natural language (I say “almost, as there are some quirks, especially with non-American English). You can for example tell Cortana “Remind me to pick up kittens next time I’m at the supermarker” and she will know when you are in a supermarket and then pop up a reminder on your phone.

Image alternative text
Image alternative text
Image alternative text
Image alternative text

Or ask “Remind me to ask about cats next time I talk to Alex”, and whether you are talking, texting or replying to a text from the little pom you will get a notification reminding you. That is useful. But you can go one step further and ask a question in the context to what she answered previously. If you asked to get the best restaurants in the area, and Cortana gives you a list of the ten best ones, you can ask to get directions to the second result. Quick, easy simple. There is a range of other features, such as telling jokes, analysing your emails for flight information, knowing you daily commute and reminding you when to leave from home or work.

Cortana on mobile and desktop

As part of Windows 10 Cortana is now for the first time of any digital assistant coming to a desktop environment. She will be available not just on mobile, but on tablet, laptop, desktop and everything in between (did you say phablet?). As well as being able to use Cortana on these devices, reminders and notifications will also carry across, so you aren’t reliant on your phone to remind you to pick up little Johnny at the bus stop (which you forgot yesterday as squirrel!!). There has been rumours of Cortana coming to iOS and Android, which would make the value proposition even greater for most users. And I would eat my hat, if we didn’t see Cortana on Xbox One in the near future as well. Being able to just say “Hey Cortana”, anywhere you are near a device and record a reminder or ask a question would indeed be powerful.

Image alternative text

Figure: Cortana on desktop

Making Cortana do Your Bidding

Of course it wouldn’t be as cool if it was just Microsoft that could tell Cortana what to do programming wise. As you would hope and expect, you can integrate Cortana into your app with relative ease. It is all done through a Voice Command Definition (VCD) file which is essentially just an XML file with some commands in it.

[code language=”xml”]
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.1">
<CommandSet xml:lang="en-us" Name="examplevcd">
<CommandPrefix>Kittens</CommandPrefix>
<Example>Find more LOL cats</Example>

<Command Name="findCats">
<Example>List funny kittehs</Example>
<ListenFor>[Find] {options} {cats}</ListenFor>
<Feedback>Searching {options} {cats}</Feedback>
<Navigate Target=”/Kittehs.xaml”/>
</Command>

<PhraseList Label="options">
<Item>black</Item>
<Item>silly</Item>
<Item>magic</Item>
</PhraseList>

<PhraseTopic Label="cats" Scenario="Find">
<Subject>Kitteh Kolor</Subject>
</PhraseTopic>

</CommandSet>

<!– Other CommandSets for other languages –>

</VoiceCommands>
[/code]

Figure: Create a voice command to search for LOL cats as a VCD (XML) file

The above example show a simple, yet powerful way to open my fictitious LOLcat app and search for kittens. I have even added a few options to the voice commands for good measure. Then all you do is to register your VCD file when the application fires up and you are good to go.

[code language=”csharp”]
private async void Application_Launching(object sender, LaunchingEventArgs e)
{
try
{
var vcdUrl = new Uri("ms-apps:///CortanaCommands.xml", UriKind.Absolute);
await VoiceCommandService.InstallCommandSetsFromFileAsync(vcdUrl);
}
catch (Exception)
{
// Handle that exception!
}
}
[/code]

It is of course up to you to interpret the variables ({options} and {cats} above) that get passed from the voice command, and make sense of it in the context of your app.

Can You See the Future?

I hope you can see the promise of Cortana a little clearer by now and what her potential is. A couple of the predictions that has been floating around are

  • Cortana on Xbox One. You can really use voice control then combined with Xbox. I wonder if the Xbox and Cortana will ever fight for control.
  • Track your packages

  • Actual predictions of sports results and other competitive events are expanding
  • Cortana is going to make an apperance on iOS and Android. It will be interesting to see how developers on those platforms can integrate her as she is unlikely to become a first class citizen.

There is no doubt that the lady originally from Halo is going to be an integral part of the Microsoft ecosystem and with little effort you can make your Windows app part of that world as well.