This session was the first part of two sessions where we got an introduction on the Azure Cognitive Speech Services, combined with LUIS, .NET Core and the Raspberry Pi.
A client application for LUIS is any application that communicates with a user in natural language to complete a task. The application sends utterances (text) to the LUIS natural language processing endpoint API and receives the results (interpretation) as a JSON response. In this JSON response, Intents describe how LUIS determines understands what a user wants to do. Entities recognized by LUIS will also be returned. The client application then uses the Intents to make decisions about how to fulfill the user's requests.
For this workshop, a (free) LUIS account was needed. This could be created on the LUIS portal at https://www.luis.ai. This portal also offers an excellent intro into Artificial Intelligence.
The sample prototype used in this session used the following hardware:
With the prototype we could turn on and off a yellow and a blue LED light by using the following voice commands: “turn on the yellow / blue light” and “turn off the yellow / blue light” “. The prototype therefore had to be able to understand whether I had to turn a LED on or off, but also which color that had to be turned on or off.
We therefore had to create a LUIS app with which we could analyze a spoken command and determine the intents of this command. A console application that received a voice command and forwarded it to our LUIS app had to be created as well. The LUIS app then sended back a response containing the received voice command and an overview of the intents.
On Azure:
Microsoft Azure - Cognitive Services: https://azure.microsoft.com/nl-nl/services/cognitive-services/
Quickstart: Use prebuilt Home automation app: https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-get-started-create-app
RaspberryIO - Pi's hardware access from .NET: https://github.com/unosquare/raspberryio/blob/master/README.md
Setting up Raspbian and .NET Core 2.0 on a Raspberry Pi: https://blogs.msdn.microsoft.com/david/2017/07/20/setting_up_raspian_and_dotnet_core_2_0_on_a_raspberry_pi/
On the Raspberry Pi:
Raspberry Pi: https://en.wikipedia.org/wiki/Raspberry_Pi
Raspberry Pi (official site): https://www.raspberrypi.org
Comparison of single-board computers: https://en.wikipedia.org/wiki/Comparison_of_single-board_computers
Raspbian Debian-based computer operating system for the Raspberry Pi: https://en.wikipedia.org/wiki/Raspbian
On LUIS:
LUIS: http://www.luis.ai
Tutorial: Recognize intents from speech using the Speech SDK for C#: https://docs.microsoft.com/nl-nl/azure/cognitive-services/speech-service/how-to-recognize-intents-from-speech-csharp
Learn Azure (learn at your own pace): https://docs.microsoft.com/en-us/learn/azure/
On Azure/LUIS/IoT combination:
Quickstart: Use prebuilt Home automation app: https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-get-started-create-app
Control your Raspberry Pi using LUIS on Azure (Part 1): https://www.ronniejongenelen.nl/control-your-raspberry-pi-using-luis-on-azure-part-1/#blog
Control your Raspberry Pi using LUIS on Azure (Part 2): https://www.ronniejongenelen.nl/control-your-raspberry-pi-using-luis-on-azure-part-2/#blog
Control your Raspberry Pi using LUIS on Azure (Part 3): https://www.ronniejongenelen.nl/control-your-raspberry-pi-using-luis-on-azure-part-3/#blog
Handson Lab - Combine LUIS and the Azure Speech Service to control a Raspberry Pi (sources with Ronnies' hands on lab): https://github.com/rwjjongenelen/CognitiveServices.Speech.IntentRecognition
Other stuff:
A nice and free tool for designing electronic circuits: http://fritzing.org
Raspberry Pi
In many Wednesday Evening Trainings we experimented with the Raspberry Pi. This is a is a series of small single-board computers developed by the Raspberry Pi Foundation to promote teaching of basic computer science in schools and in developing countries. The Raspberry Pi can be also be used in home automation, industrial automation and commercial products. Several generations of Raspberry Pis have been released. All models feature a Broadcom system on a chip (SoC) with an integrated ARM-compatible central processing unit (CPU) and on-chip graphics processing unit (GPU). The Raspberry Pi Foundation provides Raspbian, a Debian-based Linux distribution for download, as well as third-party Ubuntu, Windows 10 IoT Core, RISC OS, and specialised media centre distributions.[109] It promotes Python and Scratch as the main programming languages, with support for many other languages. .NET Core 2.0 applications will also run on a Raspberry Pi (see the resources below).LUIS
LUIS (Language Understanding Intelligent Service) is a machine learning-based service to build natural language into apps, bots, and IoT devices. LUIS enables you to integrate natural language understanding into your applications without having to create machine learning models. Instead, you can focus on the application's logic.A client application for LUIS is any application that communicates with a user in natural language to complete a task. The application sends utterances (text) to the LUIS natural language processing endpoint API and receives the results (interpretation) as a JSON response. In this JSON response, Intents describe how LUIS determines understands what a user wants to do. Entities recognized by LUIS will also be returned. The client application then uses the Intents to make decisions about how to fulfill the user's requests.
For this workshop, a (free) LUIS account was needed. This could be created on the LUIS portal at https://www.luis.ai. This portal also offers an excellent intro into Artificial Intelligence.
So what did we do this evening?
We built a prototype based on a sample app with which we could control two LED lights on a Raspberry Pi by spoken commands. The example prototype was built for a Raspberry Pi and contained a Console App for recording spoken text through a microphone, a LUIS app for voice recognition, and an ASP.NET API, directly installed on a Raspberry Pi that controlled the LED lights.The sample prototype used in this session used the following hardware:
- Raspberry Pi 3B
- One 40-way T-Cobbler Breakout Board for Raspberry Pi
- One 830 point Breadboard
- One yellow led light
- One blue led light
- Two resistances of 500 ohm
- Four male to male cables to connect the different parts
With the prototype we could turn on and off a yellow and a blue LED light by using the following voice commands: “turn on the yellow / blue light” and “turn off the yellow / blue light” “. The prototype therefore had to be able to understand whether I had to turn a LED on or off, but also which color that had to be turned on or off.
We therefore had to create a LUIS app with which we could analyze a spoken command and determine the intents of this command. A console application that received a voice command and forwarded it to our LUIS app had to be created as well. The LUIS app then sended back a response containing the received voice command and an overview of the intents.
To convert the results that we received from LUIS to GPIO commands on the Raspberry Pi, we had to build an ASP.NET Core Web API that had to be installed directly onto the Raspberry Pi. Therefore, .NET Core had to be installed on the Raspberry Pi. No problem, since .NET Core also runs perfectly on Linux.
Thanks Ronnie Jongenelen, for sharing your knowledge with us! Great stuff!
Thanks Ronnie Jongenelen, for sharing your knowledge with us! Great stuff!
Next steps...
Next Wednesday Evening Training we will continue our lab. Also: we will be experimenting on using alternative solution components: MQTT and NodeRED. Since the Arduino (single-board computer) also has our attention in the Wednesday Evening Trainings, we'll also have a brainstorm on how to implement this solution on the Arduino.Further reading
Do you want to read more on the topics in this post? Here are some resources...On Azure:
Microsoft Azure - Cognitive Services: https://azure.microsoft.com/nl-nl/services/cognitive-services/
Quickstart: Use prebuilt Home automation app: https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-get-started-create-app
RaspberryIO - Pi's hardware access from .NET: https://github.com/unosquare/raspberryio/blob/master/README.md
Setting up Raspbian and .NET Core 2.0 on a Raspberry Pi: https://blogs.msdn.microsoft.com/david/2017/07/20/setting_up_raspian_and_dotnet_core_2_0_on_a_raspberry_pi/
On the Raspberry Pi:
Raspberry Pi: https://en.wikipedia.org/wiki/Raspberry_Pi
Raspberry Pi (official site): https://www.raspberrypi.org
Comparison of single-board computers: https://en.wikipedia.org/wiki/Comparison_of_single-board_computers
Raspbian Debian-based computer operating system for the Raspberry Pi: https://en.wikipedia.org/wiki/Raspbian
On LUIS:
LUIS: http://www.luis.ai
Tutorial: Recognize intents from speech using the Speech SDK for C#: https://docs.microsoft.com/nl-nl/azure/cognitive-services/speech-service/how-to-recognize-intents-from-speech-csharp
Learn Azure (learn at your own pace): https://docs.microsoft.com/en-us/learn/azure/
On Azure/LUIS/IoT combination:
Quickstart: Use prebuilt Home automation app: https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-get-started-create-app
Control your Raspberry Pi using LUIS on Azure (Part 1): https://www.ronniejongenelen.nl/control-your-raspberry-pi-using-luis-on-azure-part-1/#blog
Control your Raspberry Pi using LUIS on Azure (Part 2): https://www.ronniejongenelen.nl/control-your-raspberry-pi-using-luis-on-azure-part-2/#blog
Control your Raspberry Pi using LUIS on Azure (Part 3): https://www.ronniejongenelen.nl/control-your-raspberry-pi-using-luis-on-azure-part-3/#blog
Handson Lab - Combine LUIS and the Azure Speech Service to control a Raspberry Pi (sources with Ronnies' hands on lab): https://github.com/rwjjongenelen/CognitiveServices.Speech.IntentRecognition
Other stuff:
A nice and free tool for designing electronic circuits: http://fritzing.org
No comments:
Post a Comment