Voice Operated Internet Control of a PICAXE (Part II)

Objectives

Experiment with more advanced features offered by Picaxe microcontroller. Measure temperature through a Maxim  DS18B20   sensor and obtain interactivity with recourse to audio response using text to voice offered by Google translate.

The electronic setup

AXE050 board with Picaxe18M microcontroller is used just as in Part I. Here is a picture of AXE050:

AXE050 Board

Temperature sensor Maxim  DS18B20  is added to the AXE050 board reads temperatures. The advantage of using Picaxe18M is that the same ports are used for control and programming.

DS18B20 Pinout

PICAXE-18M and DS18B20 connections

Reading and control of AXE050 board is done using RS232 protocol. The onboard connections are used for RS232 control.

Pre-Requirements

Before carrying out this part of project, Voice Operated Internet Control of a PICAXE (Part I) must have been implemented, i.e. experimented already with simple switching of a LED on the AXE050 board. The firmware on the chip will be the same as in Part I, and the same as in Internet Control of a Picaxe published here long time ago.

RS232 control is done again in DOS environment, using Kermit for DOS. Kermit scripts are run by PHP, to control AXE050. Kermit’s executable MSK316.exe is installed in same directory as all files. As mentioned in part I, all files will be in c:\xampp\htdocs.

Note: COM2 is used  at baudrate of 4800 setup in Control Pannel of Windows XP.

Now the scripts…

STATUS Kermit script (name it status.ksc):

set port 2
set baud 4800
clear
output c=s
input /nowrap

The new command is input /nowrap, being used to obtain output from Kermit and pipe it into the server.

This script is called in a BATCH file (name it sta.bat):

@echo off
c:\wamp\www\msk316 take status.ksc
exit

TEMPERATURE Kermit script (name it gettemp.ksc):

set port 2
set baud 4800
clear
output c=t
input /nowrap

Again we use new command input /nowrap.

This script is called in a BATCH file (name it temp.bat):

@echo off
c:\wamp\www\msk316 take gettemp.ksc
exit

All batch files are then converted into .exe using Bat_to_Exe_Converter. As result, will have two new executables sta.exe and temp.exe in the folder htdocs, besides the previous part I executables ledon.exe and ledoff.exe.

Finally again the html and php codes…

Substitute index.html of Part I with new one as shown below, again within htdocs folder of Windows-based XAMPP server.

index.html

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-language" content="en-US">
<meta charset="utf-8">

<title>Voice Operated Control </title>
<script type="text/javascript">
function checkSpeechSupport(){
var element=document.createElement("input");
if (!('webkitSpeech' in element)) {
alert("Sorry! Your browser does not support the Speech Input");
}
}
function checkanswer() {
var answer = document.getElementById('q2answer').value;

if (answer == 'light on') {

window.location.href = "http://localhost/processing.php?action=on";
}

else if (answer == 'lights on') {

window.location.href = "http://localhost/processing.php?action=on";
}

else if (answer == 'light off') {

window.location.href = "http://localhost/processing.php?action=off";
}

else if (answer == 'lights off') {

window.location.href = "http://localhost/processing.php?action=off";
}

else if (answer == 'temperature') {

window.location.href = "http://localhost/processing.php?action=tem";
}

else if (answer == 'temperatures') {

window.location.href = "http://localhost/processing.php?action=tem";
}

else if (answer == 'status') {

window.location.href = "http://localhost/processing.php?action=sta";
}

else {
window.location.href = "alert.php";
}
}
</script>
</head>
<body onLoad="checkSpeechSupport()">
<form id="speechform">
<fieldset id="inputs">
<font face="arial, verdana"><legend><h2>Voice Control of Picaxe AXE-050U:</h2></legend>
<label for="q2answer">Say commands: "light on", "light off"<br>"temperature" and "status":</label></font>
<input id="q2answer" name="q2answer" type="text" x-webkit-speech onwebkitspeechchange="checkanswer()"/><br/>
<img src="http://localhost:8888/out.jpg">
</fieldset>
</form>
</body>
</html>

Substitute processing.php of Part I with new one as shown below, again within htdocs folder

processing php

&lt;?php
// numerical to text class included
include 'num2text.class.php';
//check the GET actions variable to see if something needs to be done
    if (isset($_GET['action'])) {
    //Action has been requested
    //Issue the command we wish to send to the Picaxe

    if ($_GET['action'] == &quot;on&quot;) {
        //Turn LED on - for this simple script we are just looking for either a 1 or 0
    $page = &quot;index.html&quot;;
    header(&quot;Refresh: 4; URL=\&quot;&quot; . $page . &quot;\&quot;&quot;);
          exec ('sta.exe',$report);
           // check waht is in array $report
           // print_r($report);
          $pieces = explode(' ',$report[0]);
          if ($pieces[1] == 1) {
                echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;LIGHT IS ALREADY ON&lt;/b&gt;&lt;/font&gt;&lt;br&gt;&quot;;
                echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=light+is+already+on!">http://translate.google.com/translate_tts?tl=en&#038;q=light+is+already+on!</a>&quot;;
          }
          else {
          exec('ledon.exe');
          echo &quot;&lt;hr align='center' size=\&quot;4\&quot; color=\&quot;orange\&quot;&gt;&lt;br&gt;&quot;;
          echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;LIGHT IS NOW ON&lt;/b&gt;&lt;/font&gt;&lt;p&gt;&quot;;
          echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=light+is+now+on!">http://translate.google.com/translate_tts?tl=en&#038;q=light+is+now+on!</a>&quot;;
      }

      }

     else if ($_GET['action'] == &quot;off&quot;) {
     //Turn LED off
     //Now we &quot;open&quot; the serial port so we can write to it
     $page = &quot;index.html&quot;;
     header(&quot;Refresh: 4; URL=\&quot;&quot; . $page . &quot;\&quot;&quot;);
           exec ('sta.exe',$report);
           // check waht is in array $report
           // print_r($report);
           $pieces = explode(' ',$report[0]);
           if ($pieces[1] == 0) {
                echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;LIGHT IS ALREADY OFF&lt;/b&gt;&lt;/font&gt;&lt;br&gt;&quot;;
                echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=light+is+already+off!">http://translate.google.com/translate_tts?tl=en&#038;q=light+is+already+off!</a>&quot;;
           }
           else {
           exec('ledoff.exe');
           echo &quot;&lt;hr align='center' size=\&quot;4\&quot; color=\&quot;orange\&quot;&gt;&lt;br&gt;&quot;;
           echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;LIGHT IS NOW OFF&lt;/b&gt;&lt;/font&gt;&lt;p&gt;&quot;;
           echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=light+is+now+off!">http://translate.google.com/translate_tts?tl=en&#038;q=light+is+now+off!</a>&quot;;
           }
      }

      else if ($_GET['action'] == &quot;tem&quot;) {
           exec ('temp.exe',$report);
           $pieces = explode(' ', $report[0]);
           $n2s = new num2text($pieces[1]);
           $page = &quot;index.html&quot;;
           header(&quot;Refresh: 4; URL=\&quot;&quot; . $page . &quot;\&quot;&quot;);
           echo &quot;&lt;hr align='center' size=\&quot;4\&quot; color=\&quot;orange\&quot;&gt;&lt;br&gt;&lt;b&gt;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;font color=blue&gt;LOCAL TEMPERATURE:&lt;/b&gt;&lt;/font&gt;&lt;/font&gt;&lt;br&gt;&quot;;
           echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;&quot;.$pieces[1].&quot;&amp;nbsp;ºC&lt;/b&gt;&lt;/font&gt;&lt;p&gt;&quot;;
           echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=local+temperature%+is+.$n2s.+degrees+celsius.">http://translate.google.com/translate_tts?tl=en&#038;q=local+temperature%+is+.$n2s.+degrees+celsius.</a>&quot;;

        }

      else if ($_GET['action'] == &quot;sta&quot;) {
      $page = &quot;index.html&quot;;
      header(&quot;Refresh: 4; URL=\&quot;&quot; . $page . &quot;\&quot;&quot;);
           exec ('sta.exe',$report);
           // check waht is in array $report
           // print_r($report);
           $pieces = explode(' ',$report[0]);
           // check what is in $pieces
           // echo $pieces[1].&quot;&lt;br&gt;&quot;; // piece1
           //echo $pieces[2].&quot;&lt;br&gt;&quot;; // piece2
           echo &quot;&lt;hr align='center' size=\&quot;4\&quot; color=\&quot;orange\&quot;&gt;&lt;br&gt;&lt;b&gt;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;font color=blue&gt;LOCAL STATUS:&lt;/b&gt;&lt;/font&gt;&lt;/font&gt;&lt;p&gt;&quot;;

                if ($pieces[1] == 1) {
                echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;LIGHT IS ON&lt;/b&gt;&lt;/font&gt;&lt;br&gt;&quot;;
                echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;TEMPERATURE IS &quot;.$pieces[2].&quot;&amp;nbsp;ºC&lt;/b&gt;&lt;/font&gt;&lt;p&gt;&quot;;
                $n2s = new num2text($pieces[2]);
                echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=light+is+on.+local+temperature+is+.$n2s.+degrees+celsius+.">http://translate.google.com/translate_tts?tl=en&#038;q=light+is+on.+local+temperature+is+.$n2s.+degrees+celsius+.</a>&quot;;
                }
                else if ($pieces[1] == 0) {
                echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;LIGHT IS OFF&lt;/b&gt;&lt;/font&gt;&lt;br&gt;&quot;;
                echo &quot;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;b&gt;TEMPERATURE IS &quot;.$pieces[2].&quot;&amp;nbsp;ºC&lt;/b&gt;&lt;/font&gt;&lt;p&gt;&quot;;
                $n2s = new num2text($pieces[2]);
                echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=light+is+off.+local+temperature+is+.$n2s.+degrees+celsius+.">http://translate.google.com/translate_tts?tl=en&#038;q=light+is+off.+local+temperature+is+.$n2s.+degrees+celsius+.</a>&quot;;
                }
        }
      }
?&gt;

NOTE: iframe is used to avoid opening a new page and also problems in starting automatically Google’s voice file not starting automatically.

The following php class is courtesy of Jay Gilford, it is included in above processing.php and is used to convert numerals into text, to avoid any problems in Google tanslate not recognising numerals as by default %20 is added by the Chrome in the URL

num2text.class.php

<?php
class num2text {
    private $_original = 0;
    private $_parsed_number_text = '';
    private $_single_nums = array(1 => 'one', 2 => 'two', 3 => 'three', 4 => 'four', 5 => 'five', 6 => 'six', 7 => 'seven', 8 => 'eight', 9 =>
        'nine', );

    private $_teen_nums = array(0 => 'ten', 1 => 'eleven', 2 => 'twelve', 3 => 'thirteen', 4 => 'Fourteen', 5 => 'fifteen', 6 => 'sixteen', 7 =>
        'seventeen', 8 => 'eighteen', 9 => 'nineteen', );

    private $_tens_nums = array(2 => 'twenty', 3 => 'thirty', 4 => 'forty', 5 => 'fifty', 6 => 'sixty', 7 => 'seventy', 8 => 'eighty', 9 =>
        'ninety', );

    private $_chunks_nums = array(1 => 'thousand', 2 => 'million', 3 => 'billion', 4 => 'trillion', 5 => 'quadrillion', 6 => 'quintrillion', 7 =>
        'sextillion', 8 => 'septillion', 9 => 'octillion', 9 => 'nonillion', 9 => 'decillion', );

    function __construct($number) {
        $this->_original = trim($number);
        $this->parse();
    }

    public function parse($new_number = NULL) {
        if($new_number !== NULL) {
            $this->_original = trim($new_number);
        }
        if($this->_original == 0) return 'Zero';

        $num = str_split($this->_original, 1);
        krsort($num);
        $chunks = array_chunk($num, 3);
        krsort($chunks);

        $final_num = array();
        foreach ($chunks as $k => $v) {
            ksort($v);
            $temp = trim($this->_parse_num(implode('', $v)));
            if($temp != '') {
                $final_num[$k] = $temp;
                if (isset($this->_chunks_nums[$k]) && $this->_chunks_nums[$k] != '') {
                    $final_num[$k] .= ' '.$this->_chunks_nums[$k];
                }
            }
        }
        $this->_parsed_number_text = implode(', ', $final_num);
        return $this->_parsed_number_text;
    }

    public function __toString() {
        return $this->_parsed_number_text;
    }

    private function _parse_num($num) {
        $temp = array();
        if (isset($num[2])) {
            if (isset($this->_single_nums[$num[2]])) {
                $temp['h'] = $this->_single_nums[$num[2]].' Hundred';
            }
        }

        if (isset($num[1])) {
            if ($num[1] == 1) {
                $temp['t'] = $this->_teen_nums[$num[0]];
            } else {
                if (isset($this->_tens_nums[$num[1]])) {
                    $temp['t'] = $this->_tens_nums[$num[1]];
                }
            }
        }

        if (!isset($num[1]) || $num[1] != 1) {
            if (isset($this->_single_nums[$num[0]])) {
                if (isset($temp['t'])) {
                    $temp['t'] .= ' '.$this->_single_nums[$num[0]];
                } else {
                    $temp['u'] = $this->_single_nums[$num[0]];
                }
            }
        }
        return implode(' and ', $temp);
    }
}

?>

alert.php

&lt;?php
$page = &quot;script.html&quot;;
    header(&quot;Refresh: 4; URL=\&quot;&quot; . $page . &quot;\&quot;&quot;);
    echo &quot;&lt;hr align='center' size=\&quot;4\&quot; color=\&quot;orange\&quot;&gt;&lt;br&gt;&lt;b&gt;&lt;font face=\&quot;arial, sans-serif\&quot;&gt;&lt;font color=blue&gt;OOPS! TRY AGAIN!&lt;/b&gt;&lt;/font&gt;&lt;/font&gt;&lt;p&gt;&quot;;
    echo &quot;<a href="http://translate.google.com/translate_tts?tl=en&#038;q=oops!try+again!">http://translate.google.com/translate_tts?tl=en&#038;q=oops!try+again!</a>&quot;;
?&gt;

 

YAWCAM

In order to view the LED switiching on and off a webcam setup is required as in Part I.

Trying out

We are now ready to open index.html. Point your Chrome 11 browser to http://localhost/index.html. Tou may now see the following (after you have directed your webcam to the AXE050 board):

Main Page Showing PICAXE050

Simply click on the microphone and this should prompt you to speak.

Say the words “light on” and if the answer is right processing.php link will be opened. The browser records your voice input, it contacts Google Servers to do the translation into text for you, and the value will be put in the text field. The page processing.php will be opened if everything is OK as shown below:

Page showing LIGHT is on and voice message

In backgroung status.exe and edon.exe will be run for a short duration and a LED on the AXE050 will switch on. The processing page will automatically redirect to index.html page which will now show a live image of AXE050 with a LED switched on:

Main page shows a duly lit LED

If the command is not recognised you will prompted to say it again, as page alert.php will open and you will hear the robotic voice from Google saying “Oops! Try again!”:

Alert of Unrecognised Speech

The page will be shown briefly redirecting to main page index.html. If the command “light on” is repeated, even tough the LED is already on, the following page will be shown:

Already on alert and message

This page will redirect also to index.html.

Now try out the voice command “status” and the page will show the actual status of LED and local temperature:

Local status of LEd and temperature and message

Finally, try voice commands “light off”, “temperature” and “status” when light is both “on”! and “off”. If all is functioning correctly. all these voice commands will be recognised, and there will be interactivity when commands are repeated for cases such as of “light on” and “light off”.

Further work

More vocabulary  of voice recognition can be added improving speech recognition. Databases can be set up, not only to log issued commands, but also to obtain local conditons, and general status. Additional sensors can be added, such as humidity and light (LDR on AXE050 can be used to measure light at certain extaint). Graphs can be displayed, and trigger points set, adding support for email and/or sms when  events or alarms occur.

Conclusions

Voice recognition has been achieved and demonstrated using Google servers (that effectively are becoming Cloud voice servers), obtaining control and interactivity with a real world application. In less than a decade, voice recognition is starting to come to age, although is still not there fully yet. An exciting perspective that opens up, is the near future availability of speech recognition in mobile phones and more and more devices responding to voice commands.

Advertisements

About Tayeb

Electronics engineer, part-time webmaster and owner of "Aliatron", a tech-oriented company registered in Portugal and Mozambique.
This entry was posted in Electronic Projects, Internet of Things and tagged , , , , , . Bookmark the permalink.

8 Responses to Voice Operated Internet Control of a PICAXE (Part II)

  1. Pingback: Voice Operated Internet Control of a PICAXE (Part I) | RedAcacia

  2. Redd says:

    Hello Tayeb,
    I was impressed by your projects and while I was looking through them, I thought if you could help me. I have a home alarm system (to be made in LabView VI) project and I need help with it as I have been encountering many obstacles. Please let me know if this is possible. I await your reply.

  3. drodecker says:

    Impressive. I would imagine you could make this app completely hands free. Instead of requiring the user to click the input microphone, there is probably a way to have the page auto-fire the microphone capture button upon page_load as well as on a continuous, repetitive basis.

    • Tayeb says:

      Welcome to my blog. At the moment one cannot make it automatic. Hopefully google will allow it in the future.

      • Tayeb says:

        Welcome to my blog. I don’t know sincerely how to help you. I shared here CentOS solution as I was also learning how to do it and since it worked perfectly others could benefit

  4. i had some problems with the Picaxe18M microcontroller, but I got it all setup and its fun to play with! Thanks for the information your page helped me out 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s