Tested on Nokia N97 and Blackberry Bold 9700.

Introduction

After having recently designed a virtual gamepad to control a robot, making use of a serial bluetooth device, I thought why not do the same with a cell phone.

My Nokia N97 has bluetooth and using it to control the robot seemed to be the most attractive proposal.

My first development tool was Psy60 (Nokia’s Python), to control the robot, but soon I found out that the bluetooth libraries seemed to be geared towards controlling a PC.

So, effectively having gained experience with Processing, my next immediate thought was to use Mobile Processing, after having read about a Roomba hacking and a patch for serial bluetooth library.

Please check addenda below: tested and verified on Blackberry Bold 9700.

Required equipment

 A 8051 robot “Robo-51” (could be a robot with any other micrcontroller) with ZX-Bluetooth module, running a firmware for bluetooth control (or any other serial control), and PC with Windows and Mobile Processing software.

8051-based robot “Robo-51”

Serial bluetooth module “ZX-Bluetooth”

Please check the Virtual Gamepad project. The sketch as explained in the first part of code is based on the Roomba hacking and ideas from a NXT Symbian project.  It uses the patched bluetooth library of Roomba project. All the patch does is to allow one to specify short UUIDs when searching bluetooth devices.

Mobile Processing Sketch

So here is the sketch:

// Based on original code Tic-Tac-Toe code by Francis Li, found at
// http://mobile.processing.org/learning/example.php?name=tictactoe
// and on roombaCtrl by Tod E. Kurt, tod@todbot.com
// http://roombahacking.com, and extra coding ideas from
// Pedro Miguel   ap0cn3t@gmail.com http://nxt-symbian.sourceforge// .net
// code adapted for Robo-51 by Tayeb Habib, tayeb.habib@gmail.com
// https://redacacia.wordpress.com (check the site for more projects)
// compatibility tested for Nokia N97 and Blackberry Bold 9700

// This needs my patched bluetooth library that allows short UUIDs
// found at http://roombahacking.com

import processing.bluetooth.*;

// state machine sates
final int STATE_START    = 0;
final int STATE_FIND     = 1;
final int STATE_DRIVE    = 2;

// state machine state var
int state;

// bluetooth library
Bluetooth bt;
// discovered services
Service[] services;
// status message
String msg;
// connection to robo51
Client cl;

PFont font;

// ------------------------------------------------------------------
// robo51 opcodes
// ------------------------------------------------------------------

byte[] fwd = new byte[]{ (byte)0x64 };   // ascii key letter "d"
byte[] bwd = new byte[]{ (byte)0x61 };   // ascii key letter "a"
byte[] left = new byte[]{ (byte)0x63 };   // ascii key letter "c"
byte[] right = new byte[]{ (byte)0x62 };   // ascii key letter "b"
byte[] stop = new byte[]{ (byte)0x73  };   // ascii key letter "s"

///
void setup() {
  font = loadFont();
  textFont(font);
  bt = new Bluetooth(this, Bluetooth.UUID_SERIALPORT);
  state = STATE_START;
}

void destroy() {
  background(255);
  fill(0);
  text("Exiting...", 2, 2, width - 4, height - 4);
  bt.stop();
}

//
void draw() {
  background(0xcc,0xcc,0xff);

  if (state == STATE_START) {
    fill(0);
    textAlign(LEFT);
    text("Welcome to Robo-51\n\nPress a key to search the robot", 2, 2, width - 4, height - 4);
  }
  else if (state == STATE_FIND) {
    fill(0);
    textAlign(LEFT);
    if (services == null) {
      text("Looking for Robo-51...\n\n" + msg, 2, 2, width-4, height-4);
    }
    else {
      String list = "Ports found:\n";
      for (int i = 0, len = length(services); i < len; i++)
        list += i + ". " + services[i].device.name + "\n";
      text(list, 2, 2, width-4, height-4);
    }
  }
  else if (state == STATE_DRIVE) {
    noFill();
    text("Robo-51 control:\n"+
         "Use arrows to drive\n"+
         "press A, or B, or C, or D to stop\n"+
         msg, 2, 2, width-4, height-4);
  }

}

//
void libraryEvent(Object library, int event, Object data) {
  if (library == bt) {
    switch (event) {
    case Bluetooth.EVENT_DISCOVER_DEVICE:
      msg = "Found device at: " + ((Device) data).address + "...";
      break;
    case Bluetooth.EVENT_DISCOVER_DEVICE_COMPLETED:
      msg = "Found " + length((Device[]) data) + " devices, looking for serial port...";
      break;
    case Bluetooth.EVENT_DISCOVER_SERVICE:
      msg = "Found serial port on " + ((Service[]) data)[0].device.address + "...";
      break;
    case Bluetooth.EVENT_DISCOVER_SERVICE_COMPLETED:
      services = (Service[]) data;
      msg = "Search complete. Pick one.";
      break;
    case Bluetooth.EVENT_CLIENT_CONNECTED:
      cl = (Client) data;
      msg = "wtf. client connectd.";
      break;
    }
  }
}

//

//
void keyPressed() {
  if (state == STATE_START) {
    services = null;
    bt.find();
    state = STATE_FIND;
    msg = "Looking for devices...";
  }
  else if (state == STATE_FIND) {
    if (services != null) {
      if ((key >= '0') && (key <= '9')) {
        int i = key - '0';
        if (i < length(services)) {
          msg = "connecting...";
          cl = services[i].connect();
          state = STATE_DRIVE;
          msg = "ready";
        }
      }
    }
  }
  else if (state == STATE_DRIVE) {

    parse_robo51key(keyCode,key);

  }

}

void parse_robo51key(int akeyCode, int akey) {
  switch(akeyCode) {
    case UP:
      msg = "forward";
      robo51_send(fwd);
      break;
    case DOWN:
      msg = "backward";
      robo51_send(bwd);
      break;
    case LEFT:
      msg = "left";
      robo51_send(left);
      break;
    case RIGHT:
      msg = "right";
      robo51_send(right);
      break;
    case CENTER:
      msg = "stop";
      robo51_send(stop);
      break;
    default:
      msg = "stop";
      robo51_send(stop);
      break;

    } // switch(keyCode)
}

void robo51_send(byte[] buffer){
  cl.write(buffer);
  cl.flush();
}

Deploying

If you are absolute beginner with Mobile Processing I suggest you follow the tutorial “How to write apps with Mobile Processing on your N95” (if the link does not work please check: Mobile Application Development with Mobile Processing) . The tutorial will teach you how to make simple Mobile Processing applications and to deploy into your cell phone.

The control program

The program controls Robo-51 from Inex with the commands:

Key letter “d” foward, “a” backwards, “c” turn leftwards, “b” turn rightwards and “s” stop.  The commands are defined in the above sketch under “Robo-51 opcodes”. Arrow keys on N97 keyboard control the robot. The key ENTER on the N97 keyboard is used as FIRE button to stop the robot

When the program is launched, after it identifies the bluetooth devices, SPP device will appear with appropriate number 0, 1, 2 or whatever tag before it. One has to choose SPP with numeric option on N97 keyboard.

The firmware that accepts commands in above sketch was written by Pedro Vicente, a Summer vacation trainee in my company Aliatron. Pedro has already published his project here. The following video, recently published in youtube by Raquel (Pedro’s girlfriend), shows “Robo-51” in action:

The program on the PC’s screen shown in the video is an application written by Pedro that allows web control of Robo-51.

I hope to next control the robot with a Psy60 program.  When it is done, I will publish it here. I have mentioned N97 as the program was tested with Nokia’s N97. Perhaps the program will work perfectly in other Nokia cell phones with Symbian OS (N97’s OS is Symbian).

——————————

Addenda on 06/07/2011:

I have tested the unaltered jar file made for Nokia N97, on my Blackberry Bold 9700 (v.6.6.0.124) by installing the jar file, after copying it into the cell phone’s memory. The robot responded well, and moved forward, backward, left and right, and stopped as expected with respective voice indicating the action. All the movements of the robot were made by playing with Blackberry’s joystick, i.e. track pad. The bluetooth connection of the Blackberry and the robot’s bluetooth serial module were duly detected. When the bluetooth connection on the Blackberry was not on, the cell phone asked for permission to switch it on, as was expected. Summing up: a fully satisfactory control of the robot is achieved with Blackberry Bold 9700. Implementations in other Bloackberry cell phones should give similar results.

——————————-

Addenda on 09/17/2010:

Adding sound to the robot control

In my previous project of Virtual Gamepad text-to-speech was added and so something similar would be interesting. So, effectively that’s what I did this time.

Text-to-speech

Though Nokia N97 supports text-to-speech, Mobile Processing has no library, as Psy60 has.

I didn’t want to record sounds and make them sound “robotic”, so I searched the Web, and found a freeware that does it. Download “TTSReader” and create .wav files for all the commands. All commands can be downloaded here (you will need 7z to uncompress):

Sound files (kept at Bitbucket, a free repository for smal users)

Copy these files into a sub-directory that you will name data in your MobileProcessing project folder (found normally under c:\my documents\.

Now you are ready to follow the above steps with following new Mobile Processing sketch:

// New sketch with sound support
// Based on original code Tic-Tac-Toe code by Francis Li, found at
// http://mobile.processing.org/learning/example.php?name=tictactoe
// and on roombaCtrl by Tod E. Kurt, tod@todbot.com
// http://roombahacking.com, and extra coding ideas from
// Pedro Miguel, ap0cn3t@gmail.com http://nxt-symbian.sourceforge.net
// code adapted for Robo-51 by Tayeb Habib, tayeb.habib@gmail.com
// https://redacacia.wordpress.com (check the site for more projects)
// compatibility tested for Nokia N97

// This needs my patched bluetooth library that allows short UUIDs
// found at http://roombahacking.com and msound library found at
// http://mjs.darkgreenmedia.com/site/

import processing.bluetooth.*;
import mjs.processing.mobile.msound.*;

MSound mySound;

// state machine sates
final int STATE_START    = 0;
final int STATE_FIND     = 1;
final int STATE_DRIVE    = 2;

// state machine state var
int state;

// bluetooth library
Bluetooth bt;
// discovered services
Service[] services;
// status message
String msg;
// connection to robo51
Client cl;
PFont font;

// ------------------------------------------------------------------
// robo51 opcodes
// ------------------------------------------------------------------

byte[] fwd = new byte[]{ (byte)0x64 };   // ascii key letter "d"
byte[] bwd = new byte[]{ (byte)0x61 };   // ascii key letter "a"
byte[] left = new byte[]{ (byte)0x63 };   // ascii key letter "c"
byte[] right = new byte[]{ (byte)0x62 };   // ascii key letter "b"
byte[] stop = new byte[]{ (byte)0x73  };   // ascii key letter "s"

///
void setup() {
  font = loadFont();
  textFont(font);
  bt = new Bluetooth(this, Bluetooth.UUID_SERIALPORT);
  state = STATE_START;
  mySound = MSoundManager.loadSound("start.wav");
  mySound.play();
}

void destroy() {
  background(255);
  fill(0);
  text("Exiting...", 2, 2, width - 4, height - 4);
  bt.stop();
}

//
void draw() {
  background(0xcc,0xcc,0xff);

  if (state == STATE_START) {
    fill(0);
    textAlign(LEFT);
    text("Welcome to Robo-51\n\nPress a key to search the robot", 2, 2, width - 4, height - 4);
  }
  else if (state == STATE_FIND) {
    fill(0);
    textAlign(LEFT);
    if (services == null) {
      text("Looking for Robo-51...\n\n" + msg, 2, 2, width-4, height-4);
    }
    else {
      String list = "Ports found:\n";
      for (int i = 0, len = length(services); i < len; i++)
        list += i + ". " + services[i].device.name + "\n";
      text(list, 2, 2, width-4, height-4);
    }
  }
  else if (state == STATE_DRIVE) {
    noFill();
    text("Robo-51 control:\n"+
         "Use arrows to drive\n"+
         "press ENTER to stop\n"+
         msg, 2, 2, width-4, height-4);
  }

}

//
void libraryEvent(Object library, int event, Object data) {
  if (library == bt) {
    switch (event) {
    case Bluetooth.EVENT_DISCOVER_DEVICE:
      msg = "Found device at: " + ((Device) data).address + "...";
      break;
    case Bluetooth.EVENT_DISCOVER_DEVICE_COMPLETED:
      msg = "Found " + length((Device[]) data) + " devices, looking for serial port...";
      break;
    case Bluetooth.EVENT_DISCOVER_SERVICE:
      msg = "Found serial port on " + ((Service[]) data)[0].device.address + "...";
      break;
    case Bluetooth.EVENT_DISCOVER_SERVICE_COMPLETED:
      services = (Service[]) data;
      msg = "Search complete. Pick one.";
      break;
    case Bluetooth.EVENT_CLIENT_CONNECTED:
      cl = (Client) data;
      msg = "wtf. client connectd.";

      break;
    }
  }
}

//

//
void keyPressed() {
  if (state == STATE_START) {
    services = null;
    bt.find();
    state = STATE_FIND;
    msg = "Looking for devices...";
  }
  else if (state == STATE_FIND) {
    if (services != null) {
      if ((key >= '0') && (key <= '9')) {
        int i = key - '0';
        if (i < length(services)) {
          msg = "connecting...";
          cl = services[i].connect();
          state = STATE_DRIVE;
          msg = "ready";
        }
      }
    }
  }
  else if (state == STATE_DRIVE) {

    parse_robo51key(keyCode,key);

  }

}

void parse_robo51key(int akeyCode, int akey) {
  switch(akeyCode) {
    case UP:
      mySound = MSoundManager.loadSound("forward.wav");
      mySound.play();
      msg = "forward";
      robo51_send(fwd);
      break;
    case DOWN:
      mySound = MSoundManager.loadSound("backward.wav");
      mySound.play();
      msg = "backward";
      robo51_send(bwd);
      break;
    case LEFT:
      mySound = MSoundManager.loadSound("left.wav");
      mySound.play();
      msg = "left";
      robo51_send(left);
      break;
    case RIGHT:
      mySound = MSoundManager.loadSound("right.wav");
      mySound.play();
      msg = "right";
      robo51_send(right);
      break;
    case FIRE:
      mySound = MSoundManager.loadSound("stop.wav");
      mySound.play();
      msg = "stop";
      robo51_send(stop);
      break;
    default:
      mySound = MSoundManager.loadSound("stop.wav");
      mySound.play();
      msg = "stop";
      robo51_send(stop);
      break;

    } // switch(keyCode)
}

void robo51_send(byte[] buffer){
  cl.write(buffer);
  cl.flush();
}

You will now need to Export and Deploy! That’s it!
After installing in your cell phone you will now have a very interactive control. Have fun!