Petoi Doc Center
🇺🇸English
🇺🇸English
  • Welcome to Petoi Doc Center
  • Getting Started Guide
  • 🙋‍♂️FAQ(Frequently Asked Questions)
  • Petoi robot joint index
    • Joint Pins on NyBoard
      • Nybble
      • Bittle
    • Joint Pins on BiBoard V0
      • Bittle X
      • Bittle X+Arm
    • Joint Pins on BiBoard V1
      • Bittle X
      • Bittle X+Arm
      • Nybble Q
  • Bluetooth Connection
    • BiBoard
    • NyBoard
  • Upload Firmware
    • NyBoard
    • BiBoard V0
    • BiBoard V1
  • Joint Calibration
  • Infrared Remote
    • Remote Controller
  • Mobile App
    • Introduction
    • Calibrator
      • Nybble
      • Bittle
    • Controller
  • Desktop APP
    • Introduction
    • Firmware Uploader
      • NyBoard
      • BiBoard V0
      • BiBoard V1
    • Joint Calibrator
      • NyBoard Preparation
      • BiBoard Preparation
      • Nybble
      • Bittle / Bittle X
        • Bittle (NyBoard)
        • Bittle X (BiBoard V0)
        • Bittle X (BiBoard V1)
      • Bittle X+Arm
        • BiBoard V1
        • BiBoard V0
    • Skill Composer
      • NyBoard Connection
      • BiBoard Connection
      • Interface
        • Nybble
        • Bittle / Bittle X
        • Bittle X+Arm
    • Tools
  • Block-based programming
    • Petoi Coding Blocks
      • NyBoard Preparation
      • BiBoard Preparation
    • Block-based Coding Curriculum - Learn Quadruped Robotics for Beginners
    • Python coding mode in Mind+
    • Generic Arduino Uno Blocks
    • Install Mind+ on Chromebook
  • Arduino IDE
    • Upload Sketch for NyBoard
    • Upload Sketch for BiBoard
    • Calibrate the joints with Arduino IDE
    • Serial Monitor
    • C++ Curriculum: Learn Quadruped Robotics for Beginners
    • Install Arduino IDE on Chromebook
  • Free Curriculum
    • 📚Download
  • APIs
    • 🖇️Serial Protocol
      • Feedback servos
      • Nested task queue and signal generator
    • 🐍Python API
    • 🐛8266 MicroPython controller
      • Run MicroPython on ESP8266
      • Setup WebREPL
      • Using the ESP-NOW protocol
    • 🦎8266 Arduino C Controller
    • ©️C++ API
    • 🍓Raspberry Pi serial port as an interface
      • For BiBoard V1
    • 💻Set up Development Environment on Chromebook
    • 🤖ROS
  • Nyboard
    • Overview
    • NyBoard V1_0
    • NyBoard V1_1 & NyBoard V1_2
  • BIBOARD
    • BiBoard V0 Guide
    • BiBoard Extension Hat
    • Demo Applications
      • 1.GPIO port
      • 2.Serial port
      • 3.Analog-digital converter
      • 4.Digital-Analog Converter
      • 5.EEPROM (Electrically Erasable Programmable read only memory)
      • 6.Gyro IMU(MPU6050)
      • 7.Infrared remote control
      • 8.PWM(Pulse Width Modulation)
      • 9.Servo(under construction)
      • 10.Classic Bluetooth serial port SPP
      • 11.Bluetooth low energy (BLE) serial port pass-through
      • 12.File system SPIFFS
      • 13.Add hardware partition configuration option in Arduino IDE
      • 14.Play MP3
      • 15.The usage of Wi-Fi OTA(Over-The-Air)
    • BiBoard V1 Guide
  • Communication Modules
    • Introduction (For NyBoard)
    • USB Uploader (CH340C or CH343G)
    • Dual Mode Bluetooth
    • WiFi module ESP8266
      • ESP8266 + Python Scripts Implement wireless crowd control
  • Extensible Modules
    • Introduction
    • MU Camera
    • Ultrasonic Sensor
    • Light Sensor
    • Touch Sensor
    • Gesture Sensor
    • PIR Motion Sensor
    • IR Distance Sensor
    • Voice Command Module
    • Petoi AI Vision Module
    • Advanced development and application of AI vision modules
      • Model Training
      • Model quantification
      • Model deployment
      • Training on the COCO DIY dataset
    • Robot Arm
      • Upgrade your older Bittle/Bittle X for the robotics arm gripper
    • 🎮Joystick with Micro:Bit
  • Applications
    • Melody Creation
    • Skill Creation
    • OpenCat Imitation Tutorial
    • Programmable Puppet Character
    • Tutorial for simulating Bittle In Isaac Sim
  • History
    • Upload Sketch For NyBoard (software 1.0)
  • Technical Support
    • 💾Supporting Application and Software
    • 🔧Burn Bootloader for NyBoard
    • 🛠️Useful Tools
    • 🔋Battery
  • Useful Links 🕸
    • 🔭Home of Petoi Robots
    • 🛒Shop Coding Robots
    • 💿GitHub of OpenCat
    • 🎪PetoiCamp (Forum)
    • 📽️Petoi Robot Videos
    • 📬Users' repositories
Powered by GitBook
On this page
  • Demo video
  • Function introduction
  • NyBoard
  • Hardware setup
  • Software setup
  • BiBoard
  • Hardware setup
  • Software setup
  • Play with the voice commands.
  • Common use cases
  • How to debug if the voice command doesn't work
  • Record customized voice commands
  • Advanced usage for developers
  • Understand the principle
  • The test sketch
  • Serial interface
  • How to design new reactions

Was this helpful?

Export as PDF
  1. Extensible Modules

Voice Command Module

PreviousIR Distance SensorNextPetoi AI Vision Module

Last updated 1 month ago

Was this helpful?

Demo video

Function introduction

Using this module, you can control the Petoi robot to do various skills through voice without using wake words. Currently, the module supports 40 fixed voice commands in two languages (English and Chinese) and ten customized commands by recording any sound clips.

NyBoard

Hardware setup

The module can be installed on NyBoard via the Grove interface.

Connecting to the NyBoard with wire as shown in the following picture, connect to the NyBoard Grove interface, which includes D8, D9:

Software setup

Petoi Desktop App

Arduino IDE

BiBoard

Hardware setup

Software setup

1. Upload Firmware

  • Petoi Desktop App

Please select the correct Product type, Borard version, and Serial port. The mode should be Standard, so press the Upgrade the Firmware button. for example, Bittle, BiBoard_V0_2, COM5 as follows:

  • Arduino IDE

Use the latest OpenCatESP32 code to finish the setup. For example, to modify the code for Bittle as shown below:

2. Switch mode

After uploading, the firmware is in Voice mode by default. If you need to switch, please open the serial monitor and input the serial command XA to switch to Voice mode.

Play with the voice commands.

Common use cases

When the robot is restarted, the voice module will be automatically reset to its default language setting. The robot's default language setting is English.

This feature is exclusively designed for robots using BiBoard as their mainboard.

1. Set the default language

There are two ways to set the default language:

  • Using the serial commands

      • To set the default language to English, use the command: XAa.

      • To set it to Chinese, use the command: XAb.

      • To set the default language to English, create a mobile app command called English and use the code: X65,97.

      • To set it to Chinese, create a mobile app command called Chinese and use the code: X65,98.

  • Using the voice commands

    • To set the default language to English, you need to say the voice command "Lizheng" (phonetic) first and then say "Bing-Bing" (phonetic).

    • To set it to Chinese, you need to say the voice command "Attention" first and then say "Di-Di"(phonetic).

2. Switch the language temporarily

  • To switch to English, you can say "Bing-Bing" (phonetic)

  • Switch to Chinese, you can say "Di-Di" (phonetic)

Note: Use this method after rebooting the robot. The language will switch to the default language setting. Therefore, if you accidentally switch the language mode, you can restore the default language setting by restarting the robot.

3. Turn on/off the voice command functionality(audio response and robotics reaction)

To turn on

  • Speak Play sound

To turn off

  • Speak Be quiet

4. Use the predefined voice commands

You can refer to the list of available voice commands shown below:

To avoid inadvertently triggering the robot to respond to voice commands, you can say Be quiet to the robot to disable the voice module, such as when talking with others.

For example, you can share your behavior like this:

How to debug if the voice command doesn't work

In some cases, the voice module may not respond to your voice. Please check the following:

1. On Bittle X, the dial switch on the bottom of the BiBoard extension hat is dialed to Voice Command.

2. Say Play sound to check if the robot responds with Do-Re-Mi. Sometimes, the voice may be accidentally set to muted mode triggered by Be Quiet.

3. If the module doesn't make any sound with Play sound, say Bing-Bing to switch to English mode. You may try different tones and speeds to say Bing-Bing. The robot should respond with Switch English if not in English mode. It won't react with anything if it's already in English.

4. If the voice module still doesn't make any sound, you can try to reset it in our software tools.

Mobile app:

From version 1.2.0 of the mobile app, you can create a new button with the compound code:

^X65,99;!1000;X65,98;!1000;X65,97

To reset the voice module to English mode.

X65,97 or input XAa in the serial monitor is equivalent to saying Bing-Bing, but excludes the chance that the voice is not recognized. Then you can try to say Play sound again.

Desktop app:

From version 1.2.1, you can use the debugger tool to reset the voice module.

The above steps validate that the voice module is working. It's powered separately from the motion unit and should work regardless of the robot's status.

  1. Next, if you say Hello, the robot should wave its hand and validate that the complete reaction loop is good. Then, you can try other voice commands.

  2. Try powering off the mainboard by disconnecting the USB data cable, long-pressing the battery's button, and then re-powering the mainboard.

If the above steps cannot fix the problem, contact support@petoi.com for help.

Record customized voice commands

When the robot works in English mode, you can speak Start learning (or input the serial command XAe in the serial monitor) into the custom voice command mode and record your voice commands in order.

You can record up to 10 voice commands, each with no more than six syllables.

To exit the custom voice command mode in the middle, you can speak Stop learning (or input the serial command XAf in the serial monitor).

After exiting the custom voice command mode, speak one of the recorded voice commands to trigger the reaction.

Speak Clear the learning data to delete all the recordings at once (you cannot delete a specific recording).

There are ten skill strings as custom replies already defined (but only the first five can see the actual reaction of the robot because they are predefined serial commands ) in the voice.h:

const char voice1[] PROGMEM = "T";                                    //call the last skill data sent by the Skill Composer
const char voice2[] PROGMEM = "kpu1";                                 //single-handed pushup
const char voice3[] PROGMEM = "m0 80 0 -80";                          //move head
const char voice4[] PROGMEM = "kmw";                                  //moonwalk
const char voice5[] PROGMEM = "b14,8,14,8,21,8,21,8,23,8,23,8,21,4";  //twinkle star
const char voice6[] PROGMEM = "6th";
const char voice7[] PROGMEM = "7th";
const char voice8[] PROGMEM = "8th";
const char voice9[] PROGMEM = "9th";
const char voice10[] PROGMEM = "10th";

The response actions (kpu1 means single-handed pushups, kmw means moonwalk) are already defined in the program.

Other serial commands are also supported as responses, such as joint movements(e.g. m0 80 0 -80 means shaking the head left and right) and playing a melody(e.g. b14,8,14,8,21,8,21,8,23,8,23,8,21,4)

To use these custom replies above, you need to enter the custom voice command mode, record ten voice commands (such as Single-handed Pushup, Shake Head, Moonwalk, Twinkle Star), and then exit the custom voice command mode.

If you have recorded a voice command and the corresponding custom reply is not a predefined serial command((e.g.,5th), there is no actual demonstration effect; it only prints a simple message on the serial monitor when you speak the corresponding voice command.

Advanced usage for developers

Understand the principle

  1. Convert the voice command collected by the microphone in the module into a serial command.

  2. Send the serial command to the mainboard MCU through the soft serial port Serial2.

  3. After receiving the serial command, the MCU parses it into the corresponding skill command, and finally, the reaction module, according to the skill command, controls the robot to respond accordingly.

Upload the demo sketch testVoiceCommander.ino, and you can see every serial command that is sent to MCU(including the custom voice command if you have recorded it)

After you speak the voice command to the robot, the Returned value (X A 11 or X A 21 kup) is the corresponding serial command sent to the mainboard MCU. The third number(11 or 21) is an invisible character. To understand it, we convert it to a numeric value and print it out.

The test sketch

Serial interface

There are seven related serial commands for configuration; you can input them into the serial monitor.

Serial Command
Mobile App Button Code
Function

XAa

X65,97

Set the default language to English

XAb

X65,98

Set the default language to Chinese

XAc

X65,99

Turn on the reply tone and enable reaction

XAd

X65,100

Turn off the reply tone and disable reaction

XAe

X65,101

Enter custom voice command mode

XAf

X65,102

Exit custom voice command mode

XAg

X65,103

Delete all the custom voice commands

After inputting the command above in the message box, press Enter to send the command to the robot.

How to design new reactions

  • Using the task queue to create a sequence of motions, please refer to the source code in the voice.h as below:

if (index < 61) {
    token = raw[3];         //T_SKILL;
    shift = 4;              //3;
}
const char *cmd = raw.c_str() + shift;
tQueue->addTask(token, shift > 0 ? cmd : "", 2000);
char end = cmd[strlen(cmd) - 1];
if (!strcmp(cmd, "bk") || !strcmp(cmd, "x") || end >= 'A' && end <= 'Z' || end == 'x') {
    tQueue->addTask('k', "up");

tQueue is the task queue defined in OpenCat.h; using the method "addTask" of this object, the robot can do some simple skills sequentially as a custom voice command response.

  1. Modify voice.h to bind the customized voice command to the new skills: insert 'k'+the new skill name into the string variable(e.g.voice1[]If you want to bind the first customized voice command )

    const char voice1[] PROGMEM = "kskill1";     // "k" is the token for skill, skill1 is the new skill name. 

You can use the within the Petoi Desktop App.

You can use to upload and modify the source code.

The module code is integrated into the project. Uncomment the line #define VOICE in the OpenCat.ino, as shown in the figure below. Then, use Arduino IDE to upload the sketch to the robot's main board. The robot will work in Voice mode. This sketch will allow the robot to behave under voice commands.

It is built into the as follows:

You can use the within the Petoi Desktop App.

You can use to (OpenCatEsp32.ino).

Open the :

In the :

Create called "Enable voice" and use the code: X65,99

Create called Disable voice and use the code: X65,100

See for the latest version.

If the above voice commands don't take effect in English mode, try to use the mobile app and create a new button with the code: X65,100, or input XAd in the to disable the voice module.

You can say Play sound to the robot to enable the voice module.

Use the mobile app and create a new button with the code: X65,99, or input XAc in the to enable the voice module.

The voice command Climb-up is a challenge for you. You can by yourself. Then, you can post it on the or email support@petoi.com. We may adopt it in our official firmware and send you a gift!

X65,99, or input XAc in the enables the voice module.

If the module is not in English mode, you can speak Bing-Bing (or input the serial command XAa in the ) to switch to English mode.

You can open to check the raw return values of every voice command.

The test sketch is in the OpenCat repository on GitHub (specific path: OpenCat/ModuleTests/testVoiceCommander). You can visit our GitHub repository to download the complete code, as shown in the following picture:

For the robot in , to improve the utilization rate of custom voice control commands, you can modify the last six skill strings to the skill names with actual action responses.

Using and binding the customized voice command to the new skills

Use SkillComposer to design new skills and then them into InstinctX.h

Firmware Uploader
Arduino IDE
OpenCat
BiBoard V0's extension hat
Firmware Uploader
Arduino IDE
upload the sketch
serial monitor
mobile app
a mobile app command
a mobile app command
this doc
serial monitor
serial monitor
design the behavior
Petoi Forum Challenge
serial monitor
serial monitor
the serial monitor
https://github.com/PetoiCamp/OpenCat
Voice mode
the Skill Composer
export
Debugger | Petoi Doc Center
Logo