English text to speech on a PIC microcontroller

I have always been a fan of the TTS256 - a tiny but great English text-to-speech IC based on a 8-bit microprocessor for embedded voice applications. Unfortunately, the TTS256 has been out of production for a long time and despite better technology being developed over the years, chip manufacturers do not seem to be interested in developing a similar or better text to speech IC, leaving the average electronics hobbyists searching eBay for second-hand TTS256 ICs, often listed at unreasonable price.

Nowadays, SpeakJet by Sparkfun and RoboVoice by Speechchips are among the few available text-to-speech modules for embedded projects. Both are priced at 20-30 USD and have pinout and interface commands similar to the TTS256. Although these speech modules come in handy, their price range seems a bit high for many projects. Hence, I decided to search for free alternatives.

Syntho and PICTalker

There are several open source text-to-speech projects for 8-bit microcontrollers such as Syntho and PICTalker, built for the PIC16F616 and PIC16F628 respectively. In both projects, one or more EEPROMs are used to store the phoneme database. The EEPROM size is around 64K for the PICTalker project and is made smaller by using innovative compression techniques in the Syntho project. Both projects require phonemes (and not English text) to be sent before they can be pronounced. This is due to the lack of a rule database to convert text to phonemes, presumably due to the limited amount of memory available. These solutions are somewhat closer to the SPO256, which requires phonemes as input, rather than the TTS256, which accepts English text.

If you don't know what phonemes are, read this on Wikipedia. They are simply the phonetic representations of a word's pronunciation. There are approximately 44 phonemes in English to represent both vowels and consonants.

Below is a voice sample of the Syntho project, which is trying to say "I am a really cheap computer": 



As expected, the voice sounds too mechanical and can hardly be understood.

Arduino TTS library

Next I came across another TTS library made for the Arduino and decided to give it a quick try to test the speech quality. As I do not have an Arduino board available, I ported them to a Visual Studio 2012 C++ application which accepts English text as input and saves the resulting speech as a wave file. The ported code can be downloaded here. If you intend to use this code, take note that it only writes the wave data for the generated speech and ignores the wave file header. You will probably need a professional sound editor software such as Goldwave to play and examine the generated file. This is because calculating the exact total duration of the generated speech (required for creating the wave file header) is complicated and there is no need for me to attempt that since the code is only for testing.

This is the generated voice sample. It is trying to say "Hello Master, how are you doing? I am fine, thank you.":



Although the quality is obviously better than the PICTalker, it still sounds robotic and difficult to understand.

SAM (Software Automatic Mouth) project

My next attempt is to see if the same can be done on a PIC, with better speech quality. By chance, I came across SAM (Software Automatic Mouth), a tiny (less than 39KB) text-to-speech C program. The project website contains a tool to generate a demo voice from the text entered.

After getting the Windows source code to compile and run without issues in Visual Studio (download the project here), I decided to port the code to the PIC24FJ64GA002, which is surprisingly rather straightforward. The only challenge was to get all the 32-bit data types in the original source code ported properly to the 16-bit architecture of the PIC24 micro-controller, and to get the rule and phoneme database fit nicely into the PIC24FJ64GA002 available memory. Fortunately, the entire project when compiled uses just around 50% of the total program and data memory on the PIC24FJ64GA002, leaving available space for other codes.

You may be able to fit the project into the smaller PIC24FJ32GA002 by changing the project build options to use the large memory model during compilation:

However, in my experiment, the code compiled but ran erratically when using large memory model, perhaps due to pointer behavior differences. It is therefore better to compile with the default settings and use the PIC24FJ64GA002 (or one with more memory) to save the trouble and have more code space for other purposes.

The following is a recording of the generated speech for the sentence "This is SAM text to speech. I am so small that I will work also on embedded computers", when running on the PIC24 using PWM for audio output.



Below is a longer demo speech. Can you understand what it is trying to say?



As can be seen, the quality of the generated speech is much better - less mechanical, clearer pronunciation and easier to understand. Although the voice still sounds robotic and there are some mispronounced words, the overall quality should be good enough to be used in embedded projects, as a free alternative to current commercial text to speech solutions.

With some pitch adjustments, the PIC24 can also sing "The Star-Spangled Banner", the national anthem of the United States of America:



The complete ported code, as a MPLAB 8 PIC24FJ64GA002 project, can be downloaded here. The project also contains example codes for the SYN6288, a Chinese text-to-speech module.

See also

SYN6288 Chinese Speech Synthesis Module  
LD3320 Chinese Speech Recognition and MP3 Player Module

Read More »

LD3320 Chinese Speech Recognition and MP3 Player Module

After my previous success in getting the SYN6288, a Chinese text-to-speech IC, to produce satisfactory Chinese speech and pronouncing synthetic English characters, I purchased the LD3320, another Chinese voice module providing speech recognition as well as MP3 playback capabilities.

The module's Chinese voice recognition mechanism can be initialized with the Pinyin transliterations of the Chinese text to be recognized. The module will then listen to the audio sent to its input channel (either from a microphone or from the line-in input) to identify any voice that resembles the programmed list of Chinese words sent during initialization. Audio during MP3 playback is sent via the headphone/lineout (stereo) and speaker (mono) pins. Data communication with the module is done using either a proprietary parallel protocol or SPI.

The board I purchased comes with a condenser microphone and 2.54mm connection headers for easy prototyping:

Board Schematics

The detailed schematics of the board is below:

The connection headers on the breakout board expose several useful pins, namely VDD, GND, parallel/SPI communication lines and audio input/output pins. The detailed pin description can be found below, where ^ denotes an active low signal:

VDD          3.3V Supply
GND          Ground
RST^         Reset Signal
MD           Low for parallel mode, high for serial mode.
INTB^        Interrupt output signal
A0           Address or data selection for parallel mode. If high, P0-P7 indicates address, low for data.
CLK          Clock input for LD3320 (2-34 MHz).
RDB^         Read control signal for parallel input mode
CSB^/SCS^    Chip select signal (parallel mode) / SPI chip select signal (serial mode).
WRB^/SPIS^   Write Enable (parallel input mode) / Connect to GND in serial mode
P0           Data bit 0 for parallel input mode / SDI pin in serial mode
P1           Data bit 1 for parallel input mode / SDO pin in serial mode
P2           Data bit 2 for parallel input mode / SDCK pin in serial mode
P3           Data bit 3 for parallel input mode
P4           Data bit 4 for parallel input mode
P5           Data bit 5 for parallel input mode
P6           Data bit 6 for parallel input mode
P7           Data bit 7 for parallel input mode
MBS          Microphone Bias
MONO         Mono Line In 
LINL/LINR    Stereo Line In (Left/Right)
HPOL/HPOR    Headphone Output (Left/Right)
LOUL/LOUTR   Line Out (Left/Right)
MICP/MICN    Microphone Input (Pos/Neg)
SPOP/SPON    Speaker Ouput (Pos/Neg)

The LD3320 requires an external clock to be fed to pin CLK, which is already provided by the breakout board via a 22.1184 MHz crystal. No external components are needed, even for the audio input/output lines, as the breakout board already contains all the required parts.

To use SPI for communication, connect MD to VDD, WRB^/SPIS^ to GND and use pins P0, P1 and P2 for SDI, SDO and SDCK respectively. For simplicity, the rest of this article will use SPI to communicate with this module.

Official documentation (in Chinese only) can be found on icroute's website. The Chinese datasheet can be downloaded here. With the help of onlinedoctranslator, I made an English translation, which can be downloaded here.

Breakout board issues

Before you proceed to explore the LD3320, please be aware of possible PCB issues causing wrong signals to be fed to the IC and resulting in precious time wasted debugging the circuit. In my case, after getting the sample program to compile and run on my PIC microcontroller only to find out that it did not work, I spent almost a day checking various connections and initialization codes to no avail. I could easily have debugged till the end of time and still could not get it to work if I hadn't noticed by chance a 22.1184 MHz sine wave on the pin marked as WRB, raising suspicion that the PCB trace may have issues.

I decided to use a multimeter and cross-checked the connections between the labelled pins on the connection headers and the actual pins on the IC while referring to the LD3320 pin configuration described in the datasheet:

This is the pin description printed on the connection header at the back of the board:

 

To my surprise, apart from the GND/VDD pins which are fortunately correctly labelled (otherwise I could have damaged the module by applying power in reverse polarity), the rest of the pin labels on the left and right columns of the left connection header are swapped! For example, RSTB should be INTB, CLK should be WRB and vice versa. This explained why I got a clock signal on the WRB pin as their labels are swapped! The correct labelling for these pins should be:

For the right and bottom connection headers, the labelling is correct. However, further tests showed that the condenser microphone is connected in reverse polarity and that there are several other connection issues between the microphone and the LD3320. The connections on the PCB did not seem to match the board schematics, which could indicate a faulty PCB or a mismatched schematics. Either way, the microphone input still could not work even with the ECM replaced, and I could only get it to work using the line-in input (more on that later) after removing the ECM from the board. The presence of the microphone, even if unused, will disturb the line-in input channel and prevent the module from working.

Therefore, before you apply power to the board, check to make sure that the pin labelling is correct - or at least check that the VDD and GND pins are correctly labelled.  Also, your board may not have any issue or have a different issue than those described above.

Speech recognition

The only few examples I found for this IC are from coocox's LD3320 driver and some 8051 codes downloadable from here. By comparing the codes with the initialization protocol provided in the datasheet, the steps to use this module can be summarized below:

1. Reset the module by pulling the RST pin low, and then high for a short while.
2. Initialize the module for ASR (Automatic Speech Recognition) mode. In particular, set the input channel to be used for speech recognition. 
3. Initialize the list of Chinese words to be recognized. For each Chinese word, send the Pinyin transliteration of the word (without tone marks) in ASCII (e.g. bei jing for 北京) and an associated code (a number between 1 and 255) to identify this word. The codes for the words in the list need not be continuous and multiple words can have the same identification code.
4. Look for an interrupt on the INTB pin, which will trigger when a voice has been detected on the input channel.
5. When the interrupt happens, instruct the LD3320 to perform speech recognition, which will analyse the detected voice for any patterns similar to the list of Chinese words programmed in step 3. If a match is found, the chip will return the identification code associated with the word.
6. After a speech recognition task is completed, go back to step 1 to be ready for another recognition task.

To specify which input channel will be used for speech recognition, use register 0x1C (ADC Switch Control). Write 0x0B for microphone input (MICP/MIN pins), 0x07 for stereo input (LINL/LINR pins) and 0x23 for mono input (MONO pins).

In my tests, as the microphone input channel cannot be used due to the PCB issues mentioned above, I used the stereo input channels with an ECM and a preamplifier circuit based on a single NPN transistor. The output of this circuit is then connected to the LINL/LINR audio input pins of the LD3320. Below is the diagram of the preamplifier:

To achieve the highest recognition quality possible, several registers of the LD3320 are used to adjust the sensitivity and selectivity of the recognition process:

• Register 0x32 (ADC Gain) can be set to values between 00 and 7Fh. The greater the value, the greater the input audio gain and the more sensitive the recognition. However, higher values may result in increased noises and mistaken identifications. Set to 10H-2FH for noisy environment. In other circumstances, set to between 40H-55H.
• Register 0xB3 (ASR Voice Activity Detection). If set to 0 (disable), all sounds detected on the input channel will be taken as voice and trigger the INTB interrupt. Otherwise, INTB will only be triggered when a voice is detected on the audio input channel whereas other static noises will be ignored. Set to a value between 1 and 80 to control the sensitivity of this detection - the lower the value, the higher the sensitivity. In general, the higher the SNR (signal-to-noise) ratio in the working environment, the higher the recommended value of this register. Default is 0x12.
• Register 0xB4 (ASR VAD Start) defines how long a continuous speech should be detected before it is recognized as voice. Set to value between 1 and 80 (10 to 800 milliseconds). Default is 0x0F (150ms).
• Register 0xB5 (ASR VAD Silence End) defines how long a silence period should be detected at the end of a speech segment before the speech is considered to have ended. Set to 20-200 (200-2000 ms). Default is 60 (600 ms).
• Register 0xB6 (ASR VAD Voice Max Length) defines the longest possible duration of a detected speech segment. Set to 5-200 (500ms-20sec). Default is 60 (6 seconds)

After initializing the LD3320 according to the datasheet and tweaking the speech recognition setup registers, I could get the LD3320 to recognize Chinese proper names such as bei jing (北京) and other words like a li ba ba. The quality of the recognition is satisfactory.

MP3 playback

The LD3320 also supports playback of MP3 data received via SPI. Playback is done using the following steps: 

1. Reset and initialize the LD3320 in MP3 mode.
2. Set the correct audio output channel for audio playback. 
3. Send the first segment of the MP3 data to be played.
4. Check if the MP3 has finished playing. If so, stop playback.
5. If not, continue to send more MP3 data and go back to step 4.

Three types of audio output are supported: headphone (stereo), line out (stereo), or speaker (mono). The headphone and speaker channels are always enabled whereas the speaker channel must be enabled independently. Line out and headphone output volume can be adjusted by writing a value to bits 5-1 of registers 0x81 and 0x83 respectively, with 0x00 indicating maximum volume. Speaker output volume can be changed by writing to bits 5-2 of register 0x83, with 0x00 indicating maximum volume.

According to the datasheet, the speaker output line can support an 8-ohm speaker. However, in my tests, connecting an 8-ohm speaker to the speaker output will cause the module to stop playback unexpectedly, presumably due to high power consumption, although the sound quality through the speaker remains clear. The headphone and line out channels seem to be stable and deliver good quality audio.

I also tried to connect a PAM8403 audio amplifier to the line-out channel to achieve a stereo output using two 8-ohm speakers. At first, with the PAM8403 sharing the same power and ground lines with the LD3320, the same issue of unexpected playback termination persisted, even with the usage of decoupling capacitors. Suspecting the issue may be due to disturbance caused by the 8-ohm speaker sharing the same power lines, I used a different power supply for the PAM8403 and the LD3320 managed to play MP3 audio smoothly with no other issues.

Demo video

I made a video showing the module working with a PIC microcontroller and an ST7735 128x160 16-bit color LCD to display the speech recognition results. It shows the results of the module trying to recognize proper names in Chinese (bei jing 北京, shang hai 上海, hong kong 香港, chong qing 重庆, tian an men 天安门) and other words such as a li ba ba. A single beep means that the speech is recognized while a double beep indicates unrecognized speech. Although the speech recognition quality highly depends on the input audio, volume level and other environmental conditions, overall the detection sensitivity and selectivity seems satisfactory as can be seen from the video.

The end of the video shows the stereo playback of an MP3 song stored on the SD card - using a PAM8403 amplifier whose output is fed into two 8-ohm speakers. Notwithstanding the background noises presumably due to effects of breadboard stray capacitance at high frequency (22.1184 MHz for this module), MP3 playback quality seems reasonably good and comparable to the VS1053 module.



The entire MPLAB X demo project for this module can be downloaded here.

See also

SYN6288 Chinese Speech Synthesis Module
Interfacing VS1053 audio encoder/decoder module with PIC using SPI 

Read More »

SYN6288 Chinese Speech Synthesis Module

When searching eBay for a text to speech IC equivalent to the TTS256, I came across the SYN6288, a cheap speech synthesis module made by a Chinese company called Beijing Yutone World Technology specializing in embedded voice solutions and decided to give it a try. Although the IC only comes in SSOP28 10.2mm*5.3mm package, the eBay item which I purchased provided a nice breakout board having 2.54mm pin pitch for easy prototyping:

The module operates on 5V, receives commands via a 9600bps, 8 data bit, 1 stop bit, no parity UART connection and provides mono audio output on the BP0 and BN0 pins. There is also a BUSY output pin which turns high when the module is still processing the commands sent to it via UART. A minor inconvenience is that, due to the SYN6288 breakout board physical dimensions, it is impossible to plug it into a single breadboard for testing - you will need to use two.

Because the audio output levels are low, you will need either a crystal earpiece to listen to it or a suitable audio amplifier such as the LM386 to play it on an 8-ohm speaker. In my case, I use the PAM8403, another cheap but great audio amplifier purchased from eBay:

Reading the datasheet

Because the module specifically targets the Chinese market, the datasheet understandably comes in Chinese only with no official English version. Luckily I managed to use this free tool to translate the Chinese PDF datasheet into English, while preserving the file format and layout.

Knowing that the translation tool works based on the Google Translate API, I decided to cross-compare the original Chinese version and the English translation to verify the translation quality. Below is the SYN6288 block diagram in Chinese:

This is the translated block diagram:

Although the translation is still understandable, multiple text blocks seem to overlap each other. This is because the PDF file format allocates each line or block of text at a fixed location and size. When the translated text is longer than the original text, the text block will wrap to the next line, but without knowing the position of the next text block, causing the alignment issue. Also, as the tool feeds the texts to be translated into the Google Translate API block-by-block, not knowing that many of them are parts of sentences which in turn form paragraphs, the translation quality will be reduced significantly as Google Translate is known to work better when a full sentence is provided.

Getting it to work

I found the sample code for this module here,  provided by CooCox, another Chinese company. Although it compiles fine under MPLAB C30, I spent a few frustrating hours figuring why there is no audio output from the SYN6288 using the code. It turns out that the provided sample code does not send the completed frame header and the function SYN_SentHeader() needs to be modified to fix this:

void SYN_SentHeader(uint16_t DataAreaLength) {
    FrameHeader[1] = (DataAreaLength & 0xFF00) >> 8;
    FrameHeader[2] = (DataAreaLength & 0xFF);
    SendUART2(FrameHeader[0]);
    SendUART2(FrameHeader[1]);
    SendUART2(FrameHeader[2]);

    // Added by MD - missing from original code!
    SendUART2(FrameHeader[3]);
    SendUART2(FrameHeader[4]);
}

The module supports 4 different types of Chinese encoding, GB2312, GBP, Big5 or Unicode. Interestingly, this module also supports playing 5 different types of background music together with the speech. The encoding currently in use and the type of background music will need to be specified prior to sending the text to be spoken. The code also needs to monitor the SYN_BUSY output pin and only sends the text when the module is no longer busy processing the input. This is shown in the following code:

void SYN_Say(const uint8_t * Text) {
    uint16_t i;
    uint16_t Length;

    for(Length = 0;;Length++){
        if(Text[Length] == '\0')
        break;
    }

    FrameHeader[3] = 0x01;
    FrameHeader[4] = BackGroundMusic | TextCodec;
    SYN_SentHeader(Length + 3);
    for(i = 0;i < Length;i++){
        SendUART2(Text[i]);
    }
    SendUART2(CheckXOR(Text,Length));

    // wait for processing, otherwise the SYN_BUSY won't turn on immediately and 
    // the following wait is useless.
    delay_ms(20);

    // wait until speaking is completed before exiting the function.
    while (SYN_BUSY);

    // wait a bit more before exiting, otherwise the chip is still busy
    // and may ignore the next command set
    delay_ms(20);
}

With this code, the module is able to speak some Chinese characters. The following is a voice recording of the SYN6288 trying to say 恭喜，万事如意 (Gōngxǐ, wànshì rúyì, Congratulations and good luck) - you can hear the background music if you turn the volume loud enough:




The above recording is done by amplifying the SYN6288 output using PAM8403, feeding the audio into an 8-ohm speaker, and recording the sound using an iPhone. Although we can still tell that the module tries to speak the Chinese text syllable by syllable, the audio quality is quite good and much more natural than the mechanical voice of the TTS256.

The recording when done by feeding the output audio directly into the computer's line-in input will have slightly better quality. The following is the sample audio provided by the manufacturer:




The module also supports some predefined tones to be played together with the Chinese text. This is done by prefixing the text with a predefined string (e.g msga, msgb, msgc, etc.) specifying the tone to be played. The following will show the SYN6288 playing "ding-dong" and counting numbers in Chinese:



Finally the module supports spelling the English alphabet, from A to Z:



It does not support English words. If you try to send English words, it will try to spell them character-by-character. The following shows what the module will say when we send the text "This is an English sentence":



My last test on this module is to view the audio output on an oscilloscope and see the waveforms. The purpose is to see whether the module is using PWM (Pulse Width Modulation) with a simple RC filter, or a more complicated mechanism for audio output. The following video shows the output waveform during playback:


The waveform looks quite smooth, hence I concluded that it does not use PWM, but more likely a simple DAC (Digital to Analog Converter) in order to achieve the desired audio quality.

MPLAB 8.92 and Chinese characters

When testing the SYN6288, I encountered some problems using MPLAB to edit text files with Chinese characters. First, the code page for the current file needs to be changed by right clicking the editor for the file, selecting Properties and opening the Text tab:

There is no Unicode in the selection list - only GBK/GB2312 for simplified Chinese and Big5 for Traditional Chinese. However, when I select any of the Chinese encodings, the MPLAB editor starts to show erratic behaviour - texts are misplaced and the cursor behaves weirdly:

I decided to use the default encoding (ISO 8859-1 Latin I). In this mode, Chinese text will display wrongly once pasted in the editor but the editor will operate normally without any other issues:

To get the Chinese text display properly without any other issues, we will need to use MPLAB X, which is Netbeans-based and supports Unicode natively.

The verdict

My conclusion after testing is that the SYN6288 is a great simple speech synthesis module for the Chinese language. Although there will always be words which are not pronounced correctly due to the complexity of Chinese (the datasheet, on page 21, indicates that the chip can pronounce around 98% of Chinese characters accurately), I feel that the SYN6288 will work well for small Chinese embedded voice applications. It is unfortunate that there is no equivalent module for the English language. The TTS256 with its horrible mechanical voice is long dead, and some companies are now trying to make huge profits by making clones of the TTS256 and selling them at high price, with little or no improvement at all in the speech quality.

Downloads

Original Chinese datasheet
Translated English datasheet
SYNC6288 C30 library

To use the library, you need to first set the configuration (background music, encoding) for the SYN6288 module:

SYN_SetBackGroundMusic(SYNBackGroundMusic1);
SYN_SetTextCodec(SYNTextCodecGBK);

After that, use SYN_Say to send a text string to the module:

SYN_Say((unsigned char *)"恭喜，万事如意");
SYN_Say((unsigned char *)"msga A B C D E F G H I");

Of course prior to calling the SYN6288 functions, you will need to configure the UART module properly to send commands. The code provided above also contains a simple UART library to facilitate this task.

See also

LD3320 Chinese Speech Recognition and MP3 Player Module 
English text to speech on a PIC microcontroller 

Read More »