Word Level Highlighting

Word level highlighting

The main goal of this project is to provide an android app for “Word-Level Highlighting Synchronized with TTS” using open source TTS engine named “flyte”.This helps dyslexic users to read and study text documents in more understandable format.

This TTS application provides access to word level synchronization that is implemented in underlying c code.

Implementation details

Analyzing duration information during synthesis of text.

Duration information is analyzed using a new datastructure called Utterance

Utterance:

class Utterance
{
public:
  String getText();
  String getID();
  virtual ~Utterance();
  vector<float> word_durations;
  vector<int> word_indices;
  vector<float> seg_durations;
  vector<String> seg_labels;
  int cur_index;
  int cur_seg_index;
private:
  String id_;
  String text_;
};
}

All the duration information needed to calculate the event of word level callback are stored in the instance of Utterance.

All the computation of duration is being done in an androidtts callback function NativeFliteCallback.

Making it usable to application by a callback function.

As we are adding new functionalities to the existing tts api. We need to bundle whole application to access the functionality. We have designed a new package "edu.cmu.cs.speech.tts.flite.NativeFliteTTS;" Accessing new functions:

Interface to assign a callback function:

edu.cmu.cs.speech.tts.flite.NativeFliteTTS.OnWordCompletedListener

• Functions to be implemented in the interface:

Calls at the end of each word and also returns the current index as argument startPosition.

public void onWordCompleted(int startPosition)

Calls when synthesis of whole text is being successfully completed.

public void onDone()

Added public method:

setOnWordCompletedListener(OnWordCompletedListener)

• Sets the listener that will be notified of various events related to the synthesis of a given utterance.

• Parameters listener the listener to use.

• Returns ERROR or SUCCESS