Showing posts with label IME. Show all posts
Showing posts with label IME. Show all posts

Monday, 12 December 2011

Add Voice Typing To Your IME

[This post is by Luca Zanolin, an Android engineer who works on voice typing. — Tim Bray]



A new feature available in Android 4.0 is voice typing: the difference for users is that the recognition results appear in the text box while they are still speaking. If you are an IME developer, you can easily integrate with voice typing.

To simplify the integration, if you download this library and modify your IME as described below, everything will work smoothly on any device with Android 2.2 or later. On 4.0+, users will get voice typing, and earlier versions will use standard voice recognition; the difference is illustrated below.

To see how to integrate voice typing you can take a look at this sample IME. The IME is really simple and contains only one button: a microphone. By pressing the microphone, the user triggers voice recognition.

Here are the steps that you need to follow to integrate voice recognition into your IME.

Download the library

Download this library and add it to your IME APK.

Create the voice recognition trigger

The library contains the VoiceRecognitionTrigger helper class. Create an instance of it inside the InputMethodService#onCreate method in your IME.

public void onCreate() {
super.onCreate();
...
mVoiceRecognitionTrigger = new VoiceRecognitionTrigger(this);
}

Add the microphone icon to your IME

You need to modify the UI of your IME, add a microphone icon, and register an OnClickListener to trigger voice recognition. You can find the assets inside the sample IME. The microphone icon should be displayed only if voice recognition is installed; use VoiceRecognitionTrigger#isInstalled().

public View onCreateInputView() {
LayoutInflater inflater = (LayoutInflater) getSystemService(
Service.LAYOUT_INFLATER_SERVICE);
mView = inflater.inflate(R.layout.ime, null);
...
mButton = (ImageButton) mView.findViewById(R.id.mic_button);
if (mVoiceRecognitionTrigger.isInstalled()) {
mButton.setOnClickListener(new OnClickListener() {
@Override
public void onClick(View v) {
mVoiceRecognitionTrigger.startVoiceRecognition();
}
});
mButton.setVisibility(View.VISIBLE);
} else {
mButton.setVisibility(View.GONE);
}
return mView;
}

If your IME supports multiple languages, you can specify in which language recognition should be done as a parameter of startVoiceRecognition().

Notify the trigger when your IME starts

When your IME starts, you need to notify the trigger, so it can insert into the text view any pending recognition results.

@Override
public void onStartInputView(EditorInfo info, boolean restarting) {
super.onStartInputView(info, restarting);
if (mVoiceRecognitionTrigger != null) {
mVoiceRecognitionTrigger.onStartInputView();
}
}

Modify your AndroidManifest

In order to start a voice recognition through the Intent API, the library uses a service and an activity, and you need to add them into your manifest.

<manifest ... >
<application ...>
...
<service android:name="com.google.android.voiceime.ServiceHelper" />
<activity
android:name="com.google.android.voiceime.ActivityHelper"
android:theme="@android:style/Theme.Translucent.NoTitleBar"
android:excludeFromRecents="true"
android:windowSoftInputMode="stateAlwaysHidden"
android:finishOnTaskLaunch="true"
android:configChanges="keyboard|keyboardHidden|navigation
|orientation"/>
</application>
</manifest>

Update the microphone icon dynamically (optional)

This step is optional, but you should implement it if possible as it will improve the user experience. Voice recognition requires network access, and if there is no network, your IME should notify the user that voice recognition is currently disabled. To achieve this, you need to register the VoiceRecognitionTrigger.Listener and enable/disable the microphone accordingly.

The listener is registered in InputMethodService#onCreate, and you have to unregister it in InputMethodService#onDestroy, otherwise you will leak the listener.

@Override
public void onCreate() {
super.onCreate();
...
mVoiceRecognitionTrigger = new VoiceRecognitionTrigger(this);
mVoiceRecognitionTrigger.register(new VoiceRecognitionTrigger.Listener() {
@Override
public void onVoiceImeEnabledStatusChange() {
updateVoiceImeStatus();
}
});
}

...
@Override
public void onDestroy() {
...
if (mVoiceRecognitionTrigger != null) {
mVoiceRecognitionTrigger.unregister(this);
}
super.onDestroy();
}

private void updateVoiceImeStatus() {
if (mVoiceRecognitionTrigger.isInstalled()) {
mButton.setVisibility(View.VISIBLE);
if (mVoiceRecognitionTrigger.isEnabled()) {
mButton.setEnabled(true);
} else {
mButton.setEnabled(false);
}
} else {
mButton.setVisibility(View.GONE);
}
mView.invalidate();
}

And add this permission into your manifest:

<manifest ... >
...
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
...
</manifest>

That’s all there is to it

Voice recognition makes it easy for users to do more with their Android devices, so we appreciate your support in adding it to your IMEs.

Friday, 5 March 2010

Speech Input API for Android

People love their mobile phones because they can stay in touch wherever they are. That means not just talking, but e-mailing, texting, microblogging, and so on. So, in addition to search by voice and voice shortcuts like "Navigate to", we included a voice-enabled keyboard in Android 2.1, which makes it even easier to stay connected. Now you can dictate your message instead of typing it. Just tap the new microphone button on the keyboard, and you can speak just about anywhere you would normally type.



We believe speech can fundamentally change the mobile experience. We would like to invite every Android application developer to consider integrating speech input capabilities via the Android SDK.
One of my favorite apps in the Market that integrates speech input is Handcent SMS, because you can dictate a reply to any SMS with a quick tap on the SMS popup window.

Speech input integrated into Handcent SMS

The Android SDK makes it easy to integrate speech input directly into your own application—just copy and paste from this sample application to get started. Android is an open platform, so your application can potentially make use of any speech recognition service on the device that's registered to receive a RecognizerIntent. Google's Voice Search application, which is pre-installed on many Android devices, responds to a RecognizerIntent by displaying the "Speak now" dialog and streaming audio to Google's servers—the same servers used when a user taps the microphone button on the search widget or the voice-enabled keyboard. (You can check if Voice Search is installed in Settings ➝ Applications ➝ Manage applications.)

One important tip: for speech input to be as accurate as possible, it's helpful to have an idea of what words are likely to be spoken. While a message like "Mom, I'm writing you this message with my voice!" might be appropriate for an email or SMS message, you're probably more likely to say something like "weather in Mountain View" if you're using Google Search. You can make sure your users have the best experience possible by requesting the appropriate language model: "free_form" for dictation, or "web_search" for shorter, search-like phrases. We developed the "free form" model to improve dictation accuracy for the voice keyboard on the Nexus One, while the "web search" model is used when users want to search by voice.

Google's servers currently support English, Mandarin Chinese, and Japanese. The web search model is available in all three languages, while free-form has primarily been optimized for English. As we work hard to support more models in more languages, and to improve the accuracy of the speech recognition technology we use in our products, Android developers who integrate speech capabilities directly into their applications can reap the benefits as well.