|
In this monthly column, an industry expert will answer common questions about VoiceXML and related technologies. Readers are encouraged to submit questions about VoiceXML, including development, voice-user interface design, and speech technology in general, or how VoiceXML is being used commercially in the marketplace. If you have a question about VoiceXML, e-mail it to and be sure to read future issues of VoiceXML Review for the answer.
By Matt Oshry
Q: Some dialogs within my voice application use very large or ambiguous grammars, so the recognition can be tricky. In those situations I may need to confirm with the user that my application received the user's intended response from the recognizer. How do I decide when to confirm the selected response with the user?
A: The technique you're referring to is called "confidence-based confirmation", and there are several ways to implement it in VoiceXML. Let's start with a simple dialog that requests a city and state from the user:
<vxml version="2.0"
xmlns="http://www.w3.org/2001/vxml" >
<form>
<field name="where">
<prompt>Say a city and state.</prompt>
<grammar type="application/srgs+xml" mode="voice"
src="citystate.grxml"/>
<noinput>
Sorry. I didn't hear you.
<reprompt/>
</noinput>
<nomatch>
Sorry. I didn't get that.
<reprompt/>
</nomatch>
<filled>
<log>where = <value expr="where"/></log>
</filled>
</field>
</form>
</vxml>
|
If the filled element gets executed, we know that the recognizer was at least as confident as the value of the confidencelevel property. Otherwise a nomatch event is thrown. According to section 6.3.2 of the VoiceXML 2.0 specification, the default confidencelevel is 0.5, but you can tweak the value to be as low as 0.0 or as high as 1.0. Assuming you haven't modified the property, we know the confidence score for the result is somewhere between 0.5 and 1.0. We can determine the exact value by checking either of the following:
where$.confidence
application.lastresult$.confidence
Using the confidence score from lastresult$ can get tricky if your grammar fills multiple slots, and 2.3.1 of the VoiceXML 2.0 specification states that the "distinction between field and utterance level confidence is platform dependent", so we'll utilize the confidence shadow variable of the field for maximum portability across Voice Browser implementations.
Now that we know how to access the confidence score, we need to pick a threshold. When the confidence score exceeds the threshold, we'll assume the recognizer is correct; otherwise, we'll confirm the recognizer's selection with the user. For this example I'll choose 0.75, but you should consult your resident speech expert, since the threshold will vary depending on the grammar you're using.
Here's an implementation of a confidence-based confirmation dialog that leverages the Form Interpretation Algorithm (FIA) by embedding the confirmation dialog in the same form as the field that collects the city and state. The cond attribute of the confirm field controls whether or not the interpreter executes it. If the cond attribute evaluates to false, FIA skips the confirm field and selects and executes the block. if the cond attribute evaluates to true, FIA selects the confirm field. If the user says 'no' in response to the confirm field, execution of the clear resets the guard condition on the where and confirm fields making them eligible for selection again during the next iteration of the FIA's main loop. In fact, the block will not be executed until either
1) the form item variable where is filled, and the associated confidence score is greater than or equal to 0.75, or
2) the user says 'yes' in response to the confirm field's prompt.
<vxml version="2.0"
xmlns="http://www.w3.org/2001/vxml" >
<catch event="noinput nomatch">
Sorry. I didn't get that.
<reprompt/>
</catch>
<form>
<field name="where">
<prompt>Say a city and state.</prompt>
<grammar type="application/srgs+xml" mode="voice"
src="citystate.grxml"/>
</field>
<field name="confirm"
cond="typeof(where) != 'undefined' && where$.confidence < 0.75">
<prompt>
I heard you say <value expr="where"/>, is that correct?
</prompt>
<grammar type="application/srgs+xml" src="yesno.grxml"/>
<filled>
<if cond="confirm == 'no'">
<clear namelist="confirm where"/>
</if>
</filled>
</field>
<block>
<submit next="listing.cgi" namelist="where"/>
</block>
</form>
</vxml>
|
What if the recognizer never obtains a confidence score greater than or equal to 0.75, and the user repeatedly responds 'no' to the confirm dialog? After the second and certainly the third confirmation attempt, the user is likely to give up. You can preempt this situation by keeping track of the number of times the user is asked to confirm her choice and take an appropriate action, for example, transferring the user to your call center. If you don't have that luxury, you can attempt to obtain the information from the user in a different way - for example, via DTMF ("type the first few letters..."), or by presenting a list of choices.
Here's a slightly modified version of the previous example that keeps track of confirmation attempts using a variable named confirmCount. Because the confirmCount variable is declared at form scope, it automatically gets initialized each time you enter the form. When the confirmCount reaches 2 in the confirm dialog, the application throws the event "com.yourcompany.yourapp.transfertoagent". The handler for this event presumably navigates to a form containing a transfer element that whisks the user off to a customer care representative.
<vxml version="2.0"
xmlns="http://www.w3.org/2001/vxml" >
<catch event="noinput nomatch">
Sorry. I didn't get that.
<reprompt/>
</catch>
<form>
<var name="confirmCount" expr="0"/>
<field name="where">
<prompt>Say a city and state.</prompt>
<grammar type="application/srgs+xml" mode="voice"
src="citystate.grxml"/>
</field>
<field name="confirm"
cond="typeof(where) != 'undefined' && where$.confidence < 0.75">
<prompt>
I heard you say <value expr="where"/>, is that correct?
</prompt>
<grammar type="application/srgs+xml" src="yesno.grxml"/>
<filled>
<if cond="++confirmCount < 2">
<clear namelist="confirm where"/>
<else/>
<throw event="com.yourcompany.yourapp.transfertoagent"/>
</if>
</filled>
</field>
<block>
<submit next="listing.cgi" namelist="where"/>
</block>
</form>
</vxml>
|
You can reuse the confirm field anywhere you require confidence-based confirmation
by doing the following:
- Copy and paste the confirm field into the form after the field that requires confidence-based confirmation.
-
Replace the use of the variable 'where' with the value of the name attribute of the data collection field.
The data collection field is referenced in three places:
- The cond attribute of the confirm field
- The prompt
- The namelist attribute of the clear element
- Adjust the confidence threshhold to an appropriate value.
- If you track the number of confirmation attempts, be sure to declare the counter within the form
into which you copy the confirmation field.
Q: According to the latest draft of VoiceXML 2.1, the markname and marktime variables store values corresponding to the mark that was last executed "before barge-in occurred or the end of audio playback occurred." If the user listens to the prompt in its entirety, how can my application accurately detect the user's reaction time if they speak during the timeout interval?
A: Your careful reading of the specification is correct. If the user doesn't barge-in on the prompt, the marktime property will only reflect the interval between the last mark that was executed and the end of audio playback. It will not include the timeout interval. You can extend the design of this feature setting the timeout property to zero seconds and by adding to the end of the prompt queue a silent audio file (e.g. timeout.wav) the duration of which is equivalent to the desired timeout. Here's some sample code:
<field name="city">
<property name="timeout" value="0s"/>
<grammar mode="voice" src="citystate.srgs"/>
<prompt>
<mark name="pre"/>
<audio src="citystate.wav">say a city and state</audio>
<mark name="timeout"/>
<audio src="timeout.wav"/>
</prompt>
<filled>
<log>
markname=<value expr="city$.markname"/>,
marktime=<value expr="city$.marktime"/>
</log>
Okay. <value expr="city"/>
</filled>
</field>
|
|
