Documente Academic
Documente Profesional
Documente Cultură
GeneralMagic Demo
access to the Web Will allow any telephone to be used to access appropriately designed Webbased services Server-based Voice portals
2
via key pads, spoken commands, listening to prerecorded speech, synthetic speech and music. An advantage to people with visual impairment Web access while keeping hands & eyes free for other things (eg. Driving).
3
Motivation
Far
more people today have access to a telephone than have access to a computer with an Internet connection. Many of us have already or soon will have a mobile phone within reach wherever we go.
Motivation
Easy
to use - for people with no knowledge or fear of computers. Voice interaction can escape the physical limitations on keypads and displays as mobile devices become ever smaller.
Motivation
Many
companies to offer services over the phone via menus traversed using the phone's keypad. Voice Browsers are the next generation of call centers, which will become Voice Web portals to the company's services and related websites, whether accessed via the telephone network or via the Internet.
7
Motivation
Disadvantages
browsing is more passive due to the persistence of the visual informationleading role is The Voice browsing is moreto turned over active since the user has to USER issue commands. the Graphical Browsers are client-based, whereas Voice Browsers are serverbased.
Possible Applications
Accessing
business information:
The corporate "front desk" which asks callers who or what they want Automated telephone ordering services Support desks Order tracking Airline arrival and departure information Cinema and theater booking services Home banking services
10
public information:
Community information such as weather, traffic conditions, school closures, directions and events Local, national and international news National and international stock market information Business and e-commerce transactions
11
personal information:
Voice mail Calendars, address and telephone lists Personal horoscope Personal newsletter To-do lists, shopping lists, and calorie counters
12
13
The
World Wide Web Consortium (W3C) develops interoperable technologies (specifications, guidelines, software, and tools) to lead the Web to its full potential as a forum for information, commerce, communication, and collective understanding.
14
VoiceXML
VoiceXML
is a dialog markup language designed for telephony applications, where users are restricted to voice and DTMF (touch tone) input.
Browser
text.html
Web Server
text.vxml
Internet
Speech Synthesis
The
specification defines a markup language for prompting users via a combination of prerecorded speech, synthetic speech and music. You can select voice characteristics (name, gender and age) and the speed, volume, pitch, and emphasis. There is also provision for overriding the synthesis engine's default pronunciation.
17
Speech Recognition
Speech Grammars Speech
USER
Touch Tone
Semantic Interpretation
DTMF Grammars
DTMF Grammars
Touch
tone input is often used as an alternative to speech recognition. Especially useful in noisy conditions or when the social context makes it awkward to speak. The W3C DTMF grammar format allows authors to specify the expected sequence of digits, and to bind them to 19 the appropriate results
Speech Grammars
In most cases, user prompts are very carefully designed to encourage the user to answer in a form that matches context free grammar rules. Speech Grammars allow authors to specify rules covering the sequences of words that users are expected to say in particular contexts. These contexual clues allow the recognition engine to focus on likely utterances, improving the chances of a correct match. 20
Semantic Interpretation
The
recognition process matches an utterance to a speech grammar, building a parse tree as a byproduct. There are two approaches to harvesting semantic results from the parse tree: 1. Annotating grammar rules with semantic interpretation tags (ECMAScript). 2. Representing the result in XML.
22
"I would like a medium coca cola and a large pizza with pepperoni and mushrooms.
could be converted to the following semantic result
{ drink: { beverage: "coke drinksize: "medium } pizza: { pizzasize: "large" topping: [ "pepperoni", "mushrooms" ] }
23
Pronunciation Lexicon
Application
developers sometimes need to ability to tune speech engines, whether for synthesis or recognition. W3C is developing a markup language for an open portable specification of pronunciation information using a standard phonetic alphabet. The most commonly needed pronunciations are for proper nouns such as surnames or 24 business names.
Call Control
Fine-grained control of speech (signal processing) resources and telephony resources in a VoiceXML telephony platform. Will enable application developers to use markup to perform call screening, whisper call waiting, call transfer, and more. Can be used to transfer a user from one voice browser to another on a competely different machine.
25
Mechanisms to transfer application state, such as a session identifier, along with the user's audio connections. The user could start with a visual interaction on a cell phone and follow a link to switch to a VoiceXML application. The ability to transfer a session identifier makes it possible for the Voice Browser application to pick up user preferences and other data entered into the visual application. 26
the user could transfer from a VoiceXML application to a customer service agent. The agent needs the ability to use their console to view information about the customer, as collected during the preceding VoiceXML application. The ability to transfer a session identifier can be used to retrieve this information from the customer database.
27
extensions are proposed to HTML 4.0 and CSS2 to support voice browsing content is likely to include music and different speakers. These effects can be reproduced to some extent via the aural style sheets features in CSS2.
28
Prerecorded
Volume
Rate Pitch Direction
Authors want control over how the document is rendered. Aural style sheets (part of CSS2) provide a basis for controlling a range of features:
Spelling out text letter by letter Speech fonts (male/female, adult/child etc.) Inserted text before and after element content Sound effects and music
29
do I connect? Do I speak to the browser or does the browser speak to me? What is seen on the screen? How do I enter input?
30
Problems
How
does the browser understand what I say? How can I tell it what I want?
what
if it doesnt understand?
31
Synthesis
Text to Speech
Speech
Recognition
Interpretation
32
machine voice by arranging phonemes (k, ch, sh, etc.) into words. There are several algorithms for performing Speech Synthesis. The choice depends on the task they're used for.
33
easiest way is to just record the voice of a person speaking the desired phrases.
This is useful if only a restricted volume of phrases and sentences is used, e.g. schedule information of incoming flights. The quality depends on the way recording is done.
34
More
Text To Speech
Text-To-Speech algorithms split the speech into smaller pieces. The smaller the units, the less they are in number, but the quality also decreases. An often used unit is the phoneme, the smallest linguistic unit. Depending on the language used, there are about 35-50 phonemes in western European languages, i.e. we need only 35-50 single recordings.
36
Text To Speech
The
problem is, combining them as fluent speech requires fluent transitions between the elements. The intelligibility is therefore lower, but the memory required is small. A solution is using diphones. Instead of splitting at the transitions, the cut is done at the center of the phonemes, leaving 37 the transitions themselves intact.
Text To Speech
This
means there are now approximately 1600 recordings needed (40*40). The longer the units become, the more elements there are, but the quality increases along with the memory required.
38
Text To Speech
Other
units which are widely used are half-syllables, syllables, words, or combinations of them, e.g. word stems and inflectional endings. TTS is dictionary-driven. The larger the dictionary resident in the browser is, the better the quality. For unknown words, falls back on rules 39 for regular pronunciation.
Text To Speech
Vocabulary
But
is unlimited!!!
depends on the context in which a word occurs. Limited linguistic analysis is needed.
40
Text To Speech
Another example:
I have read the first chapter. I will read some more after lunch.
For
these cases, and in the cases of irregular words and name pronunciation, authors need a way to provide supplementary TTS information and to indicate when it applies.
41
Text To Speech
But
specialized representations for phonemic and prosodic information can be off putting for non-specialist users. this reason it is common to see simplified ways to write down pronunciation, for instance, the word "station" can be defined as:
For
station: stay-shun
42
Text To Speech
This approach encourages users to add pronunciation information, leading to an increase in the quality of spoken documents, compared to more complex and harder to learn approaches. This is where W3C comes in: Providing a specification to enable consistent control (generating, authoring, processing) of voice output by speech synthesizers for varying speech content, for use in voice 43 browsing and in other contexts.
Recognition
Interpretation
44
Speech Recognition
45
Speech Recognition
46
Speech Recognition
47
Speech Recognition
48
Speech Recognition
Automatic
speech recognition is the process by which a computer maps an acoustic speech signal to text. Speech is first digitized and then matched against a dictionary of coded waveforms. The matches are converted into text.
49
Speech Recognition
Types of voice recognition applications: Command systems recognize a few hundred words and eliminate using the mouse or keyboard for repetitive commands. Discrete voice recognition systems are used for dictation, but require a pause between each word. Continuous voice recognition understands natural speech without pauses and is the 50 most process intensive.
Speech Recognition
A
speaker dependent system is developed to operate for a single speaker. These systems are usually easier to develop, cheaper to buy and more accurate, but not as flexible as speaker adaptive or speaker independent systems.
51
Speech Recognition
A
speaker independent system is developed to operate for any speaker of a particular type (e.g. American English). These systems are the most difficult to develop, most expensive and accuracy is lower than speaker dependent systems. However, they are more 52 flexible.
Speech Recognition
A
speaker adaptive system is developed to adapt its operation to the characteristics of new speakers. It's difficulty lies somewhere between speaker independent and speaker dependent systems.
53
Speech Recognition
Speech
recognition technologies today are highly advanced. There is a huge gap between the ability to recognize speech and the ability to interpret speech.
54
2. Spectral representation and analysis 3. Segmentation. 4. Phonetic Modeling 5. Search and Match
55
HMM-based systems are currently the most commonly used and most 56 successful approach.
Speech Grammars
The
grammar allows a speech application to indicate to a recognizer what it should listen for, specifically: Words that may be spoken,
Speech Grammars
In
simple speech recognition/speech understanding systems, the expected input sentences are often modeled by a strict grammar (such as a CFG). this case, the user is only allowed to utter those sentences, that are explicitly covered by the grammar.
Good for menus, form filling, ordering services, etc.
58
In
Speech Grammars
Experience
shows that a context free grammar with reasonable complexity can never foresee all the different sentence patterns, users come up with in spontaneous speech input. This approach is therefore not sufficient for robust speech recognition/ understanding tasks or free text input 59 applications such as dictation.
For Example
Possible
answers to a question may be "Yes" or "No, but it could also be any other word used for negative or positive response. It could be "Ya," "you betch'ya," "sure," "of course" and many other expressions. It is necessary to feed the speech recognition engine with likely utterances representing the desired response. 60
Speech Grammars
What
is done?
61
63
64
= {aij} where aij = P(Xt+1 = qj |Xt = qi) (transition probabilities) B = {bi(k)} where bi(k) = P(Ot = vk | Xt = qi) (observation probabilities) t = P(X0 = qi) (initial state distribution)
65
Hidden Markov Model (HMM) is a five-tuple (Q,V,A,B,). = {A,B,} denote the parameters for a given HMM with fixed Q and V.
Let
66
Language Models
A
68
The Equation
to compute P(W)
Step
1: Decompose probability P(And nothing but the truth) = P(And) P(nothing | and) P(but | and nothing) P(the | and nothing but) P(truth | and nothing but the)
70
each word depends only on the previous two words (three words total tri means three, gram means writing)
P(the| whole truth and nothing but) P(the|nothing but) P(truth| whole truth and nothing but the) P(truth|but the)
71
72
Back to Trigrams
How
do we find the probabilities? Get real text, and start counting! P(the | nothing but) Count(nothing but the) Count(nothing but)
73
N-grams
Why
stop at 3-grams? If P(z|rstuvwxy) P(z|xy) is good, then P(z|rstuvwxy) P(z|vwxy) is better! 4-gram, 5-gram start to become expensive...
74
language models are used in large vocabulary speech recognition systems to provide the recognizer with an a-priori likelihood P(W) of a given word sequence W. N-Gram language model is usually derived from large training texts that share the same language 75 characteristics as expected input.
The
Using an N-Gram model in the recognizer and a CFG in a (separate) understanding component Integrating special N-Gram rules at various levels in a CFG to allow for flexible input in specific context using a CFG to model the structure of phrases (e.g. numeric expressions) that incorporated in a higher-level N-Gram model 76 (class N-Grams)
Speech Recognition
Speech Grammars Stochastic n-gram models
Semantic
Interpretation
77
Semantic Interpretation
Problems
What
does the user mean? We have the right keywords, but the phrase is meaningless or unclear.
78
Semantic Interpretation
As
stated before, the technologies of speech recognition exceed those of interpretation. Most interpreters are base on key words.
Sometimes this is not good enough!
79
VoiceXML - Example 1
<?xml version="1.0"?> <vxml version="2.0"> <form> <block>Hello World!</block> </form> </vxml>
The top-level element is <vxml>, which is mainly a container for dialogs. There are two types of dialogs: forms and menus. Forms present information and gather input; menus offer choices of what to do next.
81
VoiceXML - Example 1
<?xml version="1.0"?> <vxml version="2.0"> <form> <block>Hello World!</block> </form> </vxml>
This example has a single form, which contains a block that synthesizes and presents "Hello World!" to the user. Since the form does not specify a successor dialog, the conversation ends.
82
VoiceXML - Example 2
Our second example asks the user for a choice A field is an input field. of drink and thenThe user must provide a value for submits it to a server script:
<?xml version="1.0"?> field before proceeding to the the <vxml version="2.0"> next element in the form. <form> <field name="drink"> <prompt>Would you like coffee,tea, milk, or nothing?</prompt> <grammar src="drink.grxml" type="application/grammar+xml"/> </field> <block> <submit next="http://www.drink.example.com/drink2.asp"/> </block> </form> 83 </vxml>
VoiceXML - Example 2
A sample interaction is: C (computer): Would you like coffee, tea, milk, o nothing? H (human): Orange juice. C: I did not understand what you said. (a platform specific default message.) C: Would you like coffee, tea, milk, or nothing? H: Tea C: (continues in document drink2.asp)
84
The implementation platform generates events in response to user actions (e.g. spoken or character input received, disconnect) and system events (e.g. timer expiration).
Scope of VoiceXML
Output
The language providesfiles. for collecting Output of audio means character and/or spoken input, assigning the Recognition of spoken input. input to document-defined request variables, Recognition of DTMF input. and making decisions that affect the interpretation of Recording of spoken input. documents written in the language. A document may be linked to other documents Control of dialog flow. through Universal Resource Identifiers (URIs).
Telephony
VoiceXML
Voice
XML is intended to be analogous to graphical surfing. There are limitations. Excellent for menu applications. Awkward for open dialog applications There are other languages: VoXML, omniviewXML
87
Navigation
The
user might be able to speak the word "follow" when she hears a hypertext link she wishes to follow. user could also interrupt the browser to request a short list of the relevant links.
The
88
Navigation example
User: links? Browser: The links are: 1 company info 2 latest news 3 placing an order 4 search for product details Please say the number now User: 2 Browser: Retrieving latest news...
89
command could be used to request a list of the document's headings. This would allow users to browse an outline form of the document as a means to get to the section that interests them.
90
Browsers allow entering a wanted URL in the browser window is this supported in Voice Browsers?
Think:
91
voice browsers are meant to replace human operator dialog, they must be fast in response. Speech Recognition / Interpretation / Synthesis depend on implementation When a user requests a certain document, several related documents can be downloaded for easier access.
92
be? Friendly is sometimes time consuming. What percentage of the time does the user talk and what percentage of the time is he listening? What parameters can I control?
93
94
Inserted text
When a hypertext link is spoken by a speech synthesizer, the author may wish to insert text before and after the link's caption, to guide the user's response.
For example:
Inserted text
The
words "For and "Press 1" were added to the text embedded in the anchor element.
first glance it looks as if this 'wrapper' text should be left for the voice browser to generate, but on further examination you can easily find problems with this approach.
96
On
Inserted text
For
97
Inserted text
The CSS2 draft specification includes the means to provide "generated text" before and after element content.
style='cue-before: "To";
Users might easily enter unexpected or ambiguous input, or just pause, providing no input at all.
Some examples to errors which might generate events:
When presented with a numbered list of links, the user enters a number that is outside the range presented . The phrase uttered by the user matches more than one template rule.
99
Authors will have control over the browser response to selection errors and timeouts. Other errors might be dealt with by the browser or platform.
100
assistant demo Bank service demo (cough, ambiguity) Financial Center Demo (ums) Telectronics Demo
101
Caf General Magic HeyAnita's FreeSpeech Developer Network IBM Voice Server SDK Beta Program based on VoiceXML Version 1.0 Motorolas Mobile Application Development Toolkit (MADK)
102
Developer Network Open VXI VoiceXML interpreter PIPEBEACHs speechWeb Teleras DeVXchange Tellme Studio VoiceGenie
103