I'm doing a paper for VoIP services and currently researching disability regulation, and it got me thinking.
Is there currently any method, plugin, way, for someone who is deaf or mute to use Skype (or any other computer based VoIP product?) I got to thinking that it would be easy to write a program that would translate incoming voice as text, and the deaf/mute person could then type their response which the program would translate into computer speak. It'd be a chat for one person and a normal phone conversation for the other.
I know the technology is possible, I'm wondering what's out there. Such a program would 1) eliminate the need for TTY type systems, 2) allow those with disabilities to make calls anywhere and everywhere without using special TTY enabled numbers, worrying about finding TTY phones when traveling (imagine a deaf person in an airport with a PocketPC, foldout keyboard, and a wi-fi network calling their loved one to let them know they're OK), and 3) eliminate the need for the special TRS facilities and such to process conventional TTY calls.
I'd be interested in such a system, too - imagine making quiet phone calls during a meeting, class, etc.
Thoughts? Anyone know of a system or something close, or someone who's currently working on something similar?
Full Version: Skype interface for deaf/mute people?
Apologises greigmg as my commnets may be a massive over simplification of the situation but isn't a TTY phone service the ability to convert text to voice and voice to text? That being the case, would the use of the Skype IM service not be more appropriate than the voice service across Skype?
Skype provides visual notifications to show when a 'call' is requested - much like the flashing bulb on a TTY telephone - so the solution is potentially someway towards meeting the needs of deaf or hh users.
I realise that this would not assist the SkypeIn/SkypeOut communication and the work we are involved with in incorporating speech to text and text to speech (our product WiSPA is one example) into Skype communications may provide some opportunities to address this specific requirement. Would be interested to understand more on what a true TTY solution should deliver separate from where Skype is today.
Skype provides visual notifications to show when a 'call' is requested - much like the flashing bulb on a TTY telephone - so the solution is potentially someway towards meeting the needs of deaf or hh users.
I realise that this would not assist the SkypeIn/SkypeOut communication and the work we are involved with in incorporating speech to text and text to speech (our product WiSPA is one example) into Skype communications may provide some opportunities to address this specific requirement. Would be interested to understand more on what a true TTY solution should deliver separate from where Skype is today.
No, you're exactly right - the problem being that not everyone I want to call uses any sort of chat system (e.g. my car's in the shop - how do I explan to the mechanic that I need him to install AOL IM on his business computer so I can get updates on the job?).
As I understand it, those disabled currently have two options - 1) using a TTY phone, which is essentially a rudimentary chat device (keyboard and one or two line display) they call a TTY number, and chat with the person on the other end. Many large businesses and govt agencies you'll see have an option for a TTY (sometimes labeled TDD) number. Or 2) again using a TTY, they can call conventional numbers, and the conversation in essence goes through a translator, called a "communications assistant," that basically has a TTY and a regular phone in front of them, and as the disabled person types, they read the message to the other person, and as the other person talks, they type the message to the disabled individual.
I am not exactly sure how option #2 works, thought I do know the FCC mandates similar pricing, 24 hour availability, etc. for such relay systems.
I envision a plugin or service for Skype or other VoIP systems that would in essence cut out the middle man and eliminate the need for special systems like this. All the processing would occur on the disabled person's end - as speech comes into their computer/pda/etc. it gets translated into text, and as the user types it gets translated into speec which is then sent over the VoIP service.
As I understand it, those disabled currently have two options - 1) using a TTY phone, which is essentially a rudimentary chat device (keyboard and one or two line display) they call a TTY number, and chat with the person on the other end. Many large businesses and govt agencies you'll see have an option for a TTY (sometimes labeled TDD) number. Or 2) again using a TTY, they can call conventional numbers, and the conversation in essence goes through a translator, called a "communications assistant," that basically has a TTY and a regular phone in front of them, and as the disabled person types, they read the message to the other person, and as the other person talks, they type the message to the disabled individual.
I am not exactly sure how option #2 works, thought I do know the FCC mandates similar pricing, 24 hour availability, etc. for such relay systems.
I envision a plugin or service for Skype or other VoIP systems that would in essence cut out the middle man and eliminate the need for special systems like this. All the processing would occur on the disabled person's end - as speech comes into their computer/pda/etc. it gets translated into text, and as the user types it gets translated into speec which is then sent over the VoIP service.
Thanks for that clarification.
We have already developed the technology linked to Skype that could provide a solution for a mute user. This would deliver an environment where a mute user would answer their Skype telephone and listen to the caller in the normal manner but type their side of the conversation. The typed text would then be spoken over the VoIP connection as they typed for the other party to hear on their telephone.
The bigger issue is where the Skype user is deaf. This would require the spoken conversation from the telephone user to be converted to text so that it can be displayed for the deaf party to read. Although speech recognition has made significant advances in recent years, I do not believe it is yet able to handle the variety of accents, the quality variances from telephone systems as well handling an open dictionary with no context.
Most successful speech recognition solutions either have an awareness of the words they are to receive or have been trained to a specific voice.
Solutions in the first category such as speech self service system or call centre front ends etc have a context which allows the designer of the solution to predetermine the range of words that are likely to be spoken. This allows the recognition engine to make intelligent guesses of what is being spoken from a restricted vocabulary.
To take free format speech, as with a typical conversation, would require the speech recognition system to be trained to a specific voice before it was reliable. This is obviously impracticable for a solution that is to accept telephone calls from anyone.
At the speed of advances in this space I am sure that a solution will be possible and I welcome the thoughts and experiences of others as extending Skype in this manner could deliver significant value.
We have already developed the technology linked to Skype that could provide a solution for a mute user. This would deliver an environment where a mute user would answer their Skype telephone and listen to the caller in the normal manner but type their side of the conversation. The typed text would then be spoken over the VoIP connection as they typed for the other party to hear on their telephone.
The bigger issue is where the Skype user is deaf. This would require the spoken conversation from the telephone user to be converted to text so that it can be displayed for the deaf party to read. Although speech recognition has made significant advances in recent years, I do not believe it is yet able to handle the variety of accents, the quality variances from telephone systems as well handling an open dictionary with no context.
Most successful speech recognition solutions either have an awareness of the words they are to receive or have been trained to a specific voice.
Solutions in the first category such as speech self service system or call centre front ends etc have a context which allows the designer of the solution to predetermine the range of words that are likely to be spoken. This allows the recognition engine to make intelligent guesses of what is being spoken from a restricted vocabulary.
To take free format speech, as with a typical conversation, would require the speech recognition system to be trained to a specific voice before it was reliable. This is obviously impracticable for a solution that is to accept telephone calls from anyone.
At the speed of advances in this space I am sure that a solution will be possible and I welcome the thoughts and experiences of others as extending Skype in this manner could deliver significant value.
You're right about the need to "train" a voice rec program before it works effectively. I think a forward approach would be a combination of voice recognition and sentence prediction - Microsoft's handwriting recognition on Windows XP Tablet does something like this, and works very well, it combines standard handwriting recognition with a dictionary so it can make "educated guesses". However, I don't think any software currently available could even come close to recognising voices over phone calls.
Right, the speech-to-text end seems to be the rub. Bill Gates says within 2-5 years it'll be at that point.
Working with Speech Recognition systems over the last four years and seeing the advances made in that time I would say that's a fair assessment.
This raises the question - is there sufficient benefit to deliver a solution for users with speech difficulties or is the real problem with those who are deaf or hard of hearing?
Huh? What exactly do you mean?
Sorry for the confusion. In my earlier post I mentioned that a solution for users that are mute or have speech difficulties is technically feasible today. It is the technology to support the deaf or hard of hearing that was the current difficulty.
My question therefore related to the fact that I do not know whether there are already acceptable solutions readily available for the mute.
If there is value for offering such a solution for the mute accepting that a comparible solution for the deaf is further away then I would seriously consider putting some of our resources into offering such an offering from the building blocks that we already have available.
My question therefore related to the fact that I do not know whether there are already acceptable solutions readily available for the mute.
If there is value for offering such a solution for the mute accepting that a comparible solution for the deaf is further away then I would seriously consider putting some of our resources into offering such an offering from the building blocks that we already have available.
I see what you mean now - I'm not sure, you'd have to do a little research. I think perhaps so - think of people who are in hospitals with their mouths wired shut, or throat cancer victims, etc. etc. who have all their faculties but temporarily are unable to speak? A few Skype-enabled notebooks with the right plugin might be kind of nice.
Personally, I'd far rather type chat to someone who is deaf / mute than interface with some synthesised bot.
Sure, but not everyone has that capability - think of everyday phone calls you make where that would be impractical. Calling a restaurant for reservations, etc. etc. etc.
As technology progresses, I definitely see the spoken voice becoming more and more human like. Currently the latest text-to-voice software boasts the ability to place intonation into the speech, correctly pronounce colloquialisms, and even begin to place accents on the speech.
I think in 5-10 years the tech could be to the point where one might be hard pressed to distinguish between computer-produced speech and a human.
As technology progresses, I definitely see the spoken voice becoming more and more human like. Currently the latest text-to-voice software boasts the ability to place intonation into the speech, correctly pronounce colloquialisms, and even begin to place accents on the speech.
I think in 5-10 years the tech could be to the point where one might be hard pressed to distinguish between computer-produced speech and a human.
Hi all, first time that i have used a forum.... And was wondering how this thought has come about and whether I could either hop in and have a chat or ask your advice since you are having this discussion which is kinda along the lines of what I have struggled with for the last 20 years!!
I am profoundly deaf and a CI wearer (Cochlear Implant), and whilst my brother and I were searching the net for a service we found this string of disscussion.
My problem I have with the current methods of 'talk to chat/type' is that it is very restrictive and time consuming.
I am trying to find a company or service whereby I can access more than one voice. e.g. a telephone conferance call. This would need to be of a business level quality and complies with the data act and confidentaility side of things.
If you guys have any thoughts or pointers that you can give to me I would be very grateful. There is a market out there, however small it might be... You are already thinking of wider markets when you mentioned ppl in hospitals etc. I know that both my company and the charity I work for (DELTA - Deaf Education through Listening and Talking) would be intrested.
I am gonna stop here, and let you guys have a chance to feedback on this. Cos I could go on forever...
Regards
Steve Matthews
Boy - I have just spotted the date on this discussion... Will this still reach the ppl that were in this forum...?
Steve Matthews
I am profoundly deaf and a CI wearer (Cochlear Implant), and whilst my brother and I were searching the net for a service we found this string of disscussion.
My problem I have with the current methods of 'talk to chat/type' is that it is very restrictive and time consuming.
I am trying to find a company or service whereby I can access more than one voice. e.g. a telephone conferance call. This would need to be of a business level quality and complies with the data act and confidentaility side of things.
If you guys have any thoughts or pointers that you can give to me I would be very grateful. There is a market out there, however small it might be... You are already thinking of wider markets when you mentioned ppl in hospitals etc. I know that both my company and the charity I work for (DELTA - Deaf Education through Listening and Talking) would be intrested.
I am gonna stop here, and let you guys have a chance to feedback on this. Cos I could go on forever...
Regards
Steve Matthews
Boy - I have just spotted the date on this discussion... Will this still reach the ppl that were in this forum...?
Steve Matthews
Weird to be adding my 2c worth so long after the fact.
Anyhow, can't Skype handle this as a conference call? So two people are conversing and the HOH one catches as much as they can. Meanwhile the 711 operator "lurks" and transcribes whatever the hearing person is saying, and it shows up in the chat window.
If there are more than two people conversing that hardly changes things, except that the operator has more people to transcribe.
Most states contract with various companies and perhaps others to provide these services, so why not fund the same thing when goes through Skype or WLM?
Anyhow, can't Skype handle this as a conference call? So two people are conversing and the HOH one catches as much as they can. Meanwhile the 711 operator "lurks" and transcribes whatever the hearing person is saying, and it shows up in the chat window.
If there are more than two people conversing that hardly changes things, except that the operator has more people to transcribe.
Most states contract with various companies and perhaps others to provide these services, so why not fund the same thing when goes through Skype or WLM?
Nope.
Hi!
I heard some univerities are doing researches related to transfrom sign-language to text or voice.
This could be a new interface to help people with such disabilities.
What do you think?
gabor
I heard some univerities are doing researches related to transfrom sign-language to text or voice.
This could be a new interface to help people with such disabilities.
What do you think?
gabor
Accurate speech to text will be wonderful when it finally arrives. Meanwhile human interpreters provide speech-to-text translation. In most states this service is free to people with handicaps preventing normal phone. It is financed with taxes on all phone calls. For example I use Sprint's interpreter service in California. Skype might be able to qualify to be paid to do this and provide the interpreters.
As a non-signing HOH individual with some residual hearing, I find Skype useful without an interpreter when I call others who use Skype online. We can switch between voice and text, depending on how demanding the material is. Ordinary speech has redundancy that helps when I don't catch every word, however there is no redundancy with things like addresses, phone numbers and so forth that need to be 100% accurate.
If we have enough bandwidth so video works smoothly, that helps me lipread.
As a non-signing HOH individual with some residual hearing, I find Skype useful without an interpreter when I call others who use Skype online. We can switch between voice and text, depending on how demanding the material is. Ordinary speech has redundancy that helps when I don't catch every word, however there is no redundancy with things like addresses, phone numbers and so forth that need to be 100% accurate.
If we have enough bandwidth so video works smoothly, that helps me lipread.
Fail troll. 
Hey guys,
I have been trying to research some information about Skype for Deaf AND Deaf-Blind users so I was happy to come across this post.
I work with Deaf-Blind adults, specifically in the field of assitive tech. I've found that speech to text is very difficult for most - if even they have the capability of clear speech. Software programs, like Dragon Naturally Speaking with the JAWS and J-SAY interface is under this idea, but the user need to train their voice until it will work as they want it to.
I use a program called NexTalk (www.nextalk.net) for my Deaf and Deaf-Blind students. It has text to text calls, relay calls via text as well as a VCO (Voice Carry Over) service where the user can pick up the phone and use their voice, but read on the screen. There is an operator on the line, but they never interfere with the VCO calls. The program also support screen-readers for those who use braille.
I don't want to get off the point or try to "promote" another service. I really like Skype and many of my full blind users love it. It's those user who are completely Deaf who also have Low Vision that it's very difficult.
The call will pop up on the screen, but it's not enough of a signal to get their attention, even if they're right in front of the screen. Way back when, when there was dial-up, we used to be able to attach an external light signal to the dial-up modem so it would flash when a call came in. Now with advanced technology we're not able to do this anymore.
I'm wondering is Skype can have a "full screen" option for a signal flasher or have some way to attach an external light/vibrator (like the VibraCall) to let the Deaf-Blind person know they are getting an incoming call.
Maybe one of the Skype designer/engineers can comment on this???
Thanks for taking the time to read this.
Kelly
I have been trying to research some information about Skype for Deaf AND Deaf-Blind users so I was happy to come across this post.
I work with Deaf-Blind adults, specifically in the field of assitive tech. I've found that speech to text is very difficult for most - if even they have the capability of clear speech. Software programs, like Dragon Naturally Speaking with the JAWS and J-SAY interface is under this idea, but the user need to train their voice until it will work as they want it to.
I use a program called NexTalk (www.nextalk.net) for my Deaf and Deaf-Blind students. It has text to text calls, relay calls via text as well as a VCO (Voice Carry Over) service where the user can pick up the phone and use their voice, but read on the screen. There is an operator on the line, but they never interfere with the VCO calls. The program also support screen-readers for those who use braille.
I don't want to get off the point or try to "promote" another service. I really like Skype and many of my full blind users love it. It's those user who are completely Deaf who also have Low Vision that it's very difficult.
The call will pop up on the screen, but it's not enough of a signal to get their attention, even if they're right in front of the screen. Way back when, when there was dial-up, we used to be able to attach an external light signal to the dial-up modem so it would flash when a call came in. Now with advanced technology we're not able to do this anymore.
I'm wondering is Skype can have a "full screen" option for a signal flasher or have some way to attach an external light/vibrator (like the VibraCall) to let the Deaf-Blind person know they are getting an incoming call.
Maybe one of the Skype designer/engineers can comment on this???
Thanks for taking the time to read this.
Kelly
hi,
I'm an engineering students, and I'm intended to do my PhD research on helping deaf/mute communication ability by using new technology
I have started reading and doing some research about it, I come across to the previous discussion,
and somehow i have some question that might help me, also i am planning to visit deaf/mute school in my country(Malaysia)
1. do u hear about Sign Language Voice Translator;
(it consists of Camera and Speaker) it will capture the sign language movement and translated into voice.
if yes, can you explain me, how is the performance, and the capability of this technology, and what do you suggest?
if no, what do you like from the modern technology to provide deaf/mute people?
2. if you have any information you think that might help me, i would be appreciated to hear it.
thanks
best regard
Osamah Kelantan
I'm an engineering students, and I'm intended to do my PhD research on helping deaf/mute communication ability by using new technology
I have started reading and doing some research about it, I come across to the previous discussion,
and somehow i have some question that might help me, also i am planning to visit deaf/mute school in my country(Malaysia)
1. do u hear about Sign Language Voice Translator;
(it consists of Camera and Speaker) it will capture the sign language movement and translated into voice.
if yes, can you explain me, how is the performance, and the capability of this technology, and what do you suggest?
if no, what do you like from the modern technology to provide deaf/mute people?
2. if you have any information you think that might help me, i would be appreciated to hear it.
thanks
best regard
Osamah Kelantan
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.