I'm doing a paper for VoIP services and currently researching disability regulation, and it got me thinking.

Is there currently any method, plugin, way, for someone who is deaf or mute to use Skype (or any other computer based VoIP product?) I got to thinking that it would be easy to write a program that would translate incoming voice as text, and the deaf/mute person could then type their response which the program would translate into computer speak. It'd be a chat for one person and a normal phone conversation for the other.

I know the technology is possible, I'm wondering what's out there. Such a program would 1) eliminate the need for TTY type systems, 2) allow those with disabilities to make calls anywhere and everywhere without using special TTY enabled numbers, worrying about finding TTY phones when traveling (imagine a deaf person in an airport with a PocketPC, foldout keyboard, and a wi-fi network calling their loved one to let them know they're OK), and 3) eliminate the need for the special TRS facilities and such to process conventional TTY calls.

I'd be interested in such a system, too - imagine making quiet phone calls during a meeting, class, etc.

Thoughts? Anyone know of a system or something close, or someone who's currently working on something similar?

Apologises greigmg as my commnets may be a massive over simplification of the situation but isn't a TTY phone service the ability to convert text to voice and voice to text? That being the case, would the use of the Skype IM service not be more appropriate than the voice service across Skype?

Skype provides visual notifications to show when a 'call' is requested - much like the flashing bulb on a TTY telephone - so the solution is potentially someway towards meeting the needs of deaf or hh users.

I realise that this would not assist the SkypeIn/SkypeOut communication and the work we are involved with in incorporating speech to text and text to speech (our product WiSPA is one example) into Skype communications may provide some opportunities to address this specific requirement. Would be interested to understand more on what a true TTY solution should deliver separate from where Skype is today.

No, you're exactly right - the problem being that not everyone I want to call uses any sort of chat system (e.g. my car's in the shop - how do I explan to the mechanic that I need him to install AOL IM on his business computer so I can get updates on the job?).

As I understand it, those disabled currently have two options - 1) using a TTY phone, which is essentially a rudimentary chat device (keyboard and one or two line display) they call a TTY number, and chat with the person on the other end. Many large businesses and govt agencies you'll see have an option for a TTY (sometimes labeled TDD) number. Or 2) again using a TTY, they can call conventional numbers, and the conversation in essence goes through a translator, called a "communications assistant," that basically has a TTY and a regular phone in front of them, and as the disabled person types, they read the message to the other person, and as the other person talks, they type the message to the disabled individual.

I am not exactly sure how option #2 works, thought I do know the FCC mandates similar pricing, 24 hour availability, etc. for such relay systems.

I envision a plugin or service for Skype or other VoIP systems that would in essence cut out the middle man and eliminate the need for special systems like this. All the processing would occur on the disabled person's end - as speech comes into their computer/pda/etc. it gets translated into text, and as the user types it gets translated into speec which is then sent over the VoIP service.

Thanks for that clarification.

We have already developed the technology linked to Skype that could provide a solution for a mute user. This would deliver an environment where a mute user would answer their Skype telephone and listen to the caller in the normal manner but type their side of the conversation. The typed text would then be spoken over the VoIP connection as they typed for the other party to hear on their telephone.

The bigger issue is where the Skype user is deaf. This would require the spoken conversation from the telephone user to be converted to text so that it can be displayed for the deaf party to read. Although speech recognition has made significant advances in recent years, I do not believe it is yet able to handle the variety of accents, the quality variances from telephone systems as well handling an open dictionary with no context.

Most successful speech recognition solutions either have an awareness of the words they are to receive or have been trained to a specific voice.

Solutions in the first category such as speech self service system or call centre front ends etc have a context which allows the designer of the solution to predetermine the range of words that are likely to be spoken. This allows the recognition engine to make intelligent guesses of what is being spoken from a restricted vocabulary.

To take free format speech, as with a typical conversation, would require the speech recognition system to be trained to a specific voice before it was reliable. This is obviously impracticable for a solution that is to accept telephone calls from anyone.

At the speed of advances in this space I am sure that a solution will be possible and I welcome the thoughts and experiences of others as extending Skype in this manner could deliver significant value.

You're right about the need to "train" a voice rec program before it works effectively. I think a forward approach would be a combination of voice recognition and sentence prediction - Microsoft's handwriting recognition on Windows XP Tablet does something like this, and works very well, it combines standard handwriting recognition with a dictionary so it can make "educated guesses". However, I don't think any software currently available could even come close to recognising voices over phone calls.

Right, the speech-to-text end seems to be the rub. Bill Gates says within 2-5 years it'll be at that point.

Working with Speech Recognition systems over the last four years and seeing the advances made in that time I would say that's a fair assessment.

This raises the question - is there sufficient benefit to deliver a solution for users with speech difficulties or is the real problem with those who are deaf or hard of hearing?

Huh? What exactly do you mean?

Sorry for the confusion. In my earlier post I mentioned that a solution for users that are mute or have speech difficulties is technically feasible today. It is the technology to support the deaf or hard of hearing that was the current difficulty.

My question therefore related to the fact that I do not know whether there are already acceptable solutions readily available for the mute.

If there is value for offering such a solution for the mute accepting that a comparible solution for the deaf is further away then I would seriously consider putting some of our resources into offering such an offering from the building blocks that we already have available.

I see what you mean now - I'm not sure, you'd have to do a little research. I think perhaps so - think of people who are in hospitals with their mouths wired shut, or throat cancer victims, etc. etc. who have all their faculties but temporarily are unable to speak? A few Skype-enabled notebooks with the right plugin might be kind of nice.

Personally, I'd far rather type chat to someone who is deaf / mute than interface with some synthesised bot.

Sure, but not everyone has that capability - think of everyday phone calls you make where that would be impractical. Calling a restaurant for reservations, etc. etc. etc.

As technology progresses, I definitely see the spoken voice becoming more and more human like. Currently the latest text-to-voice software boasts the ability to place intonation into the speech, correctly pronounce colloquialisms, and even begin to place accents on the speech.

I think in 5-10 years the tech could be to the point where one might be hard pressed to distinguish between computer-produced speech and a human.

Hi all, first time that i have used a forum.... And was wondering how this thought has come about and whether I could either hop in and have a chat or ask your advice since you are having this discussion which is kinda along the lines of what I have struggled with for the last 20 years!!

I am profoundly deaf and a CI wearer (Cochlear Implant), and whilst my brother and I were searching the net for a service we found this string of disscussion.

My problem I have with the current methods of 'talk to chat/type' is that it is very restrictive and time consuming.

I am trying to find a company or service whereby I can access more than one voice. e.g. a telephone conferance call. This would need to be of a business level quality and complies with the data act and confidentaility side of things.

If you guys have any thoughts or pointers that you can give to me I would be very grateful. There is a market out there, however small it might be... You are already thinking of wider markets when you mentioned ppl in hospitals etc. I know that both my company and the charity I work for (DELTA - Deaf Education through Listening and Talking) would be intrested.

I am gonna stop here, and let you guys have a chance to feedback on this. Cos I could go on forever...

Regards

Steve Matthews


Boy - I have just spotted the date on this discussion... Will this still reach the ppl that were in this forum...?

Steve Matthews