Reading Bilingual Books on iOS is An Albatross, and Something should be Done about It
Apple's iOS operating system doesn't suffer from a paucity of ebook readers. In fact, the situation is so exemplary that one might ask for advice when it comes to selecting between the $14.99 Voice Dream Reader and the free but feature-rich Dolphin EasyReader. And I haven't yet mentioned Apple's very own ebook reader and purchasing hub, iBooks, which is to be called Books in an upcoming iOS update. Oh, and let's not forget Google Play Books, NOOK and Amazon Kindle. These are all accessible and you can even purchase extra high-quality voices for some of them to enhance your reading experience. Guys, we're practically spoiled! But I'll be Panglossian if I claim reading all sorts of books on iOS is a pleasant experience because, well, it isn't.
What's wrong with book reader apps on iOS?
The issue of complicated, nonstandard tables and graphs aside, nothing will go wry as long as your chosen book is monolingual. That is, as long as the text of your book is in a single language, you won't encounter hurdles. But as soon as you reach a line, sentence or paragraph written in a different language, you'll notice the problem.
To make the record straight, however, a clarification is in order.
VoiceOver itself is perfectly capable of altering the TTS engine voice and switching to the proper language as soon as it detects text written in a different language. Needless to say, this hinges on the proper use of language tags. So if a French block of text isn't specified as being French, VoiceOver can't make the switch and the culprit is to be found elsewhere. However, in spite of the fact that VoiceOver is a consummate multilingual screen reader, most -- if not all -- book readers on iOS aren't, and this shortcoming manages to torpedo the pleasure of reading bilingual or multilingual books and passages -- stuff mostly found in educational offerings.
Three examples are worth a thousand paragraphs!
If you're reading this article with VoiceOver on iOS, you should hear the following sentence read via a French voice: Ceci est une courte phrase française et sert d'exemple pour mon article. That's because I've used the proper "span lang" tag to mark the above sentence as French.
Now let me provide a more complicated bilingual example, taken from the provided language notes for my favorite French courses, Coffee Break French:
gender: Just like many other languages, French uses different “genders”. Some words are identified as “masculine” and others are identified as “feminine”. Note that masculine words are not only “male” beings or creatures, and likewise that feminine words are not only “female” beings or creatures. For example, the word maison (meaning “house”) is feminine, and the word jardin (meaning “garden”) is masculine! There’s no reason for this - the best idea is just to learn whether a word is masculine or feminine when you learn it. Note that while nouns are masculine or feminine, there are also masculine and feminine forms of adjectives too. We’ve already come across two of these: bon/bonne and enchanté/enchantée.And another example from the very same courses, this time in the form of French expressions followed by their English translations:
à plus: see you later. This is quite informal. Note that you pronounce the ‘s’ of plus.
à toute à l’heure: see you later.
à plus tard: see you later. In comparison to à plus, you don’t pronounce the ‘s’ of plus in the phrase à plus tard.
I can't read my bilingual language books using Voice Dream Reader and iBooks, and that doesn't bode well for visually impaired language learners or bilinguals
To put it in a nutshell, if I email a properly formatted/tagged HTM/HTML document to myself as an attached file, VoiceOver will read it as expected by making proper language switches. However, if I send the same file to Voice Dream Reader, the app will fail to switch to an installed French voice where French text blocks appear. It lacks proper language identification/switching features. More importantly, however, the same is true about Apple's iBooks. Upon trying to send the same document to iBooks, it provides me with an option to convert it to PDF, and the PDF file won't be processed by iBooks in terms of language identification and voice switches -- everything is read in pure English.
Now my question, or grouse, is why iBooks, with its universal appeal, provides zero language recognition, identification and switching features. By the same token, why should an expensive app like Voice Dream Reader fail to take the needs of those who read bilingual and multilingual passages into account? It's quite evident that language learners or bilinguals can't get the most out of their expensive iDevice purchases if the status quo isn't rectified. Any chance of diverting the "i" of "iBooks" in iOS 11.3 to the realm of proper language identification? I'm fine with an "lBooks!" How about you?
this is an excellent post and i would like to add my thoughts.
I speak french and really enjoyed the examples taken from coffee break french. i am learning spanish and i wanted to purchase that course. however, if i can't hear the spanish, well, in spanish, i don't see its value of me spending 40 uk pounds for a course with explanatory notes that will be quite frankly, unusable. what i did notice though was when the original poster stated to read these texts with voiceover, i had to switch my language to the "default" option, which in my case is samamtha to allow the switching between english, and french in the examples to occur. it would not do this if the rotor language was set differently. why even for the coffee break french course, did it switch? did you mark the examples or did it just work of its own accord? could i not just read the notes for the course in safari to experience the language switching? i would really have thought, no offence here that voice-dream would have done this switching effortlessly after all it is a paid item.
First off, glad you liked the post. So you speak a language which I'm doing my best to master -- good for you!
It's true that in order for language switching to occur, one should use their default Rotor language so as to essentially turn on VoiceOver's linguistic prowess! Of course, if you have installed a higher-quality voice for your second/third languages, you'll get a better reading experience.
As for Coffee Break courses, their notes are in PDF. I open the PDF files in MS Word and since Word correctly detects some of the language notes correctly, I myself make the rest of the corrections and essentially tell Word which text blocks are, say, in French. I then save the file as an HTML file to have a perfectly modified language-tagged document for use with NVDA and JAWS. Safari on iOS also works, but I'm wondering how you can tell your iPhone's Safari to grab and open these HTML files especially when they gradually cumulate. Honestly if Voice Dream Reader had bilingual support, it could dramatically facilitate the process.
Finally, while we're at it, if you're thinking about Rocket Languages and their Spanish courses, I should inform you that you'll encounter the very same issues there. Even worse, with the Rocket Languages app which acts more or less like a website, VoiceOver does no language switching at all because Rocket Languages hasn't tagged their course notes. If they do, Rocket Languages courses and their app would instantly become the most suitable language learning platform for VoiceOver users on iOS.
Hi! Even though I have never done any kind of language course on IOS, I have read novels in English which happened to have passages written in other languages in them, French for example, so I can totally relate to this post. I admit that when I am reading a book, in iBooks, Kindle or Voice Dream Reader, I don't always use my default VoiceOver voice for that, but it seems that doing so doesn't make a difference in those book readers anyway. Even on the computer, language changes aren't always well tagged, so I'm not surprised they aren't on i-devices either. Luckily I don't read many books where there are sudden language changes, but it would still be great if VO would detect those changes when they occur, and switch to the right language when necessary in iBooks or Kindle, or that Voice Dream Reader could switch to another language's voice at the right moment.
Amir, perhaps the reason that Voice Dream Reader does not support language switching is that the text-to-speech API that it is using does not support this feature.
What do you mean when you say "I'm wondering how you can tell your iPhone's Safari to grab and open these HTML files especially when they gradually cumulate"? Could you save the files to the reading list in Safari?
hi all, how would it work though in voicedream with a pdf document, such as those for learning languages, if it was not tagged propperly?
Also, to the original poster, did you know that a reading club for coffee break languages is coming out? each week they will send you a pdf french text in your case, along with english translations. but of course the french would read say, in voicedream, but not the english if it were below the french. for a paid product, i would hope voice-dream can do this, particularly as it is marketed as an educational tool.
I'd forgotten the case of bilingual novels -- some famous ones written by Russian authors in particular. Reading them at the moment with various book readers isn't feasible on iOS given the language-related limitation you're aware of. The crucial point is that on Windows this feature can be extended to, say, Chrome which can open and read properly language-tagged files. However, on iOS book readers stop short supporting it.
TJT 2001, that API limitation might very well play into this, but we can't be certain if it's indeed the case. Even so, one can think that Apple can do something for it -- for iBooks at least.
As for reading those HTML files in Safari, what I meant was that the guides are in PDF. So I open them via MS Word on Windows, make sure the language tags are properly in place, save the files as HTML, and send them to myself as email attachments. Once accessed on my iPhone, they can be shared with apps like iBooks and Voice Dream Reader, but I can't find a Safari Reading List in the sharing options presented to me. Even if such a feature/option exists, how can Safari organize tens or even hundreds of such files for a certain course in, say, an alphabetical order? I think Safari isn't probably suitable for that.
Will, the point is that Voice Dream Reader doesn't support language switching at all, whether or not your PDF files are properly tagged. And, while we're at it, Voice Dream Reader cannot access and use high-quality Siri voices made available in iOS 11 -- it keeps using the low-quality ones even if you download high-quality voices for your Rotor from the Speech window. This means I should purchase an extra French voice for Voice Dream Reader despite the fact that I prefer Siri's high-quality female French voice to all French voices provided by other TTS engines.
hi amir, send me a private message if you wish but was just wondering. i am subscribed to coffee break spanish in overcast but if i paid for the premium version can i access all the bonus audio etc. direct in my podcast client and the notes even though they don't read right for us? or how would it work?
Yes, Will. If you purchase the Premium version, you can grab all audio files via your desired Podcast client. Course notes and language material are provided to you as downloadable PDF files.
hi amir and all, if voice-dream could implement it i'd be happy, but don't see how voiceover can switch without the tags.
Have you contacted the developer of Voice Dream Reader about the language switching ability? Perhaps it is something that can be incorporated into the app but they currently do not know it is something that customers want. I have heard that the developer is willing to listen to and work with VoiceOver users, so it is worth a try, I think.
I've not yet done so. I wanted to have some feedback here before informing VDR's developer because the more people want such a feature, the more we can be optimistic about its implementation provided that Apple's APIs offer the required infrastructure.
"As for reading those HTML files in Safari, what I meant was that the guides are in PDF. So I open them via MS Word on Windows, make sure the language tags are properly in place, save the files as HTML, and send them to myself as email attachments. Once accessed on my iPhone, they can be shared with apps like iBooks and Voice Dream Reader, but I can't find a Safari Reading List in the sharing options presented to me."
Have you tried sending to Dropbox? I just did this with an HTML file that has both English and Spanish content. The HTML files can be read accessibly in the Dropbox app and also sorted alphabetically. This may not be as ideal as viewing in VDR, but it works.
Thanks for the solutions you provide.
I haven't tried Dropbox because, ideally of course, I'm seeking a solution which doesn't require internet connection. With Dropbox and Safari I can't auto-save the cursor position around various reading sessions. That combo doesn't offer note-taking features of VDR or similar book reading apps either. But I'll take a look at it.
I realize that is a controversial headline.
It would be nice for developers to understand that we need special tools to help us to learn when we have no visual cues. Part of learning another language involves recognizing what objects and people are talking about.
For years I've thought about taking a second language course. But, when I looked into it, it seemed very difficult especially since these courses are not designed for people with vision loss. The web resources I checked out for the blind say that learning another language is difficult for various reasons. I agreed with those reasons. I did not do well at French in high school. I live in Canada. It would be nice to know our second official language.
amir, how would i tag the files (i use a mac though) to make the mac itself or voiceover on my iphone switch the languages? i would be prepared to if i can, tag every lesson note of all languages to get it working for us. it will be a huge undertaking but i feel it would help us as blind users.
I come from the Windows side, so am not sure if the process is the same on your side of the pond. But in MS Word 2013 and 2016 I select the block of text I want tagged, and press the following key combination: ALT+R, then U, then L. That will open the Proofing language dialog, and I can select my desired language there for the selected text -- it may be a word, a sentence, a line/paragraph, or an entire document. Please note that once you're there, make sure the "Detect language automatically" check box isn't checked. It's checked by default, so you should uncheck it. If it remains checked, once you close your file and re-open it, MS Word will screw up your language tags by trying to detect the proper language on its own.
FYI, ALT+R opens the Review tab or ribbon, U opens the Language Grouping window, and L opens the Language preferences dialog.
does anyone know if i set languages in pages for highlighted text as Amir just described for windows?
I can't speak for many developers. Still, I believe that voice dream, at the least, should be able to add this feature given that it already employs language detection so that documents can be read in any single language. Extending it to detect properly-tagged languages should be very feasible. I am also going to collect the requisite data for voice dream to use the siri voices in IOS 11. These have ceased to be available because apple has introduced new versions of the databases, but adding support to them again should be relatively straightforward as long as we can identify the names of the new databases. I would suggest that we recommend these changes to the developer of voice dream. Although I had assumed that development had been slowed given a long-unsolved bug in voice dream where the app would refuse to have any audio output after it had been paused for a while, an update was recently released that supposedly patches that bug (although a quick test seems to indicate that it may have added a different bug). As soon as I have found the names of the new siri databases, I will be suggesting both changes to the voice dream developers. I encourage you to do the same.
Thanks, Nathan, for your encouraging words. At the very least, I think informing the VDR developer would be a good idea. I'm going to tweet him and direct him, using the @VoiceDreamApp handle, to this article. If others do the same, I think we'll take a great step forward.
On a similar topic, you're absolutely right that the development of VDR has slowed down. Correct me if I'm wrong, but the news of the latest VDR update, made available a couple of days ago, wasn't tweeted at all. I'd also like to be able to use new Siri voices provided by iOS 11.
Very interesting observations, Bobcat. As a current translator/interpreter and a former language teacher, I can attest to the fact that most language textbooks, loaded with images and visual pointers, aren't suitable for the visually impaired. Heck, even scanning/converting them to readable text via off-the-shelf or specialized OCR apps is a mind-numbing challenge. Of course, the situation is a whole lot better with modern online language course providers such as Rocket Languages and Radio Lingua, developers of the popular Coffee Break series, provided that they commit to offering properly language-tagged transcripts.
It has definitely been slowed down. Unfortunately, my statement about adding a new bug has proven extremely correct. Many voices will become virtually unusable after crossing a boundary (a heading in a format with those, but also some random areas of other types). Resolving this requires force-restarting the application. I'm doing testing to determine which voices are affected--I know at least some of the IOS voices are included, but the acapela ones do not seem affected from very brief testing. Now I have another reason to contact the devs, and I'd recommend not installing that update if you often use the built-in voices or documents with many boundaries.
The harm is already done as I installed the last VDR update few days ago. I've also seen the bug, but thought it's something extremely random which can be ignored. But it turns out to be more serious than I thought.
Your efforts are appreciated!
hi amir et and all, have just e-mailed winston from voicedream and linked to this discussion. could voicedream do the language switching or would the files still have to be tagged for it to readz this would still not let us even in vdr read coffee break language notes amir.
Thanks, Will. I'll also be doing that later today.
In order for the books and files to be properly read, they should be tagged. However, tagging them can be done quite easily -- even by ourselves, and the web is also brimming with properly tagged stuff.
just trying to see how in pages or on mac in general i set language specific tags then i could get to work on the coffee break stuff
got this from coffee break languages, when i mentioned about the tagging and could it be done. i know we'd have to get the pdf in to%text edit add tags i guess, and save as html. can we who are interested in at least reading languages try and get on-board? if someone can message me the syntax for tagging texts in html i'll start to work on the sample that Mark will provide. i will of course share the example with their permission with applevis so we can see if it works when reading in safari.
Coffee Break Languages, 12:05 pm, Hi William,
Thanks for your message. We have been looking at more ways to make our content as accessible as possible and your message is very interesting.
I've actually been looking into this this morning and trying to look at the methods required to produce correctly tagged documents to allow the foreign voice to read the content correctly in the language. My research has actually taken me to the AppleVis website where I've read the discussion about the Coffee Break courses.
We work with Macs and create our lesson notes in Pages. It used to be the case that Pages allowed you to identify the language or a particular word or phrase and we used character styles in order to identify these. Pages now identifies the language automatically, but having done a number of experiments this morning it appears that this is only for spelling autocorrect, rather than to output any kind of tagged content.
I've tried recreating the files in Word, Google Docs and text editors, but only Word allows you to identify the language of a word or sentence, but even then the PDF export or the HTML export doesn't include the tags.
I've also downloaded GhostReader Plus and tried that - it seems to work perfectly for reading the text aloud on my computer and creating audio versions of the text with the correct language, but it doesn't seem to allow exporting the text with tags.
I'm very happy to look further into this, but I would want to make sure that if we move ahead with reworking our entire back catalogue into a format that allows correctly tagged documents for visually impaired learners, we do so in an appropriate way. We have many hundreds of documents across our courses, and if this is something we can take on ourselves we would be happy to start that work. It is a huge undertaking, and the ideal scenario would be that we could find a way to produce the documents in the first place with a system which allows for correct text to speech based on the multiple languages in one document.
Can I ask what system you would use for marking up the text with the language tags?
The other aspect which we'd need to consider is that our text documents are part of our premium content: visually impaired learners wanting to access our free content can use our audio podcasts. The lesson notes would only be for paying customers, and therefore a marked up version of these notes would also be for paying customers. I hope that makes sense.
I'm very happy to continue the discussion and hope that this is of some use. Equally, I'm happy to provide access to some content for you to work on. You can let me know what format works best.
First off, thanks for helping us move this important issue forward. Awesome job I'd say!
As someone who works with MS Word 2016 to tag my Coffee Break files, I'd like to correct one of Mark's statements. He said:
I've tried recreating the files in Word, Google Docs and text editors, but only Word allows you to identify the language of a word or sentence, but even then the PDF export or the HTML export doesn't include the tags.What I can tell is that if Automatic Language Detection is disabled in MS Word, it successfully exports all language tags upon creating HTM, HTML and MHT files. I've not tested PDF creation, though, but am sure all tags are preserved in the exported HTML documents.
I'd be glad to help them in case they need assistance in one or more phases of this so-called language-tagging transition. Kudos to them for taking accessibility so seriously!
If you are writing directly in HTML, defining alternate languages is done with the span tag and the lang attribute. For example, one would type <span lang="es"> to initiate the change to Spanish (language code es) and a </span> to change back. The primary language is set on the html tag, again with the lang attribute (<html whatever-specification-is-in-use-here lang="en">
I would recommend not doing this manually, but at least you know how to do so. There are a number of methods that could be used. One is to use a custom program that reads in data and puts the tags automatically--for example, a script that would recognize a specific string could just replace it with the proper span tags and would take five minutes to write. It may also be feasible to do so in markdown or another shorthand system. I don't know how to do it from within pages, and I imagine that even in word retroactively marking up any document would be painful.
ok mark from coffee break languages, has sent me some notes from a french course for premium users in pdf, rtf and doc. the audio version he has produced, was done with ghostwriter on the mac. where can i get this? this works beautifully and uses, in his case Daniel for the english, and a compact french voice. would those of us who want to study these courses, want audio notes, or written? i assume it is quite easy to use ghostwriter as it seems to mark up the switch for us, but isn't that like listening to another podcast?
Obviously we couldn't see with the cursor how a verb for example, is spelt. what do people want from this? i'd be happy if i could get ghostreader, to do this for people. i would show the example he sent me but of course its copyright. where can i get this markdown or something similar as i want to manipulate the doc file to get it in to htm to read in safari and see if i can make voiceover switch. thoughts?
Will, personally speaking, I'd rather have access to their tagged text because this way I can check spellings. Moreover, as we progress into their courses, being able to check spellings will become more critical. With their marked text, you can even use better/higher-quality voices for the readout of foreign language stuff. For instance, on iOS I use the high-quality French Siri voice for that purpose and prefer it to the rest of French voices provided by VoiceOver.
Hi everyone. As you'll be aware I've been in touch with Will and we're very keen to make our courses more accessible. Obviously this is a long process, but if we can change our current processes to ensure that, moving forward, our new content will be using a format which will help visually-impaired learners use our lesson notes, then we can start working on our back catalogue too.
I'm not quite sure if it's possible to post a file here in the forum, but I'll try to post our experiments with GhostReader. They only allow you to export the file as an audio file, but I've contacted the developers to see if it's possible in some way to export the file as a correctly tagged html. I'll keep you posted.
First off, accolades are in order for your stupendous efforts aimed at making your course notes and lessons more accessible to the visually impaired. I know it takes a lot of time to go through all courses to tag their extensive notes, and that's just an emblem of your commitment to accessibility.
I just took a look at the MP3 file you posted and the audio is quite usable now. However, what I'm mostly interested in is the tagged HTML file itself because that way I can pause the reading process and check spellings when desired. Moreover, I can read both the English and, in this case, the French text using my favorite TTS engines -- TTS is quite a subjective issue among the visually impaired. So I'd appreciate it if you could look into the possibility of distributing the tagged HTML files as part of your Pro/Subscription-based packages.
Looking forward to hearing from you.
To Mark, I am wondering what source format you are using for this process--it seems that it already has language tags. Given this, a program that converts it to a properly-tagged HTML file, assuming such does not already exist, would be relatively straightforward to write. If you are interested, I am willing to volunteer some time to write such an application given the current specifications you are using. I have never used your language courses before, but from the short excerpts posted here, it does seem relatively straightforward. In addition, there exist certain detection libraries for determining the language used in an arbitrary text string, usually the result of machine learning algorithms. This system could be put to use in tagging of older data that currently lacks this. If any of these suggestions are of interest to you, you can of course reply here or send me a message and I will do my best.
To the other posters here, the issue of Coffee Break languages, while perhaps important to some, is merely a symptom of the larger problem noted in the post. Even if the properly-tagged HTML is created, it will mostly be usable in web browsers. E-book programs seem to disregard these tags anyway, so we should re-focus our efforts to requesting that the book reading programs be updated to support language changes. This might not be a simple change to make, as some programs might not currently plan on multilingual data, but it should be well within the capabilities of developers to add this feature with some time. If you are a user of a book-reading app, it could be quite helpful for you to suggest this change to the developer of it.
Thanks for paying close attention to the points raised here. You're absolutely right about ebook readers being the major issue. A piece of good news is that Winston, the Voice Dream Reader developer, told me on Twitter that he'd take a look at this article and address the issues raised. I'm looking forward to hearing from him.
I have just tested some book reader apps to see what they do with multilingual text and documents that are properly tagged. Unfortunately, I can substantiate all reports of bad handling. I tested four. Easy reader from dolphin gets eliminated first because it can't read HTML, PDF, or basically anything. Evidently I should have read the manual first. So it bombed out rather fast. ClaroSpeak lost second because, although it can support certain formats, HTML is not among them. I exported the test documents to PDF, which worked in that the data was there, but all language tags were ignored. To clarify, I tested one document that was entirely in another language and another with mixed content. The only two that got the whole-document language were voice dream reader and capti narrator. They were both able to detect the language for the single-language document and switch appropriately. Thus, they should both be able to add language switching. For the mixed-language document, voice dream stayed with the original language of the document and didn't switch. Capti took the opposite approach and stayed in the second language. I have no good explanation, as the language tag was English, the first words were in English, and the Spanish didn't start until the middle. Either they got stuck on the Spanish because of the single-language document I ran before, or capti is just insane. Either way, neither app is any good at all. If you use either of them, it would be helpful to ask the developer to add language switching in HTML documents. Once we can get properly-tagged ones, at least that should function well.
Thanks, Nathan, for sharing the results of your tests. Given the prevalence of Voice Dream Reader and the innovations it's managed to introduce, I hope Winston can raise the bar and make multilingual support a reality. I also expect to see the very same level of support extended to Apple's iBooks/Books.
it's not the application problem it's the epub problem. they need to make the tag readable while using it on the app. I read the book with another language but if the book didn't do the proper tag then yes i can not read it but if they did then I won't have any problem.