Apple previews new accessibility features coming later this year: improved visual descriptions, auto-generated subtitles, and more

While these accessibility features are definitely helpful for users, I feel that the new additions are quite minimal and do not represent any major or groundbreaking improvements. Although they offer some benefits, the changes are relatively small compared to what I was expecting.

I will be very grateful to Apple if they fix quick nav issues on the Mac. For anything else, thanks Apple, I am really happy with the fixes, if they ever come.

Because at least this is honesty from apple. Bad, shows clear neglect of the screen reader on this os, but I am happy that at least they don't act like vo on mac is great like its ios and other platforms, which is simply isn't.
My poor iphone will not get any of the apple intelligence :( maybe time to upgrade? :) Will definitely try this out on the mac though.

wish it will become useful, and to see, how apple will do it.
as for me, my hearing aid always concentrate on 1 sound source, so, if apple can help me in crouded environments, it will be great.

These are nice upgrades. From the rumours, I didn't expect anything major or groundbreaking this time around. To clarify rumours are pointing to a Snow Leopard type update, with a heavy focus on Apple Intelligence, particularly the rumoured Siri upgrade. Also, during the beta cycle we will no doubt discover features or improvements neither discussed here nor are in the public domain. As always, time will tell what, if any, these will be. For now, let's all appreciate that even today Apple is commited to accessibility in their products.

Unfortunately, this is apple.
wish they will ofer more on ios27

Nothing too exciting for me. I wanted some more voices to check out for VoiceOver. Just because they didn’t announce it doesn’t mean we won’t get any of them so I’ll definitely be looking for those ones iOS 27 is released. I would like to know where to find one of those grippers for the phone I think that might be helpful for me as I have some mild CP and sometimes I get tired of holding the phone my hands so maybe this would help my hands feel better. Hope I have one that comes in pink.

Hi @Singer Girl,

Here is a link to the Hikawa Grip & Stand for iPhone on the Apple Store: https://www.apple.com/shop/product/hsjf2zm/a/hikawa-grip-stand-for-iphone-magsafe-compatible-orange-swirl

I'm definitely looking forward to any hidden gems and/or bug fixes in macOS 27. I sadly have an iPhone 11, and it's highly unlikely I'll get iOS 27, but that's okay since I'm primarily an Android user. I still can't believe they didn't talk about built-in accessible Mac remote control from last year. Am I the only one that's really excited about that?

Last year was a huge update for accessibility, addding all the braille access stuff. while it's a shame they didn't directly mention any braille updates, the thing apple desperately needs is another iOS 12 or a mac os snow leopard. they're taking apple intellligence more seriously, which is definitely much needed.

This was before I read most of it is for apple intelligence only, we really shouldn't be limiting accessibility features like this, talkback can describe images on any device that has talkback 13.1 and newer.

Further adding I would like a comparison between Apple's new Vision Explorer, Microsoft's Seeing AI, and Google's Look Out Assistant for those of us among the Android space, one of these days.

Ah, adding EyeGuide to the list, since it also has a similar feature.

I will be happy if apple fixes the blank bug. I am not ask for much. It is not like I am asking for peace in the world. Just a blank bug fix. Long live cats and down with apple.

while yes, android has on device image description, testing it out myself on my pixel... a, it's nowhere near as good as apple's image descriptions, unless you send i up to the cloud. which, nice. also, for those on device image descriptions, you need 8 or 12 gb of ram. also, google released its gemini intelligence, and that is even more strict. it can only be ran on the pixel 10. not even the pixel 9, with its 8 gb of ram. but either way, I'm glad apple is making more changes, and some pretty significant ones at that. they're taking apple intelligence more seriously.

It sounds like for the most part, they're improving already existing features. That makes me feel hopeful that stability and bug fixes are the top priority this year. I mean... What's the point of adding a whole bunch of shiny new features when prexisting ones can be improved and perhaps long-standing bugs will be fixed? For people who are underwhelmed or say they expected more, I'm genuinely curious what else you think needs to be added at this point.

I can think of four ways to generate AI image descriptions off the top of my head. Three of them are completely free. I don't personally need another.
You know what would be great though? Knowing that any software Apple develops will be just as usable to me as anyone else.
A main focus of the accessibility team seems to be describing visual information, both from third-party apps and the environment. That's a great thing, but the priority should be making the mainstream software work better for people with disabilities. No, that's not glamorous. But neither is QA testing, and people would be justifiably upset if the QA team decided to stop testing Apple software and start developing an app that can perform QA testing on any object within the camera viewfinder. I'm not saying that's a bad idea. It might save me from purchasing a faulty air frier someday. but they have an important job that should take priority.
I hope these updates are just the flashier ones, with lots of under-the-hood improvements that weren't worth explaining to the public just yet.

Well some love shining bobbles and they easily can distractive by them. Them they worship at the apple alter and when you point it out, they call you negative and send you to the principle and he send you and email to behave yourself. Long live cats. PS. Probably get one today or tomorrow.

If the descriptions and followup questions can be directly generated from a photo in the Photos app, particularly while using the editing tool, it would help me a lot. It's exhausting to constantly go through the share sheet to get a description for each change. It doesn't replace the haptic exploration of images in Seeing AI, but that is usually for final adjustments, in my usage.
I'll have to upgrade phones, but I need to do that anyway.

Looks like I'm getting the thing I wanted most, native AI image descriptions. Of course the video Apple made was a good demo, but it makes them sound awesome!

I wonder how this Apple Intelligence integration into image explorer will compare to third party apps like Be My AI, Seeing AI, and probably a few more that I haven't used. I wonder if it will completely replace them for some people. The workflow sounds a heck of a lot easier than trying to send an image to another app via the share sheet, especially for ones on websites. I can't always get the share sheet to come up.

I wish this image explorer integration thing would let you choose the app you use, so you could use the on-device Apple models, or the apps we all already use without having to go through the share sheet.

For people who use native AI image descriptions on Android, what's the workflow like? I'm thinking with the new descriptions in iOS 27, you'll just flick down until you get to image explorer, and maybe there's a button there to generate a description, or maybe it automatically does it with a VO setting. Do you have to pick a service to share an image to on Android like you currently do on iOS?

I'm thinking that for the new stuff Apple announced today, they focused more on other disabilities, and I'm fine with that. Last year Braille was a huge deal. That wheelchair eye tracking thing for the AVP sounds like it would take a lot to make work, though I wonder how practical it would be.

I do wonder, if people who use the features Apple develops for people with other disabilities -- like Voice Control -- complain that they have bugs that have sat unresolved for years. Or is it just us? Or maybe we just see it more because we're in our own little bubble?

We strongly wrote mails send videos to Apple to fix Auto language detection for South Indian languages across the platforms but they are not caring but I loved the live recognition features

On android the options are in the talkback menu or can be assigned to gestures

If my comments above came across as overly confrontational, I apologize. It was not my intention to say that this entire update is going to be awful, and honestly, I don't think that in the first place.
Every iPhone user is different. I rarely need, or even want, onscreen images described. What I need most, and what the accessibility team used to do exceptionally well at maintaining, is a relatively smooth and reliable experience across core iOS apps.
These image-related features might have been requested by others in the past, and I am glad they are getting what they asked for, even if I think it should have come from another source. But not everyone wants or needs the same things. I, for example, am hoping the Notes editing toolbar can be decluttered. That does not mean I think VoiceOver is terrible. It doesn't mean I think the image-related features are inherently a bad idea. It doesn't even mean I'm not curious to try the new update. But it does mean that I feel frustrated. To me, it's a little like seeing a university campus get a brand new dormitory building while their science center is quietly falling apart.
I hope that comment clarifies things. Also, and this is unrelated, but now I'm curious... Does iOS even grant app developers enough system access for a third-party screen reader to work? If so, that's surprising, but I'd love to experiment.

I think a lot of the new features that we’re getting for iOS 27 won’t even matter because from me the devices I have don’t even have Apple Intelligence capabilities. I have a regular iPhone 15 and an iPhone SE third generation. So it’s really not gonna be much an update for the older devices. I’m mean we don’t know this yet, but I kind of wonder if those devices will even still be supported in the fall. I hope they are because I really don’t think I can go getting any new phones right now. So it seems like people with older devices are really not gonna be getting a whole lot in the flow. I mean, I could be wrong but that’s what I’m thinking at this point.

I'm not demotivating android users but Apple has far good image description feature than android you can also assign gestures

We’ll just have to see what happens. I hope that my phones will still continue to be supported.

If it does not have the RAM it may not. Have 16 pro max. Just want to be able to read my notifications. I am not asking for world peace. Long live cats.

I am really appreciating these new accessibility features, although they are a drain on the battery. I reported 2 issues to apple accessibility by mail. One of them is to do with screen recording when VO is running, the VO voice gets into the in ear speaker when doing this and the second one is to do with recording a video, where VO voice is captured in the video. I am awaiting a revert and a fix for these issues it would really help if they can get them addressed as I do a lot of recordings for demonstrating to sighted engineers in my team to show them about accessibility for applications on the iPhone

First, nope, IOS does not allow to build your own screenreader. It's easy with android though.
Second, I think the problem of most people here is pretty understandable in that they want bug fixes above everything else. Bugs are a massive problem on IOS.
3. On device immage description are a bad Idea because of the battery drain alone, let's just not take into account the huge amount of storage they consume.

Wake me up when quality control is the one feature they're really committing time and energy to. PR-generating nice-to-haves, with perhaps the arguable exception of AI description (still less important on macOS than on iOS, for obvious reasons) really aren't worth writing home about, at this stage.

@Chris Indeed, they brought us screen sharing between Tahoe Macs, which is surely a rarely-used feature in our community, albeit one that I appreciate very much as a Mac-server person, but failed to bring us working support for iPhone mirroring or remote control over FaceTime between two iPhones. Mysterious are the ways of the almighty Apple!

@Holger What? Why don't you want peace in the world? You are clearly a very bad person and I am disappointed to find that all you really need are a few longstanding bugs to get fixed. Shame on you. :)

Yes, I am. I should go to church and ask forgiveness. If Bella was here, she would had punish me. World peace will never happen, apple fixing bugs? Perhaps before I go to heaven? Long live cats.

Sorry Holger, I'm afraid you're doomed; there's a heavy price for desiring functional Apple products. And Bella, well she's clearly in on it too; they're Satan's little helpers, are cats.

@Brian There's this scoop from Bloomberg's Mark Gurman suggesting you can pick your AI model in iOS 27 for OS features. Maybe you're right, and you'll get your wish to have integrated features and a choice of offline and online models, and that would def be very powerful. Article here:
https://www.bloomberg.com/news/articles/2026-05-05/ios-27-features-apple-plans-to-let-users-swap-models-across-apple-intelligence

OK. I will wait for apple and iOS 27 and not complain and behave. After that, I am getting a lawyer to look into accessibility and apple. LLC.

The improvements to visual descriptions are certainly very welcome. I believe this runs locally hence needing Apple Intelligence. The other day I got the description "an elephant and a calf" when I'm pretty sure the photo was of my dogs, unless mrs grieves had been keeping something from me. So if this gives me a big improvement here that would be a big selling point for me. The live recognition thing, well I'll wait and see, but if this is a precursor to this kind of function being available on Apple Glasses then that's great.

Nothing else for me here. I thought there was already some kind of automatic subtitling across the system, so I'm not sure what this does differently. I'm guessing it is moving towards what I thought it could already do. At any rate it is a really important feature for those that need it. I've been trying to convince my parents to play with this sort of thing for ages but they never seem to.

The Vision Pro features are pretty great if you need them. Apple likes to bring out a big surprising feature like this for an audience you didn't think they would necessarily be catering to. This could make a huge difference to someone's life.

As always with these things, though, it's the little things I care about rather than the big bang things. I want my day to day to be improved and this means bug fixes, as opposed to feature I may not use much. If They are never going to announce that here, but that would make me happy.

If the visual description thing is good, though, it may be time to upgrade my iPhone 13.

As much as I love Apple, I dislike the fact that we are required to purchase only products that are approved by the company. Often, third-party accessories deliver disappointing performance. For example, I recently bought a USB-C to headphone jack converter for my iPhone 17 Pro Max from Amazon, which had excellent reviews. However, when I plugged in my headphones, although they worked, the sound quality was neither clear nor crisp when I was speaking to my partner on the phone.

By comparison, when I purchased the official converter from Apple, my EarPods worked perfectly and seamlessly. This experience highlights my frustration: if Apple does not improve any VoiceOver features or fix existing bugs, I might consider switching to Android and would bid farewell to Apple for good. I really dislike being constrained by such a restrictive policy, where I am unable to use third-party products with my phone without risking poor performance. I believe customers should have the freedom to choose accessories that best suit their needs, without being penalised for not buying official products.

I am curious if this more about the product and not apple? If apple is doing something to affect the third party product, is that legal? Long live cats.

I bought 3 lightning to jack adapter from apple for $30, only last 3 years 1 per year.
Got one from ugreen and it's rock solid for twice the price and it's been almost 3 years too and it's working as good as when it was brand new.

I’ll be genuinely happy to see meaningful improvements or truly useful new features in Apple products. What some people get overly excited about are often just global system features that happen to be exposed through Accessibility, not features specifically designed to improve accessibility itself.
What would actually make me happy is if Apple seriously reconsidered the state of VoiceOver on the Mac and fixed long-standing fundamental issues, especially basic things like Quick Nav and reliable text navigation. Those are core usability problems, not edge cases.
I’m also not the kind of docile customer who applauds everything Apple does as if it’s automatically revolutionary. Real improvements deserve praise; marketing hype alone does not.

I agree, it's a shame we don't have VoiceOver accessible remote control between iOS and iPadOS devices. I'd rather have that than image description features I'll barely use.
As for device compatibility, we already know macOS 27 will require an M1 chip or later. The rest of the devices are harder to predict, though if I had to take a stab at it, I'd say anything with the A12 or A13 chips won't get 27 with the possible exception of the iPad 9 since it's still fairly new. This means the minimum devices may include iPhone 12, iPhone SE 3, iPad 9, iPad Mini 6, the iPad Pros with M1, the Apple TV from 2017, and possibly the Apple Watch 9. We'll get the definitive list on June 8.
Don't worry about your device not supporting Apple Intelligence, at least not for very long. I predict it won't be an issue in 3 or so years when they finally drop everything that isn't compatible. We've been through this with Siri and many other things, all in the name of marketing and forcing people to upgrade their hardware. I suppose there are some unavoidable hardware requirements for some capabilities, but certainly not for things like Chat GPT which reaches out to a server.

They come late, but they play to win.
While I am sure that I am going to be a heavy user of all these visual descriptions and live descriptions features, I was hoping for built-in video description as well. Afterall, if Seeing AI can do it, the built-in video player on IOS should be able to do it.

Maybe Apple is saving video description for next year, so they still have something interesting to announce. After all, they do need at least one shiny new feature each year to keep things moving.

When I first started reading about what’s coming, I thought, “Okay, this sounds interesting.” Then I started to feel like a lot of it was more about catching up than breaking new ground.

That’s not to say I don’t want Apple to surprise us. I’d love to see them come out of left field with something like they did with Eloquence. Unfortunately, my phone is too old to take advantage of Apple Intelligence, or what Apple considers Apple Intelligence.

The only reason I would get a new phone right now is because I need a new phone, not because of a shiny new feature. Apple hasn’t really impressed me in that way since the M1 chip was introduced.

Vision Pro sounded interesting, sure, but it is overpriced for what it can currently do. If it eventually came in a much smaller form factor, like a pair of glasses, maybe I’d feel differently. Even then, a lot of the visual aspects of Vision Pro are not really designed for someone who has had no concept of sight whatsoever.

OK, How old is voiceover? Asking!! Have there been a major update to it? Long live cats.

I don't think that it is illegal, but why the companies should restrict us to use the 3rd party's products, if they are better. for example: any headphones would work great on androide, unless it is the airpods.

Most headset or similar to airpods have an app that you need to download and set. Most work well. I had listened to Flossy Carter when he demonstrates headset or airpods for android and apple. If using iPhone, he mentions the app that needs to go with it. Long live cats.

Below are my thoughts on the announcements, in no particular order:

• It is my understanding that all of the image descriptions from both Image Explorer and Live Recognition will be done entirely on-device.
• I have high hopes for the enhanced descriptions from Image Explorer, though my optimism is cautious. I think Apple has a tremendous opportunity to make things simpler here where some friction exists currently. Right now, getting an image described requires sharing it to a third-party service, and if image descriptions can be provided on a level with Be My AI or PiccyBot (the two services that I have the most familiarity with), that would be more than good enough for me and I would have much less need for third-party apps. The nuance is in the details, of course. How in-depth will image descriptions be? Will users be able to ask follow-up questions about described images, or just about Live Recognition? Will Image Explorer be able to describe whole screens, for example a radar screen in a weather app? If Apple-generated image descriptions are on par with what third-party options offer, this will be huge.
• The improved Live Recognition sounds very cool. If the feature works in real-world conditions the way it was shown in the video, that would be a great initial release and would definitely reduce reliance on third-party description apps for me.
• In the coming years, I think the next big thing we will see from Apple for blindness accessibility is on-device video descriptions generated with Apple Intelligence. I don't know what all is involved with describing a video using AI, and what resources this requires. But my best guess is that this will be the next big innovation in description and that when it comes, descriptions will be generated entirely on-device. I do not see Apple settling for an internet-based solution for this and if they do, I see it only being Apple's custom integration with whichever third-party LLM they are partnering with at the time.

If JAWS was able to do it, apple should do it better. Picture smart is good and does a nice job of describing the picture or image. I use it in facebook and does a good job. Long live cats.

VoiceOver on iPhone has been around since 2009.
What would you consider to be a major update? It has received incremental updates every year since then, and is now a very feature rich screen reader I would argue.
I wouldn’t disagree with you that it needs some clean up in terms of bugs, focus jumping being the biggie for me, but not sure if that’s what you mean in your question.
VoiceOver on Mac however I feel does need a major overhaul.
Dave

These are very exciting features. I can't wait for these features to come. The most important one I think will be useful is AI Generated Image Descriptions.

Thank you Apple. Happy GAAD to everyone.

Major overhold. Adding news features and making sure VO works for it. Yearly update to the VO. Every yeaar apple releases new iOS and updates app yet VO does not get one. NVDA, JAWS get updates every 3 months due to how the OS in windows get updated. Expecting VO to continue to work when the OS get updated is not realistic. This is not being negative. Anyone knows that if an OS or a system gets updated and their apps also, the overall system need to do so. If VO came out in 2009 3 more years it will be 20 years old. What do we need to wait to see if apple is planning something for VO in 2029? Long live cats.

I totally get what you’re saying. If we got VoiceOver on iPhones in 2009 then yeah it is almost 20 years old. But maybe the updates have been coming with in the features themselves. I’m thinking that maybe as there were new accessibilities that came for new features that voiceover got some of those, but I’m not totally sure. We’ll just have to see what happens with iOS 27. And yeah that’s totally not negative. That’s just a really good point.

My personal thoughts on the new features is that those of us without a max phone or better won't get them. I have seen several voice over bugs, particularly the one where when you try and double tap to answer a call the call ends instead. There is also one where when you try and unlock your phone with a bluetooth keyboard, you can't enter the password. Bugs need to be fixed, and there needs to be stability. But as many have said in this thread, we shall see.