Yet another shortcut to make getting descriptions of screenshots easy

Unfortunately, Apple Intelligence is not available in all regions.
Also, I personally prefer the classic interface to voice assistants. And, in practice, asking Siri takes longer: the VoiceOver gesture is performed almost instantly, and then everything happens automatically - there is no need to save the screenshot somewhere, et cetera, et cetera.

Unfortunately, there was a bug in the original version that prevented the model name from being given a default value. The updated Shortcut can be installed from this link.

I also updated the original post, at the same time slightly simplifying the instructions for adding new models, making some fields optional.

PS. Apple has quite specific behavior for Shortcuts when imported onto other devices, for example, all top-level text fields are cleared. Unfortunately, I cannot test the correct behavior of Shortcut when imported to another device. Therefore, I will be glad for pointing out any errors that you find during testing.

I've updated the Shortcut again, sorry for the inconvenience. New link

Original post also updated.

It seams like Import Questions has binded to the incorrect fields, should be fixed now.

I just tried specifying 'Portuguese' as the language and everything seems to work fine -- Shortcut provides the description in the specified language.
Please note that the language must be specified exactly as written (Portuguese) without any accented letters or similar. Shortcuts tries to find the language code in a `dictionary` which keys are language names, so it should match exactly.

As I already stated, there are two main issues:
1) Apple Intelligence is only available in a fairly limited number of regions.
2) Personally, and possibly many others, I don’t really like voice assistants as a way to interact with a smartphone.
The ideal solution would be to integrate image descriptions directly into VO, and considering that VO is already able to successfully read text from images and describe what is happening (the Surrounding Recognition function), it seems that the technical part has already been implemented; it is enough to make it possible to receive descriptions not only from the device’s camera, but also directly from the screen. However, since VoiceOver does not provide an API for writing custom scripts, we are limited to the capabilities provided by Shortcuts app.

You can reconfigure any setting at any time by simply going to the Shortcuts app, finding the CloudVision shortcut, selecting the Edit action in VO Rotor, clicking the Info button at the bottom of the screen, then Import Questions, and finally using the Setup Shortcut button at the bottom of the screen.

That's exactly why I called this feature experimental and disabled it by default. The problem is that the screen curtain must turn itself off while the screenshot is being taken, and it does this if you use physical buttons to take a screenshot, but in the case of a programmatic call to a similar function in the Shortcuts application, the state of the screen curtain is not taken into account in any way. Trying to temporarily disable VoiceOver is essentially a dirty hack. Initially, I tried to do this without any delay at all, but it turned out that by the time VoiceOver tried to turn back on, it had not yet been completely turned off, as a result, the system simply ignored the action to turn it on. I tried adding an one second delay before VoiceOver turned on, which worked fine on my device, but it seems to vary by model. Of course, I could update Shortcut to increase the delay to, say, two seconds, but that seems to negate any benefit of automatically turning off VoiceOver -- performing a gesture to turn off the screen curtain is definitely faster than waiting for the system to allow VoiceOver to re-enable.

The main issue with temporarily turning VoiceOver off feature was that, when invoked by VoiceOver, Shortcut would stop working immediately after VoiceOver itself was disabled. You can get around this by adding an intermediate Shortcut whose only job is to run the main one. However, this approach, due to the need to execute two Shortcuts instead of one, increases the delay in the appearance of the initial menu, but given that you already have to wait for VoiceOver to restart, this may be acceptable. Link to the intermediate Shortcut, just install it next to the main one and reassign the VoiceOver command to run CloudVisionExecutor instead of CloudVision.

The original post has also been updated. Again, I highly recommend using the old Shortcut unless you need to temporarily disable VoiceOver, because while the additional Shortcut will still work, it will have a delay in showing the initial menu.

Thank you for your feedback.
It seems that this is an inevitable consequence of using a chain of two Shortcuts; the Shortcuts app in this case runs the main command instead of VoiceOver, so that disabling the latter does not interrupt the execution of the Shortcut. Apple doesn't provide an API for managing active apps within Shortcut, so we can't automatically switch to a previously opened app. Personally, this method still seems less convenient to me than simply turning off the screen curtain before receiving an image description, however, perhaps for some such compromises will be acceptable.

I've completely removed the ability to temporarily disable VoiceOver because it doesn't work correctly. The updated Shortcut can be found at this link and in the original post.

The main issue is that we have a closed loop: to take a screenshot, you need to turn VoiceOver off; The only way to continue running a Shortcut after turning VoiceOver off is to start a new Shortcut before VoiceOver got disabled, because immediately after the currently executing Shortcut is terminated; any Shortcut launched from another Shortcut will open the Shortcuts app. As far as I know, the Shortcuts capabilities in iOS does not provide a technical way to bypass these restrictions.

I tried re-setting different values ​​several times during the initial setup, but Shortcut continues to work correctly.
Could you please run this Shortcut from the Shortcuts app by simply double tapping on CloudVision, when the error appears there will be a "Show" button, clicking on it will open the Shortcut's editing screen, there will be a focusable VO question mark icon next to the desired action. Write down what action it is located near; you can use the names of the variables declared next to it as a guide. Unfortunately, I don't know how I could test iOS Shortcut more efficiently, since I can't reproduce a scenario similar to importing a Shortcut from a link on my device.

I've updated Shortcut, you can install it from this link, or in the original post.

As an experiment, I added alternative models that could be selected during the initial setup. PiccyBot is also offered by default, however, in addition to it, you can choose Pollinations (uses Open AI models), GPT-5.2 and Grok 4. Please note that if you select GPT-5.2 or Grok 4, you will also need to specify your API key; for other models, leave this field empty.

I took this Model from an AI Contents Describer NVDA Addon. It seams, that in NVDA addon it also doesn't work. I'd even tried to manually make similar request with curl or python script -- it always says that there's no image. Probably this is a model issue.

What about the error you were getting? Did you manage to fix it? If not, try following the steps I described in the comment above, based on those results I can try to figure out the causes of the issue.
PS. With NVDA I'm using CloudVision instead of AI Content Describer, it also works quite well.

Hmmmm, the only thing that happens with the API Key is that it is sent as a header in the HTTP request, something like `Authorization: Bearer <api_key>`. All further responses are generated by the server, i.e. an error occurred on the Open AI side, which is what the message appeared about. In fact, I don't have the GPT-5.2 API Key to test functionality, but the request matches the API in the documentation, so I don't quite understand the source of the problem.

1) Can you please try selecting 'n' for Play Sound? I don’t understand what could be causing the error, but literally the next action that occurs after the point you specified is checking whether the sound needs to be played and playing it.
2) Regarding the API Key: according to information on the Internet, Open AI generates this error if Project API Key is used. In this Shortcut, the call is made to an API endpoint that only supports Legacy unrestricted API Keys.

I've updated the Shortcut, it can be installed from the following link or from the original post.

Pollinations AI restricted it's free usage to only text input, so it also requires an API key.

I don’t think so: first of all, the issue happens even if you choose to simply recognize the text, and this item, in principle, does not use any kind of GPT api. Please try this test version and report about the results. In case of an error, follow the same steps to launch Shortcut from the Shortcuts app and report which action is marked as error.

Regarding the API key: yes, it involves creating a separate account, but there are quite detailed instructions on the Internet on how to do this. I had no experience interacting with Open AI, but getting an API key from Pollinations turned out to be quite simple, you just need a Github account to register there.

PS. What device are you using? iPhone/iPad/mac?

Quite a strange bug, technically the next step is simply checking the play_sound variable to determine whether to play the sound now. Here is another test Shortcut, try it and report the results similar to the previous one.

Unfortunately, I cannot reproduce this problem myself in order to simplify the testing process and immediately suggest a working variant.

It's great that everything works now!

If you have some time, could you please check out this version? The only difference between the previous version and all before it is that I did not use variables of type bool, this seems strange, because it is unlikely that only Shortcuts in iOS 26 support them. To finally verify the hypothesis, in the Shortcut from the link I replaced text with bool, this is the only change relative to the version that works for you. If it turns out that this Shortcut will not work for you either, I will correct the version in the original post by avoiding the use of the bool type. Note that when initially setting up, the value for Play Sound must be exactly either 'y' or 'n' (one lowercase Latin letter), in fact, the value entered there is validated, but in case the check for some reason does not work, the variable from which the condition is later checked may indeed not be initialized.

Based on the latest comments, I conclude that on some devices Shortcut does not work correctly if variables of type bool are used inside. This problem cannot be reproduced for me, so I cannot find out its source in more detail. However, for greater compatibility, I updated Shortcut to remove all use of bool type variables. The fixed version can be installed from this link, or from the original post.

I recommend that everyone who previously experienced an error like “Please choose a parameter for each value in this action” try the new version.

It seems that the previously used default model has been depricated by it's owner. The updated Shortcut uses a different model instead, but it seems to be slower and may shut down altogether at some point. For anyone who is not satisfied with the quality of descriptions with the default model, I recommend using one of the paid models with your own API Key. The updated Shortcut can be installed from this link or from the original post.

I checked it several times, the prompt changes perfectly to your own; during the initial setup, instead of '/default', write any text that you want to use as a prompt. Perhaps some of the variables were damaged during the editing process, try to completely remove Shortcut and install it again by following this link or from the original post.

If the problem persists, open the Shortcuts application, launch Cloudvision, just double tap on its name, the error will not appear in the notification, but as a separate pop-up window, click the "Show" button, the Shortcut editing screen will open, at the place where the error occurred, the exact action will be marked with a question mark, this is a separate element focused by VoiceOver. Report the action next to this question mark, i.e. the one you'll be taken to if you swipe right from that question mark once.

Unfortunately, I have no way to test this Shortcut on macOS. However, I have not limited it to iOS devices in any way, so you are welcome to test it and report your results.