Using action button to get AI picture descriptions

By Kelsey, 6 December, 2024

Forum
iOS and iPadOS

Hi everyone

I’m keen to create a shortcut which allows me to use the action button to take a photo, and then to automatically share that photo with an app which will give me an AI description of the photo. Ideally I’d love to do this with Be My Eyes but the app doesn’t allow for this in shortcuts yet. So I’m wondering if anyone knows of an app which, via shortcuts, will allow me to do this?

Thanks!

Options

Comments

By Cowboy on Monday, December 9, 2024 - 11:37

I have my action button set to open be my eyes. I just take the picture from there.

By Roberto Perez on Monday, December 9, 2024 - 11:37

I believe you can do that with the Chat GPT app, but you will need to spend some time setting up your personal instructions so that it can become a true visual assistant. You can also use the memory feature to improve the way it describes over time. For example, you can tell it something like: "each time you see a food container like this one, respond by telling me only what can of meal it contain, suchh as ground beefe"".

By Brian on Monday, December 9, 2024 - 11:37

For those of you with an action button, there is a beautiful solution to this. It only takes a press of two buttons, but since those two buttons are side-by-side, it really should not be an issue.
1. Set up the action button to open Be My Eyes.
2. Once Be My Eyes is open, simply point your camera at what you want to take a picture of, and press one of your volume keys.

Voilà, picture is taken, and Be My Eyes is describing it to you.

Enjoy! 😎✌️

By SeasonKing on Monday, December 9, 2024 - 11:37

I was going to exactly suggest this.
However, I think, the response time in Be My AI can be improved so that there's not so much of a lag between it clicking the picture and returning with a description. Some other app called Speekaboo does that quite well, sadly that one doesn't respond to volume key presses.
I am thinking we are at the cusp of Apple introducing a VO gesture/shortcut to enable us to do something similar. Those with a capture button should be on constant lookout, given the visual intelligence feature has just arrived and surely more is to come in that space.
Also, given Apple intelligence runs on 15 Pro and above models, I don't understand why Apple is not allowing me to open visual intelligence via some shortcut, or action button with my 15 Pro Max. Huge disappointment if the feature prooves to be useful.

By Lee on Monday, December 9, 2024 - 11:37

VI is only available on 16 phones. Why not on 15 pro upward who knows. Makes no sense we have Apple AI but not VI.

By That Blind Canuck on Monday, December 9, 2024 - 11:37

I agree on that statement. The only thing I can think of would be hardware related, but surely a simple update could allow us to use the Action button to open the Visual Intelligence feature on a 15 Pro/Pro Max.