Now that the Gemini Live feature with the ability to use the camera and share one's screen is available, I thought I would try out some of the new Gemini widgets that are available.
One widget I just added to my locked screen was the Gemini widget to use the camera.
Now, in the Gemini app itself, when you use the camera with Live mode, you have to keep asking what the camera sees. Gemini doesn't keep the camera view going as you move the phone. But, as you move the move the phone, you can keep asking what the camera is seeing in the new view.
With the Gemini camera widget, however, as soon as you activate the widget, Gemini immediately starts speaking everything it sees through the camera, and continues to update what it is seeing as you move the camera.
So, for example, as I held the phone out in front of me and w walked through the house, Gemini gave me running feedback as I passed each door, tables and chairs, opened up a kitchen cabinet to view some cans, etc. This is very different from how the Live camera function works in the app itself.
The downside of using the Gemini camera widget is that, unlike when using the Gemini app itself, one cannot interrupt the AI speaking and ask questions of Gemini. Gemini justs keeps telling you what it is seeing until you quit. For example, when I opened up my kitchen cabinet and heard that it contained many canned items, I couldn't ask what was in the cans.
In summary, there are pros and cons to each approach. I just wanted to make people aware that some of these widgets can be very useful in different ways than using the app itself.
Beside the Gemini camera widget, there are also several other Gemini widgets available for automatically viewing a photo in one's photo gallery, uploading a document, starting an interactive voice session without having to click through a number of buttons in the app, quickly starting a text chat, etc.
I'm curious to see what use cases people try and experiment with using these new widgets on the locked or home screen and how useful people find them.
Happy testing with these new capabilities!
--Pete
Comments
Sounds a little like seeing AI
Sounds a little like seeing AI, when you first open the app, with regards to the camera widget for Gemini. Perhaps they will expand on it with future updates. :-)
how does one add these vidgets?
Yes, the subject says it all...
Adding Widgets
This took me a bit of experimenting to do, but here is what should work:
1. Perform a one finger double tab and hold on the lock screen. This brings up a list of options.
2. Activate the "Customize" option near or at the bottom of the list of options This brings up more options.
3. Activate the Locked Screen or Home Screen item in the list that pops up depending on where you want to add the widget. This seems to put two windows on the screen.
a. One window near the top has the list of widgets you already have active.
b. The second window is near the bottom of the screen and contains the list of widgets that you can add. Note: I couldn't flick through the items in the top window and get to the items in the bottom window. To get to the items in the bottom window, just move your finger around near the bottom of the screen.
4. navigate through the list of widgets/apps that you can add and activate the desired widget.
5. Finally hit the "done" button at the top right of the screen.
Note: As you'll see, there are also options to delete each of the widgets you already have active. Also, if you want to reorder widgets, I believe you start by doing a single finger double tap and hold, but haven't tried that in a while.
Hope that helps. The process can be a little tricky with Voiceover.
--Pete
Protip
If you are adding widgets from within iOS, that is to say that you are not doing so from within an application's gui, then please note that some widgets have a slider that allows for larger widgets and/or alternative information.
The News widget is a good example of this.
HTH. 🙂
So, it's a single widget you…
So, it's a single widget you add which has several 'sub-widgets'.
I couldn't find the one that engages the live voice camera.
Was it the standard google camera app that simply says what it sees in vague detail using voiceover's voice rather than the live feature?
It's the phone camera
Hello Peter!
This option activates the regular camera, but through the Gemini app.
What controls the functionality is the VoiceOver.
Is this what you mean? Using the camera within applications is possible, and when preparing to take pictures, VoiceOver recognizes many elements.
This is just the regular camera
if you focus the view finder in the camera app, you should get the same thing, or you could always use voiceover recognition if you wanna go that route so you can have more than one feature on at once.
Re: Just the regular camera
Yes, as others have pointed out, adding the Gemini Camera widget isn't actually using Gemini to describe what is seen through the camera. My initial impression was that adding this widget would automatically put me in the Gemini Live mode using the camera. But, as stated, this is using the usual camera viewfinder with descriptions from Voiceover.
So I removed the Gemini camera widget since I can get the same functionality by activating the Apple camera widget on the home screen, and replace the Gemini camera widget with the Gemini Live widget. Now, when activating the Gemini Live widget, I can have an interactive conversation with Gemini from the home screen. In addition, when the Gemini Live widget is activated, there are buttons on the screen for sharing either the camera or screen. So this widget is pretty handy.
As I previously mentioned, there are also separate Gemini widgets one can add for sharing documents, opening chat mode, etc. I haven't experimented with those yet.
--Pete