Hello guys,
I have created the free app PiccyBot that speaks out the description of the photo/image you give it. And you can then ask detailed questions about it.
I have adjusted the app to make it as low vision friendly as I could, but I would love to receive feedback on how to improve it further!
The App Store link can be found here:
https://apps.apple.com/us/app/piccybot/id6476859317
I am really hoping it will be of use to some. I have earlier created the app 'Talking Goggles' which was well received by the low vision community, but PiccyBot is a lot more powerful and hopefully useful!
Thanks and best regards,
Martijn van der Spek
Comments
The app isn't working for me
I downloaded this app today, but I'm having some trouble sharing images from other apps to get descriptions with PiccyBot.
I was able to get descriptions of an image I shared from my camera roll and also one from WhatsApp, but when I shared one from Safari and another from Discord, the app continued to play the loading sound even after a whole minute had elapsed. When I tried to share a couple of images from the Dystopia app for Reddit, I was taken to a screen with no elements on it, and nothing further happened, even after waiting a whole minute.
I'd have no problem paying the subscription to help cover the development and operational costs of the app, but it doesn't seem to be able to meet my use case.
Am I doing something wrong? Why is the app able to only receive images shared with it from certain apps? This is not an issue with other image description apps.
TJT 2001 - Sharing from apps
Correct, the app can describe videos and images from your phone library, Whatsapp, and Messenger. I am still working on supporting more apps. But for now, either open PiccyBot and select the media from there, or save it from the app to your library and share it to PiccyBot from there.
Can you confirm which image description app does this properly directly? I can then study their method and hopefully implement it in PiccyBot as well.
The other apps
Thanks for your prompt reply. I didn't realise that there were differences in how image recognition apps received data from other apps.
I'm able to get descriptions from the apps I mentioned using Seeing AI and the Be My AI feature in Be My Eyes.
It makes stuff up!
I was on holiday in July and sent it a montage of photos put together into an MP4 video to describe. it did pretty well, but then started talking about 'we can hear the wind whispering through the leaves of the trees and the sound of the waves'. What nonsense! The video was silent i.e. no audio! Not that it matters enormously, but to paraphrase and channel the spirit of cricket commentator Fazeer Mohammed: Why did it do that? Unbelievable! I do not have the personality thing turned on, so that doesn't explain it.
Great app though. rEally fantastic.
How are people doing now getting it to describe the whole of a video? I still find that a bit hit and miss.
last question inadvertently get carried to the next image
example, i was discussing about baroque furniture and specifically talking about the fitness of the grand piano in the photo. then i uploaded a timelapse video about a city street. the ai explains it is a city scene and there is no piano in sight. this behavior started in the latest version, and it happens 100% of the time (that is if you do ask a question and you then send a new photo or video).
Question carry over
LaBoheme, thanks for reporting this. Checking it out. I suspect it is a side effect of the chat mode, which normally only deals with questioning a single image or video. Should be a minor fix, hopefully backend only.
Question carry over: further feedback
LaBoheme, I checked the question carry over issue. It is a result of the feature to start with a specific question. It will then continue with that same question unless you clear it. I could clear it automatically, but I can imagine you have a specific question like 'Is there a house in the picture' while going through a number of images one by one. If I clear in-between that would not be practical anymore.
So right now, if you start 'blank' PiccyBot will give you a general description of the image or video. If you enter a question, PiccyBot will then continue to use this question for any further images or videos until you clear it or edit it. Separately, you can go into chat mode and ask specific follow up questions on the same image or video.
If anyone has any suggestions for a better approach, please let me know?
Excellent app, but I have a small suggestion.
It would be nice if we could upload our picture along with our question at the same time, to avoid the full description and go straight to the important stuff. I once tried uploading a question alongside the picture at the same time, but the app ignored my question and gave me the full explanation. It seems like you can only really ask questions once the picture is already uploaded.
Suggestion for question
My suggestion would be a change of workflow. Instead of having the question box on the page before you chose an image/video, put a screen (or use an existing one) before you send the image/video to the model (the first screen would be empty except for the buttons). There you have the the question box, the history, etc.. When you have gotten your description, and the ability to chat and so on, and you choose another image/video, have the question box pre-filled with the question that was asked last, so that the flow will be speedy if you are to ask the same question about multiple images. If an image is shared to the app, put the user on the "question screen".
The only downside I see is that it might be one more button press before you get the description, but in my opinion the better logic outweighs that little inconvenience. As it is now the question box is a bit hard to grasp how it works.
simple solution for the question carry over issue
a clear button to clear the question. right now, one has to tap the text field for the clear button to appear; the clear button should be visible whether the user is editing the text area. that would make life ten times easier.
Batch Video Processing?
I've been enjoying and loving PiccyBot, especially for describing videos, which no other app can do! I wanted more, so I paid for the full version.
Most of the videos that I've been having described are from Meta Ray-Bans, which are shorter clips, less than 3 minutes each. I've been having to either count the videos or remember timestamps of which I just had described, find next to hav it described, and sometimes losing count in all of this.
A feature request would be to select videos for a given day or a set of videos, have first processed and described, then batch background process the others in the day/set, and play the descriptions in sequence.
1. Select videos for a day/set.
2. Process first/describe first.
3. While first is being described, Batch process other videos in set.
4. After first is done being described, play descriptions of other videos in sequence.
Never mind my previous coment
I have used the app quite some time ago, and it appears that many things have since been fixed, so my bad.
Clear button
I like the idea of having a Clear button. This seems to be a common way of handling th the issue of clearing a text edit box in other apps throughout the OS.
--Pete
South Indian languages immage description?
As I am from India, I would like to know or have South Indian languages detection of image when I share to this app, please improve this as well as give feature for PDF reading along with our South Indian languages and it will be better