New AI app for describing images and video: PiccyBot

This App is great for having videos described.

Hello Martijn!
I don't know if it would be possible, but would it be possible to describe what is happening at the exact moment when processing the video, like an audio description? Almost like seeing ai, but without the pause to describe.
Of course, it would have to give less information, but I think it would be amazing if it could do that. Reading the text or listening while the video is playing like this ends up causing synchronization.

Hello Martijn!
I noticed that the 2.14 version updated today mentions the ability to describe YouTube videos. This is truly a great upgrade. However, I haven't found a way to use this feature. I can't find PiccyBot in the sharing menu of YouTube videos, and copying and pasting the YouTube video link into the APP doesn't work either. Can you explain how to use this feature?
Many thanks

Hello Martin,
I discovered in the 2.14 what's new that some shortcuts were also introduced. This is a great joy. But please give us some details, what phrases they are exactly.

Hello Martin,
Thanks much for the info. I tried out all the shortcuts you had mentioned. Siri didn't reject any of them, but opened the main interface of Piccybot in reply to all of them instead of the camera or video recorder part. I note that I run an iOS older than 18 on purpose.
Thanks much for the language selection fix in 2.14. Now my native tongue, Hungarian can be found and selected in the language list, and not just by setting language to system language.
While browsing on Huggingface I found the following very promising uncensored model:
https://huggingface.co/huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated
This is the uncensored version of a very recent Qwen 2.5 vision model (the base model is developed by AliBaba group). This is the 7-billion-parameter variant, so computationally it is on par with the Janus Pro model that you currently run on your server. According to descriptions this is a quite versatile and strong model even in this size and multilingual too. It can process images in their native resolution and can be finetuned if needed. It can even process videos (even long ones) if the inference is done accordingly.
Personally I found nothing spectacular or special about the Janus Pro model, and this one is uncensored, so if feasible I propose to run this one besides, or even instead of Janus Pro on your server. Thanks in advance for considering this.

Hello Martin,
Thank you very much for your answer. I tried it just now and successfully selected PiccyBot from the share menu on YouTube. However, it's strange that when I posted a question to you a couple of days ago, I used the same method but it didn't work. Anyway, it's working now and it's functioning well.

deepseek r1 is wonderful, but my first impression of janus a while ago was oh well. it seems to have improved very significantly, now it gives a comprehensive description of the photo. Martijn must has done some good work about it.

Hello guys!
I don't know if I can report bugs in the Android version in this thread. I'm doing this since the developer looks here a lot, so it's easier for us to get support.
If necessary, I'll send it somewhere else.
I'm facing a bug where the description audio isn't being played.

I've been using this App for a couple months now and like it, but have noticed that it very often fails to process Youtube videos, either giving me the generic, "server error," message or just failing to process but then a retry will sometimes work, sometimes not. I do pay for the subscription. I have tried with multiple videos and multiple models. I tried running one of the same links through Gemini Flash to see if it was an issue with the model, but it gave me a very good description of the video broken up into time stamped segments, so the requests must not be going through the models directly which seems to make the App unreliable at the very least. I've read through all the comments here and haven't seen much about this, so either I have the misfortune of trying at bad times or it just isn't a widespread issue.

Hi Martijn,
Thank you for continuously adding new features to the app. As you mentioned in the post above, the app now has many features, especially providing many models for users to choose from. Could you please explain the advantages and disadvantages of each large model when describing pictures and videos? Of course, I know this question sounds a bit subjective, and perhaps everyone's opinion will be different, but I would like to hear your opinion, and I think there will be other friends who, like me, would like to have a reference answer.

Can you make the app guide the user so that they can take pictures with it?

Do you mean like Google Pixel's "Guided Frame"? That, would be sweet! 😃👍

I've been wanting something like that for ever now. But I guess you'd need live AI, I mean truly live AI for that?

Hello Martijn,
I found another bug in an app. I usually use Simplified Chinese, and I noticed that no matter which voice I choose, I often encounter situations where the content of the image description cannot be read out. The specific manifestation is that they will keep saying "Chinese Letter Chinese Letter Chinese Letter" until I have to pause the voice for it to stop. This is really frustrating because it happens frequently, about once for every five images I recognize this bug occurs. Could you find some time to check what's going on with this?

I noticed that too using the Chinese languages. Quite often the description gets stuck and starts repeating itself like a broken CD LOL

@Martijn if you replicate Guided frames or in other words make an accessible camera app for IOS, you would be doing a path-breaking, pioneering service for IOS users with visual impairment, not to mention facilitating their inclusion in mainstreme in a certain huge way. It need not be a feature within piccibot, rather it can be a stand-alone app, which can help us frame and take decent pictures and save to galary. I would be more than willing to contribute to it in any way I can.

I think Llama 4 has the fastest processing speed among these models, but its description effect is not as detailed as Gemini Pro2_5. However, considering both processing speed and description quality, Llama 4 is also a good choice. In my ranking of these models, it's Gemini Pro > Llama 4 > Deepseek.

Interesting addition, thanks. Completely understand why it's image only, video would both be more complicated, require more resources from the AI servers and you need to confirm how well it works before you even consider it.

Especially for situations where what you want is an accurate description and where halusinations can be problematic and/or dangerous.

I think, if you have truly found a way to remove these, you will become everybody’s new best friend.

This is something I've hoped would come along now when we have many good models that can be compared. Seems to work fine, but hard to say if it does not hallucinate of course. A possible further development: having a "fast" and a "thorough/detailed" mix, with modeels tailored to each?
Thank you very much for this!

Hi, I have downloaded the app but after using it for a while, I deleted it from my iPhone. I have Be My Eyes and Seeing AI so I feel like this app isn’t useful

No app will be useful to everyone depending on their situation and needs, that's perfectly valid. Just bear it in mind as a potential tool in the future if you need a short video described, or if you encounter a situation where the AI models in Seeing AI or Be My AI aren't helpful.

Hi there,

I know this might be a bit of an unusual request, but I wanted to put it out there anyway—because if you don’t ask, you’ll never know what’s possible.

Lately, I’ve been using a few survey websites to earn gift cards like Amazon gift or prepaid Visa cards. These help me make online purchases without needing to use my debit or credit card. One of the main platforms I use is powered by Spectrum Surveys, and while the surveys themselves are accessible most of the time, there’s one issue I keep running into: drag-and-drop tasks.

These questions usually ask me to move items—like dragging an image or word into a specific box or column—but there’s no reliable feedback about what I’m dragging or where I’m dropping it. This makes it nearly impossible to complete certain sections.

Here’s where the idea comes in. I was wondering if it would be possible to create a feature that allows me to share my screen and receive descriptive feedback about what’s happening. For example, if I’m dragging “Apple” into a “Fruit” box, the system could announce something like:
“Dragging Apple. Drop zone: Row 1, Column 1 – Fruit.” And once I release it: “Apple dropped in Fruit.” If I switch to dragging “Blue” into a colors section, it could say: “Dragging Blue. Drop zone: Row 1, Column 2 – Colors.” Then confirm: “Blue dropped in Colors.”

It doesn’t need to control anything for me—I’m not expecting the AI to do the action itself. I’d simply like to hear clear, accurate feedback so I know what I’m doing on screen. This could work in combination with screen recognition when standard reading fails, which is something I already use regularly.

I think having real-time feedback on selections and drop targets would significantly improve my experience and make these survey platforms far more accessible. This kind of assistance wouldn’t fix every challenge, but it would eliminate one of the biggest barriers I run into.

Also, I plan on purchasing the lifetime subscription soon—probably by Friday. I have a $25 prepaid Visa card ready, and I believe the lifetime cost is around $20. Regardless of whether this feature is possible now or in the future, I’m fully committed to supporting the app. I figured I’d share this idea in case it sparks anything or aligns with something already in development.

If this kind of feature is technically possible, it would be a game-changer for me. And if others feel the same way, hopefully they’ll chime in and add their thoughts as well.

Thanks for your time and for building something that’s already made such a difference. I’m looking forward to what comes next.

I like to keep some prompt histories so I don't have to retype them all the time, or go copy them and paste them in from elsewhere. I have noticed the latest version, the APP keeps randomly refreshing as if I closed it and re-opened it, and that keeps erasing my prompt history. Is it those random tips that's doing that?

I don't know if you've noticed, but the app has been having some issues lately. First, no matter which voice I set for text-to-speech, it won't read aloud anymore. I have to use VoiceOver to browse the results instead of hearing the app automatically read them out as before. Second, the app is increasingly failing to recognize images. In more and more cases, I upload a picture, wait through a long sound effect, and end up with nothing on the screen. It's a bit frustrating. Still, I'm willing to support it until these issues are fixed by the developer.

Hi,
I just downloaded this app yesterday, to get extract all text from screenshots of recipes I find on Facebook.
The descriptions are great, even with the free version.
I'll definately be supporting the developer, when I'm able, by purchasing a subscription or maybe even the lifetime option.

Keep up the incredible work! :)

I’ve actually been talking to the team about this recently. They responded with something like, “Thanks for bringing it to our attention,” and I suggested they really should reach out to others to see if the issue is more widespread. For me personally, it’s not so much an issue with images—it’s more with videos. I usually use it to describe music videos. What happens is, I’ll get one video described just fine, but when I try to run a second one right after, it stops working. It gets stuck on one of those loading screens—I’ll hear the waiting sound, or it’ll say “please wait” or “fetching data,” but then nothing actually happens. It’s pretty frustrating. The only workaround I’ve found so far is to screen record the video and then have that recording described. But obviously, that’s a hassle. So clearly the issue isn’t the video itself, because the screen-recorded version works fine. It might have something to do with how Apple’s files are handled, I don’t know. The team did acknowledge that there’s something going on and said they’re looking into it. At first, I thought it was just me, but apparently not—there does seem to be a real issue here. The pattern’s pretty consistent: one video gets described, then trying another one right after just doesn’t work. I get that this probably takes up a ton of processing power, so maybe there needs to be a buffer or delay, but even when I wait like 10 minutes—or even an hour—nothing happens. And when I say “nothing happens,” I don’t mean it’s sitting there thinking. I mean the video doesn’t even get recognized. If it doesn’t work within the first few minutes, I usually give up because I know it’s not going to load at all. So yeah, that’s definitely something that needs to be addressed. Overall, the app is great and super useful, but this video issue is a real limitation right now.

As a voice actor, I hate AI audio description. AI voices in general make me mad because not only do they just not usually sound very good, but they are a crutch that takes away from my vocation in the worst way.

As an audio editor, I utterly loathe how audio ducking is used with audio description. It is almost never ever justified in any way and takes away from music and sound as someone said. I will die on that hill speaking about this practice. Don't do it.

But as an avid gamer and lover of AI , I'm all for this. I'd rather have AI than no audio description. And if AI can describe in real time and be customized, that would be really cool. Right now I really want to have game cutscenes audio described. There still isn't a practical way to do this. I thought Google Gemini was on the verge, but not yet. ANd Pixie Bot is pretty interesting, even if I have to essentially watch the cutscene twice or three times. It too is not all that practical because I ahve to record the video with my phone, or upload it to YouTube first. I love some of the customization suggestions listed above. I Think AI has a lot of potential for audio description. I also love the idea of pausing a scene and getting more in-depth description. So much audio description is surface level at best. It gives an overview, but sometimes a bit more dept would be nice.

OK so I have another couple of ideas—I don’t know if anybody else actually wants this personally, but I figured I’d just throw it out there in case it’s something that can be done. And I really hope this doesn’t come across as annoying or anything because I feel like I’m asking for a feature that maybe nobody else would use or even think about. But hey, maybe by putting it out there, something happens, right? So first off—I’m really excited for the new update. I’m currently on the one-month plan and I do have the money for the lifetime subscription, I’m holding off a little bit until some of the current issues get sorted. But yeah, as it stands now, I’m honestly very satisfied with what I’ve got. One thing I saw mentioned was that there are going to be some new voices—which I’m genuinely excited about! So here’s my slightly selfish ask: would it be possible to somehow sign into Amazon Kindle and use one of these voices to read Kindle books directly? I know some text-to-speech apps let you do this, and I’m not sure if it would clash with your current goals for the app or if it’s something that can even be implemented, but I figured it couldn’t hurt to ask.

Basically, I’d love a feature where I can sign into my Kindle account and have one of your voices read the books in the library. The voices are really good—I hope they keep getting better. Honestly, one of the best I’ve heard so far in the TTS space is Speechify. I don’t know where they get their voices, but the quality is top-tier. It's pretty expensive though. Seriously, $140 per year? I don't know if it's because it's promoted by MrBeast, Gwyneth Paltrow, and Snoop Dogg, but, yeah. Must be for the rich. They seem to be promoting as a product for individuals with dyslexia, but again, super expensive. I’d even love the option to create virtual voices. I’ve also seen features in other apps where you can turn text or books into podcasts using virtual voices—kind of like generating a little audio book from your reading material. That would be amazing. Again, not trying to step on any audiobook publishers’ toes here—that’s not my intention. Like, imagine clicking on your Kindle book (once linked), choosing one of the voices, and having it read to you right in the app—no downloads, just direct reading. And maybe with uploaded files or personal documents, there could be more freedom. That’d be next-level accessibility.

Anyway, I know this might be a long shot or not a priority, but thought I’d throw it out there. Thanks again for listening and for continuing to improve the app—it really does make a difference.

Could anyone remind me how getting youtube videos described works? I tried sharing it from the youtube app to picciebot and it just says it isn't valid no matter what video I try using.