Description of App
Seeing AI is a free app that narrates the world around you. Designed for the blind and low vision community, this ongoing research project harnesses the power of AI to open up the visual world and describe nearby people, text and objects.
Optimized for use with VoiceOver, the app enables you to recognize:
• Short Text - Speaks text as soon as it appears in front of the camera.
• Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting.
• Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available. (works with iPhone 6 and later)
• People - Saves people’s faces so you can recognize them, and get an estimate of their age, gender, and emotions.
• Scenes (early preview) - Hear an overall description of the scene captured.
• Currency - Recognizes currency notes.
• Color - Identifies color.
• Handwriting - Reads handwritten text.
• Light - Generates an audible tone corresponding to the brightness in the surroundings.
• Images in other apps - Just tap “Share” and “Recognize with Seeing AI” to describe images from Mail, Photos, Twitter, and more.
Seeing AI is built to help you achieve more by leveraging the power of the cloud and artificial intelligence. As the research progresses, more channels may be added.
Check out tutorials with this YouTube playlist: http://aka.ms/SeeingAIPlaylist
Visit http://SeeingAI.com for more details
Available in 35 countries including the U.S., Canada, European Union, U.K., Australia, India, Hong Kong, New Zealand, Singapore. Coming soon to more countries.
Languages supported – English
Supported on iPhone 5C, 5S and later; best performance with iPhone 6S and later models.
Questions, feedback or feature requests? Email us at seeingai @ microsoft.com.
Comments
Seeing AI podcast
Apple vis has just posted a recording that I did. Please go have a listen, and feedback is welcome!
thanks for the podcast
I agree with the comment that perhaps the voiceover rate needs to be a bit slower, but I got a lot more out of this podcast than I did the one from CoolBlindTech and for that I'm greatful.
How is the heating and
How is the heating and battery draining on version 1.1? I can't download it yet
Heating and Battery Drainning
Heating and battery draining issues are still present in version 1.1. I don't think that this app will be my primary app unless they solve these issues. How much my iPhone 7 gets heated up wile using the app is scary. However, we should all keep in mind that we are still in the early versions of the app. I am sure things will get better.
Scene descriptions are surprisingly accurate
Took a photo of my husband yesterday in a restaurant. It said "man sitting at table indoors using tablet", he was actually surfing the internet on his phone. Just amazing! Tried reading all the tags that came with a shirt I bought and after some positioning difficulty, it worked as well as KNFB reader. But do remember KNFB works in airplane mode and this does not. This also needs the document positioned on a dark background, like a piece of black construction paper, to more accurately determine the document edges; KNFB is less picky.
But most amazing is the short text it grabs with video. I panned around the restaurant, capturing all the little advertising circulars scattered on our table. I point it at a random pile of papers and it reads a surprisingly large amount of visible text.
I also noticed that it kind of decides on its own when to turn the flash on and off which might be what is causing the battery and heating. When I turned more light on in my office, (powerful flourescents located above my desk) to the point that it made me uncomfortable, the app didn't keep activating the flash. So if you can take your item somewhere brighter to identify or read, it might save your battery.
Bar Code
I had the same experience with a box of Keeblers Club Crackers. On the side of the box SeeingAi thinks it sees a bar code and there is probably one there but not a real one. So I continue looking and found the real one on the bottom of the box. So I suggest to keep scanning to see if there are more than one bar code.
I also used the short text channel while riding in an Uber by holding the camera pointing out the window to see what text I can pick up. Fascinating what it can pick up. Like the name of Condo complex in my neighborhood.
It will be interesting to see if it can read the street sign names.
Hi,
Hi,
I here's what i think on your heating/battery issues. Remember, this app is probably using a lot of processing power/also it is using the camera and the flash. This, in my experience can cause the phone to heat up slightly. It shouldn't get to the point where the phone gets too hot to hold. If it gets to a point where it gets too warm, then the device should automatically shut down.
The Light and quick text
There is a setting in settings for the light to automatically come on or not.
Also, it's interesting that quicktext will read some signs and advertisements, but when I get it to read an envelope or small bit of tinier text, it often can't manage so well. Still fantastic, but surprising.
...
"I'd also recommend that you get permission before taking people's picture with this app since the picture will eventually be put through facial recognition."
face recognition is done completely on your phone, nothing is sent to the cloud. try turn on airplane mode, then you can go from building a face profile then recognize the face, you can achieve all of this without any network connection.
HHeating and battery
@Justin , I disabled the automanage lighting and the phone heats and the app drains battery the same way. I also tried the new version 1.1 and nothing changes. The short text option is awesome
Seeing A I and privacy
I would agree with some comments on here. I love the app but I don't like that it's fully cloud based. Even the bar code reader is cloud based. I confirmed this by turning on airplane mode and then attempting to scan a barcode. Also, I have scanned multiple documents with the document scan feature, and have attained very poor results. I get much better OCR recognition with Prizmo Go and I do with this app. Hopefully this will be worked on in future updates, and it would be nice if the app included a built-in OCR like Prismo go and KNFB reader do
Questions
As someone who cannot yet get the app as it’s not in my country, can I ask, does the app allow you to import documents and images for OCR or for scene recognition, as KNFB READER does for example, or is it only done live?
Also is there any product recognition other than the barcode scanning, for example like TapTapSee does?
Regarding the point above about the cloud, I would imagine the barcode recognition has to be off device because the databases it’s searching, by their very nature, have to be online.
That’s not to say that there aren’t legitimate privacy questions about the app though, for example around facial recognition. This app is a testing ground for AI research after all.
yes, it can
Yes it can import images from your photos app on your device. Just make sure you go into the 2nd more section, and toggle regignize with seeing ai to on. Then whenever you'd like to get a discription for an image, you can go to your photos app, select the photo you want to see, then select the recignize with seeing ai. I hope it's expanded to other countries in the near future. I don't mean to rub people's noses in it, but I feel very fortunate that I could get the app in mine. Hope this helps!
About bar codes
Its ability to scan bar codes is similar to store scanners, that are omnidirectional Digit-Eyes and other apps need the camera to point right at the code. The short text is great for scanning bookshelves. Walking around our living room, it can read spines if I position the camera with care. If you read many documents with it, purchase a thin black scarf and use it the way photographers use a black drape to enhance contrast. I found if I spread it under the document it detected the edges best. Black construction paper or a black rubber mat also works, but a scarf is portable.
Sharing or opening pictures from other apps
Hi,
I have read that one should be able to have a picture recognised with Seeing AI through the share options but need to turn it on first.
I have tried the share option on the photos app just like someone else here, but don't see a option to turn Seeing AI on.
There are 2 more buttons,one with copy slideshare,print etc,the other with Whatsapp,Icloud drive,Dropbox etc with toggles to switch them of and on.
But nothing like that for the Seeing AI. what am I missing?
Using Iphone 6 plus on IOS 10
ideas?
privacy, and other items
Hello all,
I was just informed, that the hot device is caused by the flashlight being turned on automatically, due to the lighting. As far as privacy goes, it's on the internet, as is most of our digital life is, this is, as someone pointed out, a testing ground for AI, so please keep that in mmind. As far as document scanning goes, in my podcast, while one of the documents tested was a glossy piece of paper, it did maintain the formatting, and I found my hands shook less because I wasn't conserned about having to capture the image my self. To the person who had poor results, did you try a different background? Also, give the ap some time. I wrote a review for this ap for a job assignment, and one of the points I stressed, is this is version 1.1, so it's kind of in the ground breaking stage. Glad to help, and I look forward to more comments!
Scanning documents with multiple pages
Hello,
Is it possible to scan multi page documents with this app? If not this is one area where KNFB still has a bit of a leg up.
Regards,
Greg Wocher
bar codes and seeing AI
I just scanned a big box of rice crispy treats and the app actually started beeping on the complete opposite side from where the bar code was. Has anybody else experienced this?
Lol yep!
Lol yep!
I found that happened the other day. Lol, I was trying to get a box scanned with the bar code feature and somehow, my shirt that i was wearing at the time must of had numbers or such on it, cause it was making the beeping sound even though there wasn't a box there in the first place. I just wanted to see what happened if i took it away from the box.
I did a successful scan the other day of some instant potatoes, my god I was very impressed with how great it was. It found the code, and it was on the top of the box.
This app just keeps getting better and better!
UK
Such a shame this isn't available in the UK yet... I can just tell how useful this app would be for me as a 14 year old school student. The ability to ready documents and short texts (let alone recognise faces and products) would be a life saver for me :)!
Seeing ai
Hello to all the community I can't understand why Seeing ai has been delayed being available in the UK Apple Store does anyone know when it will be available in the UK please let me know thank you to everyone
Seeing ai
Hello James I also live in the UK and wonder when this app is going to be available to us it doesn't make any sense that it wasn't available to us when it was launched in the United States regards
uk
I'm in the uk and using this app all the time and it works great here. Not really had any problems with anything that might explain why it's not officially released here yet except for some barcodes that are UK specific. It's quite easy to obtain this app through the Appstore but I won't give instruction here incase it violates something, just google how to do it.
Be patient
There are many possible reasons why this app is only available in a small number of countries at the moment.
For example:
They may want to check that the hardware and technology behind this service is scalable.
They may want to have a limited roll-out so that they can resolve any initial problems whilst there is still a relatively small user base.
They may not have translations in place for all languages.
There may be licensing agreements that limit or restrict how and where a technology used by the app can be used.
And probably many other possible reasons.
So, lets not give the developers a hard time because the app isn't currently available in your country. I am sure that Microsoft will want to make it as widely available as possible, as soon as possible.
Got it
Many thanks I have now got Seeing ai thanks Chris
a use and a funny experience
Hello all!
I have found a pretty neat use for Seeing AI. I have found that it does a good job reading memes! I've done some limited testing with random memes from various FB. pages. My results are that it just reads the text in memes. Which is fine because really that's all you need. I think it disscribed the photos in 1 or 2 cases as well as read text.
I also have a very funny story that has to do with this app! I have 2 photos of myself holding a Calves shirt. Well, every time I find a photos app like Braigo or Seeing AI., I run those 2 or at least 1 of those 2 photos through the app just for giggles and out of pure curiousity! Well, I ran 1 of those photos through and it read the text. That's not the funny part. Seeing AI. said that I was a 19-year-old male looking happy! haha lol Obviously, I'm not a man lol! I know it's because my hair was pulled back, but I still got a kick out of it!! And I thought you all might as well! Anyway, I just figured I'd share this experience and also the little tidbit about memes. I'll keep testing memes from random pages on Facebook.
Seeing ai
Hello I have downloaded this app a few days ago it is fantastic very good I will be glad when it is available in the UK so I can use all of the facilities regards
how get this app in uk
hi how will i get this app as still not in uk? assume all that doesn't work yet are uk barcodes? svague how still not out, rediculous really.
Google
Hi, its best if you google how to do it as I don’t wish to give specific instructions here. It’s pretty straightforward though.
how well it work
what doesn't x∂ then in the uk? assume just not recognising barcodes?
Version 1.3
What's New in Version 1.3
• Product channel optimized for iOS 11 - still works on iOS 10 as before.
• Added support for 3D Touch, providing a convenient way to launch the app with a channel pre-selected.
• It is now possible to share/save the original image of a document.
• Various minor bug fixes.
re: bar codes with seeing AI
It may just be that the particular barcode of your product was not in the database yet.
Seeing ai now in UK
Seeing AI now in the UK https://itunes.apple.com/app/id999062298
I'M 14, NOT 50. AND I'M FEMALE!
Got this app about half an hour ago as it's finally out in the UK. First thing I tried was the face recognition - as that's the feature I was most excited about.
Now I'm fully aware that I look older than my years (14) but it came back with: "50 year old man with black here and a beard looking sad". The only part of that which is true was the sad face I was pulling at the camera. I'm a: "14 year old girl with brown hair" haha!
Problem with Update
Solved. The new currency channel doesn't work in IOS 10.
Cannot share photos iwth Seeing AI
I listen to one of the Youtube videos. In one video you can share photos in the photo App with Seeing AI. If I select a photo in my photo App and select Share I do not see the option to share it with Seeing AI. I have a iPhone 5S with IOS 11.1. Hope somebody can help to make this work!
BTW: I downoaded the app from the Dutch appstore.
Simply amazing!
Good job!!!!
Almost like a pair of eyes when you need them
This app is like a pair of eyes when you need them, but it will not replace a human reading you or telling you a product.
Chuck
Cannot launch the app after updating to V2.0
Hello,
For some reason, I cannot launch the app after updating it to V2.0. Everything was working perfectly before the update but now, every time I try to open the app, it sends me back to the home screen. I deleted and reinstaled the app but it didn't work. I tried everything short of factory reset.
Has anyone experienced the same issue? If so, how did you solve it? I am using an iPhone 7 running iOS 10.3.3.
To send a photo to Seeing AI
To send a photo to Seeing AI go to photos, choose your album and photo and double tap on share. There you have two lists, the first has Facebook, Twitter and so on, and the second one has print, copy and other things. Each list has a More button at the end, pres the second one. There you can check and uncheck things that you want to appear on that list, including seeing ai.
After that it should appear on the second list.
Sending a photo to Seeing AI works now
Thank you for your help, it owrks now!
New update with fixes
Hi ashandarei, this is Anirudh from the Seeing AI Team. Thanks for reporting this. A handful of users on the older iOS have noticed an issue in starting the app. We have published an update yesterday, so give it a try. Also, highly recommended to update your phone to iOS 11.
Not available in South Africa yet
Will it come to South Africa in the future?
Thank you. I got the update
Thank you. I got the update and instaled it a few hours ago. It solved the issue. Everything is working seemlessly now.
WOW. Excellent App.
WOW. Excellent App.
Watched Some Videos about This
Hi everyone. Just yesterday I was perusing the Internet as always, and came across some really good videos about this app. It seems like something I want for the iPhone 7 which I just got. However, I'm wondering 2 things. First off, what are peoples' experiences with the app in regards to travel? My iPhone tutor at Second Sense here in Chicago said that while all these GPS apps are good, most if not all of them are a bit inaccurate when it comes to giving directions. I briefly checked out the iPhone's on-board Maps app, and it seems to be pretty good with VoiceOver. But I'm rather new to this whole GPS concept and need help. My second question is how accurate is Seeing AI in comparison with Directions for Me?
Wrong app
Jake, this app is a camera app, not a GPS app. You're probably looking for Microsoft Soundscapes if you are thinking of the newly-released GPS app that Microsoft made. GPS is generally quite accurate to a few metres from the desired location, though remember that GPS will not inform you about things like areas under construction, obstacles, etc.
Awesome App
I just downloaded this app and did a couple things with it. My sighted neighbor from across the hall was in here when I tried to scan a small box of jello that another neighbor had previously given me. We eventually got it. Then I tried scanning a jar of peanut butter from my cabinets, but was totally unsuccessful. I then went out in the front hallway to try and scan some of the photos on the bulletin board just outside my door, and the app just told me that no edges were visible. I love this app but it's going to take some practice. Lots of very useful features though, and I like the audio tutorials. Very accessible too. Nice job Microsoft.
Amazing app!!!
I absolutely recommend this app to anyone who is visually impaired. It can really change your life! Every day I discover new features, and they're all incredible. I just checked and it can even read the computer screen! I can't believe it's free! I wouldn't hesitate to pay for this app.
Fran
Thank you Applevis family!
Just wanted to say a big thank you from the Seeing AI Team to all the AppleVis family, for the support and the kind comments. Lots to improve, looking forward to an exciting future!