Seeing AI

Category

Description of App

Seeing AI is a free app that narrates the world around you. Designed with and for the blind and low vision community, this ongoing research project harnesses the power of AI to open up the visual world by describing nearby people, text and objects.

Seeing AI provides tools to assist with a variety of daily tasks:

• Short Text - Speaks text as soon as it appears in front of the camera.

• Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting.

• Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available.

• People - Saves people’s faces so you can recognize them, and get an estimate of their age, gender, and expression.

• Currency - Recognizes currency notes.

• Scenes - Hear an overall description of the scene captured. Explore the photo by moving your finger over the screen to hear the location of different objects.

• World - An Audio Augmented Reality experience to explore an unfamiliar environment, including hearing objects announced around you with Spatial Audio (requires a device with a LiDAR, and iOS 14+).

• Indoor Navigation - Available on the World Channel, enables you to create routes through a building, like "entrance to classroom", and navigate by following the sound (requires a device with an A9 or later processor, and iOS 14+).

• Colors - Identifies colors.

• Handwriting - Reads handwritten text like in greeting cards (available in a subset of languages).

• Light - Generates an audible tone corresponding to the brightness in the surroundings.

• Images in other apps - Just tap “Share” and “Recognize with Seeing AI” to describe images from Mail, Photos, Twitter, and more.

• Browse Photos - Hear descriptions of photos saved on your device.

Seeing AI continues to evolve as we hear from the community, and AI research advances.

Check out tutorials with this YouTube playlist: http://aka.ms/SeeingAIPlaylist.

Questions, feedback or feature requests? Email us at [email protected].

Version

5.4

Free or Paid

Free

Apple Watch Support

No

Device(s) App Was Tested On

iPhone

iOS Version

18.1

Accessibility Comments

This app is very accessible

VoiceOver Performance

VoiceOver reads all page elements.

Button Labeling

Most buttons are clearly labeled.

Usability

The app is fully accessible with VoiceOver and is easy to navigate and use.

Other Comments

This is an amazing app! I was pleasantly surprised to learn about this app from mainstream tech sites this morning.

Recommendations

69 people have recommended this app

Most recently recommended by Casey 1 month 1 week ago

Options

Comments

By Darrell Bowles on Wednesday, August 9, 2017 - 03:26

Apple vis has just posted a recording that I did. Please go have a listen, and feedback is welcome!

By Troy B on Wednesday, August 9, 2017 - 03:26

In reply to by Darrell Bowles

I agree with the comment that perhaps the voiceover rate needs to be a bit slower, but I got a lot more out of this podcast than I did the one from CoolBlindTech and for that I'm greatful.

By Raul on Wednesday, August 9, 2017 - 03:26

How is the heating and battery draining on version 1.1? I can't download it yet

Heating and battery draining issues are still present in version 1.1. I don't think that this app will be my primary app unless they solve these issues. How much my iPhone 7 gets heated up wile using the app is scary. However, we should all keep in mind that we are still in the early versions of the app. I am sure things will get better.

By Deborah Armstrong on Wednesday, August 9, 2017 - 03:26

Took a photo of my husband yesterday in a restaurant. It said "man sitting at table indoors using tablet", he was actually surfing the internet on his phone. Just amazing! Tried reading all the tags that came with a shirt I bought and after some positioning difficulty, it worked as well as KNFB reader. But do remember KNFB works in airplane mode and this does not. This also needs the document positioned on a dark background, like a piece of black construction paper, to more accurately determine the document edges; KNFB is less picky.

But most amazing is the short text it grabs with video. I panned around the restaurant, capturing all the little advertising circulars scattered on our table. I point it at a random pile of papers and it reads a surprisingly large amount of visible text.
I also noticed that it kind of decides on its own when to turn the flash on and off which might be what is causing the battery and heating. When I turned more light on in my office, (powerful flourescents located above my desk) to the point that it made me uncomfortable, the app didn't keep activating the flash. So if you can take your item somewhere brighter to identify or read, it might save your battery.

I had the same experience with a box of Keeblers Club Crackers. On the side of the box SeeingAi thinks it sees a bar code and there is probably one there but not a real one. So I continue looking and found the real one on the bottom of the box. So I suggest to keep scanning to see if there are more than one bar code.
I also used the short text channel while riding in an Uber by holding the camera pointing out the window to see what text I can pick up. Fascinating what it can pick up. Like the name of Condo complex in my neighborhood.
It will be interesting to see if it can read the street sign names.

By Justin on Wednesday, August 9, 2017 - 03:26

Hi,
I here's what i think on your heating/battery issues. Remember, this app is probably using a lot of processing power/also it is using the camera and the flash. This, in my experience can cause the phone to heat up slightly. It shouldn't get to the point where the phone gets too hot to hold. If it gets to a point where it gets too warm, then the device should automatically shut down.

By Remy on Wednesday, August 9, 2017 - 03:26

There is a setting in settings for the light to automatically come on or not.

Also, it's interesting that quicktext will read some signs and advertisements, but when I get it to read an envelope or small bit of tinier text, it often can't manage so well. Still fantastic, but surprising.

By LaBoheme on Wednesday, August 9, 2017 - 03:26

"I'd also recommend that you get permission before taking people's picture with this app since the picture will eventually be put through facial recognition."

face recognition is done completely on your phone, nothing is sent to the cloud. try turn on airplane mode, then you can go from building a face profile then recognize the face, you can achieve all of this without any network connection.

By Raul on Wednesday, August 9, 2017 - 03:26

@Justin , I disabled the automanage lighting and the phone heats and the app drains battery the same way. I also tried the new version 1.1 and nothing changes. The short text option is awesome

By Matt on Wednesday, August 9, 2017 - 03:26

I would agree with some comments on here. I love the app but I don't like that it's fully cloud based. Even the bar code reader is cloud based. I confirmed this by turning on airplane mode and then attempting to scan a barcode. Also, I have scanned multiple documents with the document scan feature, and have attained very poor results. I get much better OCR recognition with Prizmo Go and I do with this app. Hopefully this will be worked on in future updates, and it would be nice if the app included a built-in OCR like Prismo go and KNFB reader do

By Dave Nason on Wednesday, August 9, 2017 - 03:26

Member of the AppleVis Editorial Team

As someone who cannot yet get the app as it’s not in my country, can I ask, does the app allow you to import documents and images for OCR or for scene recognition, as KNFB READER does for example, or is it only done live?
Also is there any product recognition other than the barcode scanning, for example like TapTapSee does?
Regarding the point above about the cloud, I would imagine the barcode recognition has to be off device because the databases it’s searching, by their very nature, have to be online.
That’s not to say that there aren’t legitimate privacy questions about the app though, for example around facial recognition. This app is a testing ground for AI research after all.

By Dawn 👩🏻‍🦯 on Wednesday, August 9, 2017 - 03:26

Yes it can import images from your photos app on your device. Just make sure you go into the 2nd more section, and toggle regignize with seeing ai to on. Then whenever you'd like to get a discription for an image, you can go to your photos app, select the photo you want to see, then select the recignize with seeing ai. I hope it's expanded to other countries in the near future. I don't mean to rub people's noses in it, but I feel very fortunate that I could get the app in mine. Hope this helps!

By Deborah Armstrong on Wednesday, August 9, 2017 - 03:26

Its ability to scan bar codes is similar to store scanners, that are omnidirectional Digit-Eyes and other apps need the camera to point right at the code. The short text is great for scanning bookshelves. Walking around our living room, it can read spines if I position the camera with care. If you read many documents with it, purchase a thin black scarf and use it the way photographers use a black drape to enhance contrast. I found if I spread it under the document it detected the edges best. Black construction paper or a black rubber mat also works, but a scarf is portable.

By falcon wings on Wednesday, August 9, 2017 - 03:26

Hi,
I have read that one should be able to have a picture recognised with Seeing AI through the share options but need to turn it on first.
I have tried the share option on the photos app just like someone else here, but don't see a option to turn Seeing AI on.
There are 2 more buttons,one with copy slideshare,print etc,the other with Whatsapp,Icloud drive,Dropbox etc with toggles to switch them of and on.
But nothing like that for the Seeing AI. what am I missing?
Using Iphone 6 plus on IOS 10
ideas?

By Darrell Bowles on Wednesday, August 9, 2017 - 03:26

Hello all,
I was just informed, that the hot device is caused by the flashlight being turned on automatically, due to the lighting. As far as privacy goes, it's on the internet, as is most of our digital life is, this is, as someone pointed out, a testing ground for AI, so please keep that in mmind. As far as document scanning goes, in my podcast, while one of the documents tested was a glossy piece of paper, it did maintain the formatting, and I found my hands shook less because I wasn't conserned about having to capture the image my self. To the person who had poor results, did you try a different background? Also, give the ap some time. I wrote a review for this ap for a job assignment, and one of the points I stressed, is this is version 1.1, so it's kind of in the ground breaking stage. Glad to help, and I look forward to more comments!

By Greg Wocher on Wednesday, August 9, 2017 - 03:26

Hello,
Is it possible to scan multi page documents with this app? If not this is one area where KNFB still has a bit of a leg up.

Regards,
Greg Wocher

By Troy B on Wednesday, August 9, 2017 - 03:26

I just scanned a big box of rice crispy treats and the app actually started beeping on the complete opposite side from where the bar code was. Has anybody else experienced this?

By Justin on Wednesday, August 9, 2017 - 03:26

Lol yep!
I found that happened the other day. Lol, I was trying to get a box scanned with the bar code feature and somehow, my shirt that i was wearing at the time must of had numbers or such on it, cause it was making the beeping sound even though there wasn't a box there in the first place. I just wanted to see what happened if i took it away from the box.
I did a successful scan the other day of some instant potatoes, my god I was very impressed with how great it was. It found the code, and it was on the top of the box.
This app just keeps getting better and better!

By ellzagirliegirl on Wednesday, August 9, 2017 - 03:26

Such a shame this isn't available in the UK yet... I can just tell how useful this app would be for me as a 14 year old school student. The ability to ready documents and short texts (let alone recognise faces and products) would be a life saver for me :)!

By George on Wednesday, August 9, 2017 - 03:26

Hello to all the community I can't understand why Seeing ai has been delayed being available in the UK Apple Store does anyone know when it will be available in the UK please let me know thank you to everyone

Hello James I also live in the UK and wonder when this app is going to be available to us it doesn't make any sense that it wasn't available to us when it was launched in the United States regards

By chris R on Wednesday, August 9, 2017 - 03:26

I'm in the uk and using this app all the time and it works great here. Not really had any problems with anything that might explain why it's not officially released here yet except for some barcodes that are UK specific. It's quite easy to obtain this app through the Appstore but I won't give instruction here incase it violates something, just google how to do it.

By Eileen on Saturday, September 9, 2017 - 03:26

There are many possible reasons why this app is only available in a small number of countries at the moment.

For example:

They may want to check that the hardware and technology behind this service is scalable.

They may want to have a limited roll-out so that they can resolve any initial problems whilst there is still a relatively small user base.

They may not have translations in place for all languages.

There may be licensing agreements that limit or restrict how and where a technology used by the app can be used.

And probably many other possible reasons.

So, lets not give the developers a hard time because the app isn't currently available in your country. I am sure that Microsoft will want to make it as widely available as possible, as soon as possible.

By George on Saturday, September 9, 2017 - 03:26

Many thanks I have now got Seeing ai thanks Chris

By Dawn 👩🏻‍🦯 on Saturday, September 9, 2017 - 03:26

Hello all!

I have found a pretty neat use for Seeing AI. I have found that it does a good job reading memes! I've done some limited testing with random memes from various FB. pages. My results are that it just reads the text in memes. Which is fine because really that's all you need. I think it disscribed the photos in 1 or 2 cases as well as read text.

I also have a very funny story that has to do with this app! I have 2 photos of myself holding a Calves shirt. Well, every time I find a photos app like Braigo or Seeing AI., I run those 2 or at least 1 of those 2 photos through the app just for giggles and out of pure curiousity! Well, I ran 1 of those photos through and it read the text. That's not the funny part. Seeing AI. said that I was a 19-year-old male looking happy! haha lol Obviously, I'm not a man lol! I know it's because my hair was pulled back, but I still got a kick out of it!! And I thought you all might as well! Anyway, I just figured I'd share this experience and also the little tidbit about memes. I'll keep testing memes from random pages on Facebook.

Hello I have downloaded this app a few days ago it is fantastic very good I will be glad when it is available in the UK so I can use all of the facilities regards

By Karok on Monday, October 9, 2017 - 03:26

hi how will i get this app as still not in uk? assume all that doesn't work yet are uk barcodes? svague how still not out, rediculous really.

Hi, its best if you google how to do it as I don’t wish to give specific instructions here. It’s pretty straightforward though.

By Karok on Monday, October 9, 2017 - 03:26

what doesn't x∂ then in the uk? assume just not recognising barcodes?

By AnonyMouse on Monday, October 9, 2017 - 03:26

Member of the AppleVis Editorial Team

What's New in Version 1.3

• Product channel optimized for iOS 11 - still works on iOS 10 as before.
• Added support for 3D Touch, providing a convenient way to launch the app with a channel pre-selected.
• It is now possible to share/save the original image of a document.
• Various minor bug fixes.

By AbleTec on Monday, October 9, 2017 - 03:26

It may just be that the particular barcode of your product was not in the database yet.

By ellzagirliegirl on Saturday, December 9, 2017 - 03:26

Got this app about half an hour ago as it's finally out in the UK. First thing I tried was the face recognition - as that's the feature I was most excited about.

Now I'm fully aware that I look older than my years (14) but it came back with: "50 year old man with black here and a beard looking sad". The only part of that which is true was the sad face I was pulling at the camera. I'm a: "14 year old girl with brown hair" haha!

By JeffB on Saturday, December 9, 2017 - 03:26

Solved. The new currency channel doesn't work in IOS 10.

By Peter Verhoeven on Tuesday, January 9, 2018 - 03:26

I listen to one of the Youtube videos. In one video you can share photos in the photo App with Seeing AI. If I select a photo in my photo App and select Share I do not see the option to share it with Seeing AI. I have a iPhone 5S with IOS 11.1. Hope somebody can help to make this work!
BTW: I downoaded the app from the Dutch appstore.

By Chuck Winstead on Tuesday, January 9, 2018 - 03:26

This app is like a pair of eyes when you need them, but it will not replace a human reading you or telling you a product.
Chuck

By Faruk on Tuesday, January 9, 2018 - 03:26

Hello,

For some reason, I cannot launch the app after updating it to V2.0. Everything was working perfectly before the update but now, every time I try to open the app, it sends me back to the home screen. I deleted and reinstaled the app but it didn't work. I tried everything short of factory reset.
Has anyone experienced the same issue? If so, how did you solve it? I am using an iPhone 7 running iOS 10.3.3.

To send a photo to Seeing AI go to photos, choose your album and photo and double tap on share. There you have two lists, the first has Facebook, Twitter and so on, and the second one has print, copy and other things. Each list has a More button at the end, pres the second one. There you can check and uncheck things that you want to appear on that list, including seeing ai.
After that it should appear on the second list.

Hi ashandarei, this is Anirudh from the Seeing AI Team. Thanks for reporting this. A handful of users on the older iOS have noticed an issue in starting the app. We have published an update yesterday, so give it a try. Also, highly recommended to update your phone to iOS 11.

Thank you. I got the update and instaled it a few hours ago. It solved the issue. Everything is working seemlessly now.

By Ekaj on Saturday, June 9, 2018 - 03:26

Hi everyone. Just yesterday I was perusing the Internet as always, and came across some really good videos about this app. It seems like something I want for the iPhone 7 which I just got. However, I'm wondering 2 things. First off, what are peoples' experiences with the app in regards to travel? My iPhone tutor at Second Sense here in Chicago said that while all these GPS apps are good, most if not all of them are a bit inaccurate when it comes to giving directions. I briefly checked out the iPhone's on-board Maps app, and it seems to be pretty good with VoiceOver. But I'm rather new to this whole GPS concept and need help. My second question is how accurate is Seeing AI in comparison with Directions for Me?

By TJT 2001 on Saturday, June 9, 2018 - 03:26

Jake, this app is a camera app, not a GPS app. You're probably looking for Microsoft Soundscapes if you are thinking of the newly-released GPS app that Microsoft made. GPS is generally quite accurate to a few metres from the desired location, though remember that GPS will not inform you about things like areas under construction, obstacles, etc.

By Ekaj on Friday, November 9, 2018 - 03:26

I just downloaded this app and did a couple things with it. My sighted neighbor from across the hall was in here when I tried to scan a small box of jello that another neighbor had previously given me. We eventually got it. Then I tried scanning a jar of peanut butter from my cabinets, but was totally unsuccessful. I then went out in the front hallway to try and scan some of the photos on the bulletin board just outside my door, and the app just told me that no edges were visible. I love this app but it's going to take some practice. Lots of very useful features though, and I like the audio tutorials. Very accessible too. Nice job Microsoft.

By Fran on Wednesday, January 9, 2019 - 03:26

I absolutely recommend this app to anyone who is visually impaired. It can really change your life! Every day I discover new features, and they're all incredible. I just checked and it can even read the computer screen! I can't believe it's free! I wouldn't hesitate to pay for this app.
Fran

By Anirudh Koul on Wednesday, January 9, 2019 - 03:26

Just wanted to say a big thank you from the Seeing AI Team to all the AppleVis family, for the support and the kind comments. Lots to improve, looking forward to an exciting future!