HI, I’d be interested in getting some feedback on an assistive-navigation idea I’m considering submitting for a student engineering co-design project at NUS.
The concept came from the way I currently learn unfamiliar routes with a sighted student assistant. During a route-learning session, they often give very specific information about turns, landmarks, entrances, tactile paving, awkward junctions, and other details that are difficult to capture in ordinary navigation apps. I could record the session, but afterwards that information is buried in a long audio recording.
My idea is therefore to associate the spoken guidance with the actual route, so that when I later travel independently I could hear relevant instructions automatically, or press a help button and hear what my guide originally told me at that particular point.
I know there is some overlap with apps and projects such as Clew, Soundscape/VoiceVista, BlindSquare and route-recording systems, so I’d particularly welcome comparisons with anything that already exists.
I’d be very interested in hearingg whether you would find this useful in real-world route learning, whether you know of an existing app that already does something similar or exactly this, what features or gaps in wayfinding apps you would consider essential, what would make you distrust or stop using such a system, and whether there are situations where personalised human guidance is more useful than ordinary robotic turn-by-turn navigation.
The draft proposal
My idea relates to independent navigation and route learning as a blind person. When I learn a new route, I often walk it several times with a student assistant or another sighted person who gives me detailed verbal guidance about turns, landmarks, entrances, crossings, obstacles, changes in terrain, and other useful contextual information. Much of this information is highly specific to the route and can be difficult to remember afterwards.
Although I could record the entire route-learning session, an ordinary audio recording is organised by time rather than by location. If I later need to remember what my assistant told me at a particular junction or entrance, I would have to search through a long recording to find the relevant part.
I would therefore be interested in co-creating a navigation or route-memory system that can record a route together with the verbal guidance given while I am learning it, and associate relevant parts of the recording with particular points along that route.
For example, while I am practising a route with my student assistant, the system could record our conversation as well as information about where I am along the route. At important locations, either I or my assistant could mark or bookmark a point. Later, when I travel the route independently, the system could provide navigation guidance and allow me to press a “help” button to hear the original instruction associated with my current location, such as: “Turn right immediately after the tactile paving,” or “The entrance is the second door after the pillar.”
Ideally, the prototype could explore the use of technologies already available on a smartphone, such as GPS for outdoor locations, the camera, motion sensors, and, where appropriate, LiDAR or other indoor-positioning techniques. However, I am very open to the student team exploring different technical approaches. The central need is not any particular technology, but a way of turning a route-learning session with a human guide into a reusable, location-aware memory of the route.
Some additional possibilities, depending on the scope of the project, could include:
- bookmarking particularly important or complicated points along a route;
- automatically playing essential guidance when approaching a marked location;
- allowing the user to request more information only when needed;
- replaying a short section of the original conversation around a particular location;
- storing personalised landmarks and orientation information;
- supporting both indoor and outdoor portions of the same route;
- detecting when the user may have deviated from a previously learned route.
I think this could be especially useful for blind people who regularly need to learn complex routes around campuses, workplaces, transport hubs, or other unfamiliar environments. It would not replace a white cane, guide dog, orientation and mobility training, or real-time obstacle detection, but could serve as an additional tool for learning, remembering, and independently retracing familiar routes.
I would be happy to work closely with the student team, demonstrate how I currently learn routes, test prototypes on actual routes around NUS, and provide feedback throughout the design process.
Comments
Sounds interesting
I like the sound of this. I've just started route training - I'm still early days but my main worry is trying to remember everything. And even when my training ends, if I don't do the route for a while it would be helpful to be able to have something that can help if I forget something important.
I'm making notes and trying to read through them to cement them in my mind, but there's nothing like getting lost to panic my brain empty. If I could just summon something that said "you are here, the next thing to look out for is this" or whatever that could be pretty reassuring.
Overengineering
I usually explore new routes all by myself, by asking the locals for directions and recording my steps with watchOS Compass Waypoints. The only thing that I find missing both in Compass Waypoints and in VoiceVista's Markers is the ability to attach notes recording non-obvious information about particular locations.
A Low-Tech Way to Do This
I am certainly not trying to discourage you from developing an app for this if that's what you want to do. If you (or anyone else reading this) are just looking for an easy solution to this type of problem that you could use right now, I present the following:
The community college I went to has a large campus consisting of multiple interconnected buildings. I received some site-specific training from a fantastic orientation and mobility instructor, but it quickly became apparent to me that I needed a solution to help jog my memory when I was walking the routes on my own.
My solution was to make a set of written turn-by-turn instructions, email them to myself, then have my phone in one hand and follow the instructions as I went along. These instructions included major landmarks and even, in one case, information about how to get myself back on track if I wandered off route. It was almost like having a turn-by-turn GPS (though of course I had to rely on myself to determine landmarks) and was a tremendous confidence-booster because I did not have to rely on my memory to learn long and complex routes - all I needed to do was read my notes, pay attention to my environment, and follow the instructions.
In case this would be helpful to anyone, below is a sample of the instructions for one of those routes:
Beyond recording a route: knowing where you are and when guidanc
Recording the guidance given during route learning could preserve details that ordinary navigation apps often miss, such as which entrance to use or how to recognise a difficult junction. Attaching notes or audio to locations is relatively straightforward. The harder part is deciding when those instructions apply on a later journey.
Among tall buildings, a phone’s GPS position can sometimes be off by 20–30 metres or more. If a recording is attached to a point beside one of those buildings, how does the app determine that the user has reached it? A small arrival radius may prevent the recording from playing at all. A larger radius may trigger it too early, at the wrong junction, or on the opposite side of the road.
There are also cases where the user takes a different but valid path. At a junction requiring two crossings, for example, the recorded journey may cross one arm of the junction and then the other. On the next visit, the pedestrian signal sequence may make the opposite order more convenient, leading to the same onward pavement. Does that count as going off route? Does the user still need to reach the original recorded point, even though it is no longer necessary?
Similarly, a route might have been recorded walking north along the east side of a street, while the user is now walking north along the west side. The direction is the same, but instructions about entrances, landmarks or crossings may no longer apply. Is this an acceptable variation, or does it require a different route? Can the available positioning data reliably distinguish the two sides?
A shortcut creates another ambiguity. If the reported position moves close to a later section of the route, has the user actually taken a shortcut, or has GPS drift placed them there? Advancing too readily could skip important guidance. Not advancing could leave the app repeatedly directing the user back to points they have deliberately bypassed.
When someone genuinely goes off route, recognising that is only half the problem. How can they rejoin it? The nearest recorded point may be across a road, behind a wall or reachable only through an inaccessible passage. Pointing towards that coordinate is not necessarily a usable way back.
These situations mark an important distinction between a route-memory aid and a system that automatically guides someone along a recorded route. With manually replayed notes, the traveller identifies the relevant landmark and chooses the instruction. With automatic playback, the system has to make that connection reliably, or make its uncertainty clear. Much of the difficulty lies not in following the original route successfully, but in handling the occasions when the position is uncertain or the journey unfolds differently.
Re: A Low-Tech Way to Do This
Thanks, Michael. The example makes this very concrete. The notes about how to recognise when you’ve gone too far were particularly helpful. Your mention of the elevator also made me curious: if it doesn’t announce the floors, how do you tell which floor you’ve reached?