Novel framework for outdoor mobility assistance and auditory display for visually impaired people

Outdoor mobility of Visually Impaired People (VIPs) has always been challenging due to the dynamically varying scenes and environmental states. Variety of systems have been introduced to assist VIPs' mobility that include sensor mounted canes and use of machine intelligence. However, these systems are not reliable when used to navigate the VIPs in dynamically changing environments. The associated challenges are the robust sensing and avoiding diverse types of obstacles, dynamically modelling the changing environmental states (e.g. moving objects, road-works), and effective communication to interpret the environmental states and hazards. In this paper, we propose an intelligent wearable auditory display framework that will process real-time video and multi-sensor data streams to: a) identify the type of obstacles, b) recognize the surrounding scene/objects and corresponding attributes (e.g. geometry, size, shape, distance from user), c) automatically generate the descriptive information about the recognized obstacle/objects and attributes, d) produce accurate, precise and reliable spatial information and corresponding instructions in audio-visual form to assist and navigate VIPs safely with or without the assistance of traditional means.