Smart Saliency Detection for Prosthetic Vision

People with visual impairments often have difficulty locating misplaced objects. This can be a major barrier to their independence and quality of life. Retinal prostheses can restore some vision to people with severe vision loss. We introduce a novel real-time system for locating any misplaced objects for people with visual impairments using retinal prostheses. The system combines One For All (OFA) for Visual Grounding and Google Speech Recognition to identify the object to be located. It then uses an image processing technique called grabCut to extract the object from the background to emphasize the located object and incorporate it with the background frame to send it to the retinal implant for stimulation. The system was evaluated on a set of images containing misplaced objects and was able to locate the objects in real-time successfully. We believe that this system could be used to improve the independence and quality of life of people with visual impairments. The system is still in the early stages of development, but we are working to improve its performance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.