Your Eyes: Empowering the Visually Impaired Through AI Vision Technology

Bridging the Vision Gap

Although vision is our most dominant sense, we often take it for granted. Without sight, everyday tasks like navigating our homes or crossing streets become significant challenges.

Yet in United States alone, approximately 1 million people live with blindness, and 90% of these cases have no cure or therapy. Similar statistics exist worldwide, highlighting a critical need for assistive technologies that can bridge this sensory gap.

Your Eyes (formerly known as Beyond Sight) was born from this realization - a groundbreaking application designed to empower visually impaired individuals with an intuitive and intelligent understanding of their surroundings through real-time audio feedback.

Core Technologies

The power of Your Eyes comes from its sophisticated integration of cutting-edge AI technologies:

• Object Detection with YOLOv7: Running on-device at over 10 inference runs per second, our implementation accurately identifies and tracks objects in real-time, providing crucial information about obstacles and points of interest.

• Contextual Understanding with GPT-4 Vision: Delivers insightful and contextual descriptions that go beyond simple object recognition, helping users appreciate the full scope of their surroundings.

• Natural Audio Feedback with Deepgram: Converts text into natural-sounding speech, ensuring the information is delivered in a pleasant and understandable manner.

• Progressive Web App Architecture: Built with Next.js, Your Eyes functions as a PWA that can be installed on smartphones without requiring app store approval, making it widely accessible.

User Experience

Your Eyes offers two primary modes of audio feedback:

1. Immediate Object Detection: Provides instantaneous audio alerts about identified objects and their relative positions (left/right/center), offering efficient spatial awareness for safe navigation.

2. Descriptive Scene Understanding: Delivers detailed, AI-generated descriptions that help users appreciate the subtleties of their environment—like understanding it's a "beautiful sunny day with people walking in a park."

Challenges and Solutions

Building Your Eyes presented several technical challenges:

• Performance Optimization: Running complex neural networks on consumer devices required careful optimization to maintain reliable frame rates.

• Audio Prioritization: We developed sophisticated algorithms to determine which objects and descriptions should be communicated first, focusing on safety and relevance.

• User-Centered Design: Through market research with visually impaired individuals, we refined our feature set to address their most pressing needs.

Key Accomplishments

Our work on Your Eyes has yielded impressive results we're proud to share:

• Ultra-low Latency: We've optimized the application to achieve 100ms response time for object detection, creating a near real-time experience that's critical for navigation assistance.

• Rapid User Adoption: Within just three weeks of launch, over 500 users have integrated Your Eyes into their daily routines, validating both the need for this solution and its effectiveness.

• Community Feedback: Early users have provided invaluable feedback that has directly shaped our development roadmap, creating a virtuous cycle of improvement.

The Road Ahead

The future of Your Eyes includes several exciting developments:

• Hardware Integration: Combining the software with smart devices such as mobility canes, glasses, and watches to create a more seamless experience.

• Custom Model Training: Developing specialized YOLOv7 models to detect items particularly relevant to visually impaired users, such as handrails, tactile pavements, and audio-descriptive signs.

• Enhanced Audio Experience: Fine-tuning our audio cues to prevent overstimulation while ensuring critical information is communicated effectively.

• Danger Alert System: Creating a specialized system that can provide immediate warnings about potential hazards.