Project Illustration

Cinque Terre

Video Presentation

Indoor Infrared Stereo

Outdoor Infrared Stereo

Indoor Visible Light Stereo

Outdoor Visible Light Stereo

Project Functions

Cinque Terre

Obstacle Detection

Based on the infrared stereo camera, large-scale stereo matching is conducted with the aid of natural light and structured light to obtain depth maps. Using Mean-Shift algorithm, the distances, directions and sizes of the nearest obstacles are calculated and mapped to stereo sound to assist the navigation of the visually impaired.



Cinque Terre

Ground Detection

Based on the infrared stereo camera, large-scale stereo matching and color image guided filtering are conducted with the aid of natural light and structured light to complete depth maps. Using random sampling consensus, surface normal vector estimation and seeded region growing algorithm, the ground areas are detected and mapped to stereo sound to assist the navigation of the visually impaired.



Cinque Terre

Banknote Identification

Based on the survey on the needs of visually impaired people, banknotes identification is one of the designed functions to provide a convenient life. The banknote identification method selects image region with RGB-D information, and detects potential banknote regions with classifiers. Then, to determine if there is a banknote and its denomination, SURF feature point matching is carried out. This method extracts banknote region and provides its denomination in real time for complex environment with good robustness.



Cinque Terre

Pedestrian Crossing Light Detection

Pedestrian crossing light detection algorithm is based on the color extraction and machine learning. It accurately detects the location of the crossing lights in the image in real time and gives the status of the crossing lights. As shown in the figure, the corresponding color of the bounding box is the detection result of crossing lights.



Cinque Terre

Crosswalk Detection

To cross the road safely is the urgent need for blind people. The crosswalk detection algorithm based on the stripe extraction and clustering is used to find and locate crosswalks at traffic intersections. As shown in the figure, the locations and directions of the crosswalk are identified, which are converted to prompt messages to the user.



Cinque Terre

Face Recognition

Face recognition in the intelligent visual assistance is designed to help visually impaired people to perceive and identify friends, relatives and strangers in their daily life. Face recognition system uses color-depth fused information, target tracking and neural network technology. When a visually impaired user is using the intelligent assisting device, the face recognition system gradually collects and learns the faces that users often meet. When the recognition system reaches a sufficient training level, the face that appears in the scene is identified and the recognition result is transferred to the visually impaired user through a specific interactive way.



Cinque Terre

Traversable Area Detection

The depth image is enhanced with semi-global matching and color image guided filtering based on the RGB-D camera. Using dynamic programming algorithm, traversable areas are detected in the random occupancy grid. The spline surface is utilized to fit the ground area and obstacles are represented with stixels. Stereo sound feedback is generated to assist navigation of visually impaired people.



Cinque Terre

Water Hazards Detection

Based on polarized RGB-Depth (pRGB-D) camera, depth image is enhanced with semi-global matching and color image guided filtering. With polarization-color-depth-attitude information, traversable area and water hazards are detected simultaneously through dynamic programming. Stereo sound feedback is generated to assist navigation of visually impaired people.



Cinque Terre

Stair Detection

The dense depth image is obtained with infrared image large-scale matching under the assistance of both natural and structured light. Stairs are detected using algorithms including surface normal vector estimation, plane clustering and histogram segmentation. Distances, heights and directions of several nearest steps of the stairs are output.


Download datasets and individual papers

Download