Project illustration

Loading...


video Presentation


Indoor Infrared Stereo

Outdoor Infrared Stereo



Indoor Visible Light Stereo

Outdoor Visible Light Stereo


Project Function


Cinque Terre

Obstacle Detection

Based on the infrared stereo camera, large scale stereo matching is conducted with the aid of natural light and structured light to obtain depth maps. Using Mean-Shift algorithm, the distances, directions and sizes of the nearest obstacles are calculated and mapped to stereo sound to assist the navigation of the visually impaired.



Cinque Terre

Ground detection

Based on the infrared stereo camera, large scale stereo matching and color image guided filtering are conducted with the aid of natural light and structured light to complete depth maps. Using random sampling and consensus, surface normal vector estimation and seeded region growing algorithm, the ground areas are detected and mapped to stereo sound to assist the navigation of the visually impaired.



Cinque Terre

Banknote identification

Banknotes identification is one of the designed functions to provide convenient living based on the survey of the needs of visually impaired people. Currently, the banknote identification method is to conduct depth filtering with RGB-D camera, to frame the detected potential banknote regions with a classifier and to determine if there is a banknote and its denomination through SURF feature point detection. This method extract banknote region and provide its denomination in real time for complex environment with good robustness.



Cinque Terre

Pedestrian crossing light detection

Pedestrian crossing light detection algorithm is based on the color extraction and machine learning. It accurately detect the location of the traffic lights in the image in real time and give the status of the traffic lights. As shown in the figure, the corresponding color of the box selected traffic lights is the detection result.



Cinque Terre

Crosswalk detection

To cross the road safely is the urgent need for blind people. The crosswalk detection algorithm based on the strip extraction and clustereing is used for blind people in traffic intersections to find and locate crosswalks. As shown in the figure, the locations and directions of the crosswalk are identified and converted to prompt messages to the user.



Cinque Terre

Face Recognition

Face recognition in the intelligent visual assistance for the blind is designed to help visually impaired people to perceive and identify friends, relatives, strangers and so on in the scenes of daily life. Face recognition system uses color depth information fusion, target tracking and neural network technology. When a visually impaired user is using the intelligent assisting device, the face recognition system gradually collects and learns the faces that users often meet. When the recognition system rearches enough traning level, the face that apperars in the scene is identified and the recogtion result is transferred to the visually impaired user through a specific interactive way.



Cinque Terre

Traversable Area Detection

Depth image is enhanced with semi-global matching and color image guided filtering based on RGB-D camera. Using dynamic programming algorithm, traversable areas are detected in the random occupancy grid. The spline surface is utilized to fit the ground area and obstacles are represented with stixels. Stereo sound feedback is generated to assist navigation of visually impaired people.



Cinque Terre

Water Hazards Detection

Based on polarized RGB-Depth (pRGB-D) camera, depth image is enhanced with semi-global matching and color image guided filtering. With polarization-color-depth-attitude information, traversable area and water hazards are detected simultaneously through dynamic programming. Stereo sound feedback is generated to assist navigation of visually impaired people.



Cinque Terre

The Stairs Detection

Dense depth image is obtained with infrared image large-scale matching under the assistance of both natural and structured light. Stairs are detected using algorithms including surface normal vector estimation, plane clustering and histogram segmentation. Distances, heights and directions of several nearest steps of the stairs are output.



Image Data Sets


RGB-D-IR Image Set

Can use the color depth - infrared data for training

Download

pRGB-D dataset

You can use the polarization - color - depth data for training

Download

PTLR dataset

You can use PTLR data set for training and validation

Download

crosswalk DATASET

Training data set can be used to the crossroads

Download

Terrain Awareness Dataset

Terrain Awareness Dataset could be used for training, validation and testing

Download

VLdataset Dataset

Color-depth-infrared-GNSS data used for visual localization of scenes

Download


Team Graduation Thesis


Dongming Sun 2017

To enhance the resolution of 3D sound.

Download

Tianxue Chen 2017

3D reconstruction and recognition based on RGB-D images.

Download

Xi Yang 2017

Object recognition algorithm based on deep learning.

Download

Xinxin Hu 2017

Indoor visual localization and navigation (including ppt).

Download

Yihe Feng 2017

An interactive system design of face recognition (including ppt).

Download

Zhenyu Tang 2017

A method to obtain geometry and material information of objects based on differential motion theory under uncontrolled condition.

Download

Chen Xu 2016

Online banknote identification based on neural network (including ppt and program files).

Download

Chen Hao 2016

Short-range obstacle warning based on depth sensor (including ppt).

Download

Weijian Hu 2016

Visual human-computer interaction technique based on depth-sound mapping.

Download

Zhiyuan Wang 2016

Pedestrian detection based on deep learning.

Download

Ruiqi Cheng 2015

Visual assistance technology for the blind based on 3D terrain detection.

Download

Xiangdong Zhao 2015

Human-computer interface system design in the visual assistance for the blind.

Download

Linquan Chen 2015

The pattern recognition technique in the visual assistance for the blind.

Download