Pedestrian detection is a critical feature of autonomous vehicle or advanced driver assistance system. We developed a novel instrument for pedestrian detection by combining stereo vision cameras with a thermal camera. A new dataset for vehicle applications is built from the test vehicle recorded data when driving on city roads. Data received from multiple cameras are aligned using trifocal tensor with pre-calibrated parameters. The input to the detector can be the color image, disparity map, thermal data, or any of their combinations. Machine learned based algorithm is employed to process the features from multiple of inputs. The evaluation results show that it significantly outperforms the traditional computer vision features. The proposed pedestrian detector with multi-spectral cameras can achieve 9% log-average miss rate. Contact us if you are interested in acquiring our dataset.

Click to view the result in Youtube