In recent years if you are working on stereo depth/disparity algorithms, it seems like you will only ever get your paper accepted to CVPR/ICCV/ECCV if there's some deep learning involved in it. A lot of authors published their code on github and I've tried out multiple of them and here is what I observed. None of these deep learning based methods generalized well. Almost all methods trained on the KITTI dataset (street images) or the scene flow dataset (synthetic images). These methods perform well when the test data is similar to the training data, but fails miserably on other kinds of test data (e.g. close up human) whereas a classical traditional computer vision based method like PatchMatch would generate decent results. In my opinion, no matter how well these new deep learning methods perform on the KITTI benchmark, it's nearly useless in the real world.

I understand deep learning has the potential to approximate any non-linear function when there's enough quality training data and unlimited computation, but ground truth depth/disparity cannot be labeled by manual labor like a cat-dog classification problem. That means the ground truth training data has to come from traditional computer vision algorithms or hardware or be synthetic. Traditional computer vision algorithms are not even close to perfect yet but the research pretty much stifled because of deep learning. The ground truth of the KITTI dataset comes from a hardware LIDAR, but it's extremely sparse. If we align multiple scans from LIDAR in order to form a dense result, that's relying on some type of SLAM which again is relying on an imperfect traditional computer vision algorithm. There is no sign of hardware that can generate accurate dense depth that is coming out soon. As for synthetic data, it doesn't accurately represent real data. Since there isn't even a good way to obtain training data for stereo depth/disparity, why are the researchers so fixated on building complex deep neural nets to solve stereo depth/disparity nowadays?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.