Vision Transformers for Dense Prediction (ICCV 2021): State of the art accuracy on depth estimation and semantic segmentation (realtime >30 FPS)

Vision Transformers for Dense Prediction: 

(ICCV 2021)

State of the art accuracy on depth estimation and semantic segmentation (realtime >30 FPS)


We introduce dense vision transformers, an architecture that leverages vision transformers in place of convolutional networks as a backbone for dense prediction tasks. We assemble tokens from various stages of the vision transformer into image-like representations at various resolutions and progressively combine them into full-resolution predictions using a convolutional decoder. The transformer backbone processes representations at a constant and relatively high resolution and has a global receptive field at every stage. These properties allow the dense vision transformer to provide finer-grained and more globally coherent predictions when compared to fully-convolutional networks. Our experiments show that this architecture yields substantial improvements on dense prediction tasks, especially when a large amount of training data is available. For monocular depth estimation, we observe an improvement of up to 28% in relative performance when compared to a state-of-the-art fully-convolutional network. When applied to semantic segmentation, dense vision transformers set a new state of the art on ADE20K with 49.02% mIoU. We further show that the architecture can be fine-tuned on smaller datasets such as NYUv2, KITTI, and Pascal Context where it also sets the new state of the art. Our models are available at this https URL https://github.com/intel-isl/DPT 

Comments

Post a Comment

Popular posts from this blog

C++: Thread-safe std::map with the speed of lock-free map

Image
Introduction:  Examples of use and testing of a thread-safe pointer and contention-free shared-mutex.  In this article we will show additional optimizations, examples of use and testing of a thread-safe pointer developed by us with an optimized shared mutex  contfree_safe_ptr<T>  - this is equivalent to  safe_ptr<T, contention_free_shared_mutex<>> In the end, we will show the comparative graphs of our thread-safe-pointer tests and some of the best lock-free algorithms from libCDS on Intel Core i5 / i7, Xeon, 2 x Xeon processors. You can find libCDS by the link:  https://github.com/khizmax/libcds In all benchmarks in this article used this commit of libCDS   https://github.com/khizmax/libcds/tree/5e2a9854b0a8625183818eb8ad91e5511fed5898 Read full  article :   https://www.codeproject.com/Articles/1183446/Thread-safe-std-map-with-the-speed-of-lock-free-ma Three related articles:   1.  We make any object thread-safe   2.  We make a std
Read more

C++: We make any object thread-safe

Introduction:  Smart pointer that makes any object thread-safe for any operations, with the performance equal to that of optimized lock-free containers. In these 3 articles I’ll tell in detail about atomic operations, memory barriers and the rapid exchange of data between threads, as well as about the “sequence-points” by the example of “execute-around-idiom”. At the same time, we will try to do something useful together. There are no thread-safe containers (array, list, map ...) in the standard C++ library, which could be used in multiple threads without additional locks. In case of usage of standard containers for multithreading exchange, there is a danger for you to forget to protect one of the code sections with mutex or by mistake to protect it with another mutex. Obviously, a developer will make more mistakes if he/she uses his/her own solution instead of a standard one. And if the task is so complex that there is no any standard solution, then the developer, while
Read more