# Learning Rotated bounding box for object detection

I have checked out many methods and paper like yolo, ssd, etc with very promising result in detecting a rectangular box around object, But could not find any paper, which shows an learning a rotated bounding box. Is it difficult to learn the rotated bounding box for an object?

For example for this object Src, its bounding box should be of the same shape(the rotated rectangle shown in 2nd right image), But prediction result for the yolo will be Ist right.

Can somebody refer some paper, by which we can learn such box, or it is an expensive task to learn? Thanks

• The expense is not high, but the benefit isn't high either, which is why most vision systems go straight from rectangles without rotation and edges detected within them to 4d contour detection (h,v,d,t) and then to object motion. Although the tradition in computing is to break large problems down to smaller ones and doing those smaller ones well, artificial networks in combination with physical models seem to be the most efficient and that may be why the visual systems in animals don't calculate trig digitally. – han_nah_han_ Jan 19 at 5:22

Here's a recent paper that does what you're looking for. It looks like they achieve this simply by adding a couple rotated prior boxes and regressing the angles in between. This is similar to what standard object detectors do in terms of creating a bunch of prior box shapes and regressing the actual sizes.

• That is an expensive task as the question intonated. It works for naval images because things that move through the water are pencil shaped in satellite images, but it is a very narrow domain and not very demanding in terms of object complexity, speed of object movement, or recognition speed requirement. – han_nah_han_ Jan 19 at 5:15

Cartesian Bias and Pipeline Efficiency

You are experiencing a techno-cultural artifact of Cartesian-centric imaging running all the way back to the dawn of coordinate systems. It is the momentum of practice as a consequence of applying Cartesian 2D coordinates to rasterize images appearing at the focal planes of lenses from the dawn of television and the earliest standards of raster based capture and display.

Although some work was done toward adding tilt to bounding rectangles in the late 1990s and since, from a time and computing resource conservation perspective, it is computationally and programmatically less costly to include the four useless triangles of pixels and keep the bounding box orthogonal with the pixel grid.

Adding a tilt angle to the bounding boxes is marginally competitive when detecting ships from a satellite only because two conditions offset the inefficiencies in that narrow domain. The ship appears as an oblong rectangle with rounded corners from a satellite positioned in geosynchronous orbit. In the general case, adding a tilt angle can slow recognition significantly.

Biology Less Biased

An interesting side note is that the neural networks of animal and human vision systems do not have that Cartesian-centricity, but that doesn't help this question's solution, since non-orthogonal hardware and software is virtually nonexistent.

Early Non-Cartesian Research and Today's Rasterization

Gerber Scientific Techonology research and development in the 1980s (South Windsor, Connecticut, U.S.) investigated vector capture, storage, and display, but the R&D was not financially sustainable for a mid-side technology corporation for the reasons above.

What remains, because it is economically viable and necessary from an animation point of view, is rasterization on the end of the system that converts vector models into frames of pixels. We see this in on the rendering SVG, VRML, and the original intent of CUDA cores and other hardware rendering acceleration strategies and architectures.

On the object and action recognition side, the support of vector models directly from imaging is much less developed. This has not been a major stumbling block for computer vision because the wasted pixels at one tilt angle may be of central importance at another tilt angle, so there are no actual wasted input pixels if the centering of key scene elements is widely distributed in translation and tilt, which is often the case in real life (although not so much in hygienically pre-processed datasets).

Conventions Around Object Minus Camera Tilt and Skew from Parallax

Once edge detection, interior-versus-exterior, and 3D solid recognition come into play, the design of CNN pipelines and the way kernels can do radial transformation without actually requiring $$\; \sin, \, \cos, \, \text{and} \, \arctan \;$$ functions evaporate the computational burden of the Cartesian nature of pixel tensors. The end result is that the bounding box being orthogonal to the image frame is not as problematic as it initially appears. Efforts to conserve the four triangles of pixels and pre-process orientation is often a wasted effort by a gross margin.

Summary

The bottom line is that efforts to produce vector recognition from roster inputs have been significantly inferior in terms of resource and wait time burden, with the exception of insignificant gains in the narrow domain of naval reconnaissance satellite images. Trigonometry is expensive, but convolution kernels, especially now that they are moving from software into hardware accelerated computing paths in VLSI, is computable at lower costs.

Past and Current Work

Below is some work that deals with tilting with regard to objects and the effects of parallax in relation to the Cartesian coordinate system of the raster representation. Most of the work has to do with recognizing 3D objects in a 3D coordinate system to project trajectories and pilot or drive vehicles rationally on the basis of Newtonian mechanics.

Efficient Collision Detection Using Bounding Volume Hierarchies of k-DOPs, James T. Klosowski, Martin Held, Joseph S.B. Mitchell, Henry Sowizral, and Karel Zikan, 1998

Sliding Shapes for 3D Object Detection in Depth Images, Shuran Song and Jianxiong Xiao, 2014

Amodal Completion and Size Constancy in Natural Scenes, Abhishek Kar, Shubham Tulsiani, Joao Carreira and Jitendra Malik, 2015

HMD Vision-based Teleoperating UGV and UAV for Hostile Environment using Deep Learning, Abhishek Sawarkar1, Vishal Chaudhari, Rahul Chavan, Varun Zope, Akshay Budale and Faruk Kazi, 2016

Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds, Z Liu, H Wang, L Weng, Y Yang, 2016

3D Pose Regression using Convolutional Neural Networks, Siddharth Mahendran, 2017

Aerial Target Tracking Algorithm Based on Faster R-CNN Combined with Frame Differencing, Yurong Yang, Huajun Gong, Xinhua Wang and Peng Sun, 2017

A Semi-Automatic 2D solution for Vehicle Speed Estimation from Monocular Videos, Amit Kumar, Pirazh Khorramshahi, Wei-An Lin, Prithviraj Dhar, Jun-Cheng Chen, Rama Chellappa, 2018

• Thanks for sharing valuable insight into it. – Ankish Bansal Jan 15 at 6:04