Uncertainty estimation for adaptable and reliable robotic vision

Throughout the past decade, neural network-based vision has been increasingly integrated into robots. It is well-known, however, that neural networks can be unreliable, especially when faced with inputs that differ from their training data. With the goal of making neural network-based vision more reliable for robotic applications, this thesis explores uncertainty estimation, and adaptability.

Adaptability is explored in terms of the cameras themselves, and it is argued that their built-in parameters form a suitable basis for adapting to varying and difficult visual scenes. An optimization problem is formulated, to adjust camera parameters with the goal of minimizing uncertainty.

Depending on the application, it can be beneficial to express uncertainty across local regions in an image. A gradient-based method is proposed based on normalizing flows, which provides uncertainty estimates at the pixel-level.

The notion of adaptability based on uncertainty is further applied to visuomotor policy learning. A normalizing flow is used to directly produce fine-grained control sequences for a dual-arm robot, and its uncertainty estimate is used to improve the overall quality of generated sequences.

Calibration for regression problems is non-trivial. Several different calilbration metrics are used in literature, but they are lacking proper analysis. Such an analysis is provided by the use of toy datasets, to gain insight into what they measure, and whether they are stable estimates.

Finally, a combined uncertainty estimate is proposed based on softmax and normalizing flows, that aims to mimic the uncertainty estimate from Gaussian processes. This proposed uncertainty estimate is shown to perform well in the problem of selective classification in the presence of out-of-distribution data.