Camera relocalization has various applications in autonomous driving. Previous camera pose regression models consider only ideal scenarios where there is little environmental perturbation. To deal with challenging driving environments that may have changing seasons, weather, illumination, and the presence of unstable objects, we propose RobustLoc, which derives its robustness against perturbations from neural differential equations. Our model uses a convolutional neural network to extract feature maps from multi-view images, a robust neural differential equation diffusion block module to diffuse information interactively, and a branched pose decoder with multi-layer training to estimate the vehicle poses. Experiments demonstrate that RobustLoc surpasses current state-of-the-art camera pose regression models and achieves robust performance in various environments.

RobustLoc: Robust Camera Pose Regression in Challenging Driving Environments

This paper considers the problem of cooperative localization of multiple robots under uncertainty, communicating over a partially connected, dynamic communication network and assisted by an agile landmark. Each robot owns an IMU and a relative pose sensing suite, which can get faulty due to system or environmental uncertainty, and therefore exhibit large bias in their estimation output. For the robots to localize accurately under sensor failure and system or environmental uncertainty, a novel Distributed Learning based Decentralized Cooperative Localization (DL-DCL) algorithm is proposed that involves real-time learning of an information fusion strategy by each robot for combining pose estimates from its own sensors as well as from those of its neighboring robots, and utilizing the moving landmark's pose information as a feedback to the learning process. Convergence analysis shows that the learning process converges exponentially under certain reasonable assumptions. Simulations involving sensor failures inducing around 40-60 times increase in the nominal bias show DL-DCL's estimation performance to be approximately 40% better than the well-known covariance-based estimate fusion methods. For the evaluation of DL-DCL's implementability and fault-tolerance capability in practice, a high-fidelity simulation is carried out in Gazebo with ROS2.

Moving-Landmark assisted Distributed Learning based Decentralized Cooperative Localization (DL-DCL) with Fault Tolerance

In robust optimization, finding a solution that solely respects the constraints is not enough.
Usually, the uncertainty and unknown parameters of the model are represented by random variables.
In such conditions, a good solution is a solution robust to most-likely assignments of these random variables.
The Confidence constraint allows us to enforce this type of robustness in constraint programming.
In this paper, we generalize the Confidence constraint and introduce the Multi-valued Decision Diagram (MDD) confidence. 
The MDD Confidence constraint is defined over a set of random variables. 
Then, for a given constraint represented as an MDD, and given a threshold, the generalized Confidence constraint ensures that the probability for the constraint to be respected by a sample of the variables is greater than the threshold.
On the one hand, constraints that random variables should respect often depend on decision variables.
On the other hand, the MDD represents the constraint that random variables should respect.
That is why this paper proposes several ways to constrain the MDD using decision variables during search.
These constraints are a generalization of the Confidence constraint recently introduced by Mercier-Aubin et al.
In fact, the existing constraint is restricted to a conjunction of binary inequalities.
The proposed generalization can be applied to any constraint, and filtering algorithms are provided using their MDD representation.
Moreover, the original work was restricted to independent random variables, while our generalization can be used with Markov chains, 
allowing a better expressiveness.
Finally, the experimental section shows the efficiency of the proposed methodology on a chemical deliveries problem in factories 
where the chemicals uses are uncertain.

Generalized Confidence Constraints

Predicting high-fidelity future human poses, from a historically observed sequence, is crucial for intelligent robots to interact with humans. Deep end-to-end learning approaches, which typically train a generic pre-trained model on external datasets and then directly apply it to all test samples, emerge as the dominant solution to solve this issue. Despite encouraging progress, they remain non-optimal, as the unique properties (e.g., motion style, rhythm) of a specific sequence cannot be adapted. More generally, once encountering out-of-distributions, the predicted poses tend to be unreliable. Motivated by this observation, we propose a novel test-time adaptation framework that leverages two self-supervised auxiliary tasks to help the primary forecasting network adapt to the test sequence. In the testing phase, our model can adjust the model parameters by several gradient updates to improve the generation quality. However, due to catastrophic forgetting, both auxiliary tasks typically have a low ability to automatically present the desired positive incentives for the final prediction performance. For this reason, we also propose a meta-auxiliary learning scheme for better adaptation. Extensive experiments show that the proposed approach achieves higher accuracy and more realistic visualization.

Meta-Auxiliary Learning for Adaptive Human Pose Prediction

The presentation proposes SixPack v2, an enhanced version of the SixPack attack that allows to evade even state-of- the-art misbehavior detection systems. As the original SixPack, SixPack v2 is a dynamic attack targeting other C-ITS entities by simulating the sudden activation of the braking system with consequent activation of the Anti-lock Braking System. SixPack v2 achieves better evasion by improving the main phases of the attack (FakeBrake, Recovery, and Rejoin) through a novel path-reconstruction algorithm that generates a more realistic representation of the real vehicle trajectory.

SixPack v2: enhancing SixPack to avoid last generation misbehavior detectors in VANETs

Last year Elon Musk posted in his Twitter that he had realized that self-driving cars are a "hard problem". We believe that enabling communication between the vehicles is an essential necessary step for cracking this problem in context of fully autonomous urban driving. We will share some of our recent research results on autonomous vehicles with a focus on inter-vehicular networking and respective cooperative driving functionalities. Platooning, i.e. an automatic following of wirelessly connected vehicles closely behind each other, will be presented slightly deeper. We will explain the approach of assessing the safety of the platooning functionality by coupling the quality of radio communications to the likelihood of a rear-end collision. We gratefully acknowledge the support from the Swedish Knowledge Foundation (KKS) in the framework of ”Safety of Connected Intelligent Vehicles in Smart Cities – SafeSmart” project (2019–2023), the Swedish Innovation Agency (VINNOVA) in the framework of ”Emergency Vehicle Traffic Light Preemption in Cities – EPIC” project (2020–2022) and the ELLIIT Strategic Research Network.

Autonomous Vehicles: Wireless Networking for Cooperative Maneuvering

The technical problems for making it possible for self-driving vehicles to drive together with human driven vehicles has turned out to be more difficult than expected for some 5-10 years ago. In this talk we will discuss some of the problems that causing this delay and some approaches on how these problems can be handled. From a decision and control perspective we identify the need to predict other road users, and to understand their intentions, so that safe control action ca be taken. To have safe and efficient traffic, uncertainties of the prediction must be small. Approaches how this can be obtained will be discussed, including infrastructure, methods for predicting other road users, and algorithms limiting the uncertainty. Approaches we work on in academic projects, and with industrial partners will be highlighted.

Self-driving Vehicles, Can They Be Safe in Mixed Traffic?

Autonomous driving is one of the most exciting engineering fields of our era. The benefits that self-driving cars will have in our society are still unmeasurable, while the associated goals are also growing increasingly complex and challenging. In recent years, fully supervised Deep Learning (DL) algorithms have seen an unprecedented boom in this field and are progressively being introduced into autonomous vehicle navigation architectures. However, compared to classical methods, supervised DL-based techniques face scalability issues as they require huge amounts of labeled data and are unable to generalize to multiple domains. These issues represent a great barrier for the practical application of DL techniques in this field.In this talk, we will revise the DL developments made by our Research Lab for our Autonomous Vehicle prototype surrounding alternatives to fully supervised learning in the DL context (synthetic data, transfer learning, reinforcement learning, etc.). In addition, we will present our thoughts on the future of research in this field with the purpose of solving the challenging remaining problems.

Beyond Supervised Deep Learning for Autonomous Driving

The development and testing of autonomous vehicles requires a massive edge-to-cloud-to-edge “data loop” that begins & ends with the test fleet. In order to meet customer sprint cycle KPIs and iterate across the data loop quickly & efficiently, Microsoft solved a series technical and logistical challenges at scale across an end to end workflow featuring sensor reprocessing, CI/CD & ML pipeline management, and both perception and post-perception closed loop simulation.The focus of this presentation will be a high-level description of the entire data loop with a focus on the unique challenges of sensor reprocessing (commonly called “re-sim” or “playback”) at a massive scale. Re-sim is simultaneously a logistical challenge, a safety challenge, and a budget challenge. The presentation will explain how Microsoft Azure:• Extracts petabytes of data (daily!) from both stationary & nomadic fleets dispersed globally• Filters, processes, & curates those petabytes of data• Accurately assesses & recreates the real-world performance of a device under test• … and do all of the above quickly and cheaplyThe presentation will conclude with a forward-looking overview of anticipated future challenges Microsoft will need to solve as the technology continues to evolve and customer KPIs grow more sophisticated.

Scaling the Data Loop: Accelerating the Development and Testing of Autonomous Vehicles

Automated Guided Vehicles (AGVs) are widely used for materials transportation. Operating them in a platooned manner has the potential to improve safety, security and efficiency, control overall traffic flow and reduce resource usage. However, the published studies on platooning focus mainly on the design of technical solutions in the context of automotive domain. In this paper we focus on a largely unexplored theme of platooning in production sites transformed to the Industry 4.0, with the aim of providing safety and security assurances. We present an overall approach for a fault- and threat tolerant platooning for materials transportation in production environments. Our functional use cases include the platoon control for collision avoidance, data acquisition and processing in a certain range, and connectivity with fog and cloud levels. To perform the safety and security analyses, the Hazard and Operability (HAZOP) and Threat and Operability (THROP) techniques are used. Based on the results obtained from them, the safety and security requirements are derived for the identification and prevention/mitigation of potential platooning hazards, threats and vulnerabilities. The assurance cases are constructed to show the acceptable safety and security of materials transportation using AGV platooning. We leveraged a simulation-based digital twin for performing the verification and validation as well as finetuning of the platooning strategy. Simulation data is gathered from digital twin to monitor platoon operations, identify unexpected or incorrect behaviour, evaluate the potential implications and trigger control actions to resolve them. The applicability of the AGV platooning is demonstrated in the context of a quarry site.

Safe and secure platooning of automated guided vehicles in Industry 4.0

The Emulation Ground Vehicle (EGV) is a robotic wheeled vehicle designed and manufactured in-house at NAWCAD Lakehurst to be used as an aircraft-representative interface for testing deck handling control systems at Lakehurst’s Advanced Deck Handling Technology Evaluation Center (ADHaTEC). The EGV utilizes dynamic models of existing aircraft along with Model Reference Adaptive Control (MRAC), an adaptive machine learning strategy, to closely emulate the taxiing dynamics of those aircraft. Its size is comparable to that of shipboard aircraft, and it is equipped with functions possessed by those craft on the flight deck, such as wing folding, launch bar actuation, and tail hook actuation. Integration with the Robot Operating System (ROS) streamlines the capability of the EGV to function as a testbed for any kind of taxiing control method for both teleoperation and autonomous navigation, including methods based on machine learning strategies. Successful full scale tests of the vehicle have been performed, clearing a path for future work utilizing the platform. Plans include implementing a Fuzzy Adaptive Resonance Theory (FuzzyART) neural network-based gesture recognition control system that has already been tested on smaller robotic vehicles. Going forward, the EGV could be used to evaluate various machine learning-based control methods for aircraft operating in the shipboard environment before those methods transition to fleet use.

Underline digital video library

RobustLoc: Robust Camera Pose Regression in Challenging Driving Environments

Moving-Landmark assisted Distributed Learning based Decentralized Cooperative Localization (DL-DCL) with Fault Tolerance

Generalized Confidence Constraints

Meta-Auxiliary Learning for Adaptive Human Pose Prediction

SixPack v2: enhancing SixPack to avoid last generation misbehavior detectors in VANETs

Autonomous Vehicles: Wireless Networking for Cooperative Maneuvering

Self-driving Vehicles, Can They Be Safe in Mixed Traffic?

Beyond Supervised Deep Learning for Autonomous Driving

Autonomous Vehicles: Wireless Networking for Cooperative Maneuvering

Scaling the Data Loop: Accelerating the Development and Testing of Autonomous Vehicles

Safe and secure platooning of automated guided vehicles in Industry 4.0

Emulation Ground Vehicle (EGV)

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES