Sites Inria

Version française


Jean-Michel Prima - 6/04/2016

Model-based 3D Tracking for Aerial Robotics

A software meant to ascertain the pose of an object with respect to a camera, Sinatrack can help autonomous vision-guided robots to perform navigation and manipulation tasks. As such, this technology is of particular interest to aeronautics and aereospace where a surprising variety of applications are being considered.

We have been working on this topic ever since 2002, recounts Éric Marchand, a scientist with Lagadic, a research team that specializes in computer vision for robotics at Inria research center in Rennes, Brittany, France. Determining the pose of an object relative to a camera was something that we absolutely needed if we were to carry on our various projects in vision-based robotics. ”  In essence, visual tracking relies on an a priori knowledge: “If one knows the 3D model of an observed object, one can project this model on the image itself, and minimize the error between the projection of the model and the actual position of the object. When they match, one can infere the precise position of this object with respect to the camera, or the position of the camera relative to the object.
A first version of the software has been used in a number of R&D projects, in particular in the field of augmented reality, micro and nano robotics,  astronautics and aeronautics. “For instance, we have worked on an application whereby visual tracking —and servoing— would enable an aircraft to land autonomously on the deck of a carrier. In such context, it is necessary to compute the pose of the plane relative to the ship. ” Using one of the plane's cameras, the tracker first detects the vessel and then projects a model of the deck on the current image. “When both match, the pose between the camera and the carrier is well ascertained. In summary, it's all about minimizing the error between the projected model and the observed image. ”  Such a technology may well pave the way to the autonomous vision-based arrested landing of future carrier-based Unmanned Air Vehicles (UAVs). This research was supported by French defense procurement agency DGA and Dassault Aviation throught a Ph.D. grant.

Innovative Approach  to Autonomous Landing

Remarkably enough, vision-based automated landing could also prove useful to commercial aviation in some situations. Major airports are equipped with Instrument Landing Systems (ILS) meant to provide assistance to the pilots. However, in the event of an emergency landing at the nearest alternate runway, such ground facilities are not always available. Hence the virtue of an automated solution based on airborne visual tracking of the approach path and the strip. In this context, Inria has partnered with Airbus in Visioland, a scientific project funded by French research agency ANR with aim of exploring the best methods to achieve such visual guidance and thus ratchet up safety one more notch.
Sponsored by the EADS Foundation, the scientists also took a keen interest in the thorny problem of space debris removal. From spent rocket stages to defunct satellites, a lot of trash has accumulated over the years in Earth orbit. This pollution increases the risk of collision with active satellites. Space agencies like Nasa and ESA are mulling over the possibility of launching automated space cleaners that would capture these debris, deorbit them and force them to a self-destructive reentry into the atmosphere. Visual tracking could greatly facilitate the work of such autonomous service robots. Indeed, not only would it help the vehicle to rendezvous with a targeted debris but it would also enable a robotic arm to grab it. “The first version of our tool wasn't up to the task, Marchand explains, for it used polyhedron models whereas the 3D models of these debris are way more complex. Therefore we had to rethink the approach so that it could take into account all kind of complex 3D models. ”  This enhanced version, proposed during the Ph.D thesis of Antoine Petit, is far more generic and versatile than the previous one. It is called Sinatrack. It will be used by the Lagadic team in the EU REMOVEDEBRIS project.

Asteroid Deflection

In the future, the software might also play a role in the protection against asteroids known to be on a collision course with Earth. Sinatrack has been licensed to Airbus Space and Defence to be used within EU NEOShield project (led by Astrium). Neoshield is a project aimed at finding the most efficient techniques for deflecting those hazardous bodies. Visual tracking is being considered as a possible modality for guiding the future so-called kinetic impactors that will ultimately crash into the imbound space rocks. “In its latest version, our software can track natural objects provided a 3D model is available. Using our software, Airbus D&S, has been able to conduct experiments with the model of Itokawa, an asteroid that was once scanned by Japanese mission Hayabusa 1.
With the rise of the drone industry, demand for computer vision solutions is bound to increase dramatically. Nevertheless visual tracking won't be restricted to the realm of aerial robotics. “Augmented reality is obviously the other application domain that could benefit from our methodologies, Marchand opines. As a matter of fact, pretty soon from now, we hope to complete a technology transfer in that field as well.

Keywords: Robot Aeronotic Eric Marchand Vision Software INRIA Rennes - Bretagne Atlantique Lagadic Sinatrack