CSIRO’s Data61’s recently launched revolutionary Hovermap™ drone payload dramatically improves the value of drone-based asset inspection and mapping. This technology delivers new, never seen before capabilities to customers in industries such as mining and telecommunications. With the ever-increasing need for automation in challenging industrial settings, Hovermap delivers fully autonomous inspection and mapping capabilities that are unsurpassed in accuracy and ease of use. In the mining industry Hovermap provides mapping of drives and stopes, a particularly hazardous activity for humans, increasing safety and reducing time for mapping. In the telecommunications area Hovermap dramatically reduces the time and cost for maintenance inspections of the burgeoning number of cell towers in use today. By using a single lidar and advanced algorithms to provide SLAM-based mapping, omni-directional collision avoidance, GPS-denied flight and advanced autonomy, Hovermap enables drones to autonomously explore and map complex 3D environments even where GPS is unavailable. Valuable data that would otherwise be impossible to collect can therefore be collected in a safe, repeatable and efficient way. The core technology for Hovermap has been developed in Australia at the Commonwealth Scientific and Industrial Organization (CSIRO) by a team of world-leading researchers in SLAM and drone autonomy. The development has been backed by CSIRO’s Data61 business unit and the CSIRO ON accelerator. The team already has eight participants in the early adopter program – Insitu Pacific, Smart Survey, NQ UAV and The Little Ripper Group. They are successfully using Hovermap for mapping and inspection of Oil and Gas infrastructure, telecommunication towers, construction sites, underground and open-cut mines and railway tunnels and bridges. For all these applications, autonomy and GPS-denied flight will be essential or add significant value. The program is providing vital learnings for us as they prepare to commercialise Hovermap. It is helping them to validate the product-market fit and learn where Hovermap provides the greatest value. Hovermap produces incredibly dense, accurate point clouds which allows us to measure the antenna pointing direction within the required 1 degree in elevation and azimuth. The autonomy and collision avoidance provided by Hovermap in these environments is critical to ensuring mission success. They have already put Hovermap to work by conducting underground mapping in a mine stope which is believed to be a world first. AMSJ caught up with Stefan Hrabar to explain further about Hovermap.
“CSIRO’s Data61 Robotics Group based in Brisbane has been developing autonomy for drones for more than 10 years. Previously the work we’ve done has been on petrol powered mini-helicopters before electric multi-rotors were available,” he said.
“We’ve demonstrated a number of world-firsts in beyond line-of-sight autonomy, using stereo cameras, lasers and radar to sense the environment around the drone and avoid the risk of collision in these environments.”
“In parallel, another team in our group developed a robust way of using 3D lasers to replace the need for GPS, enabling ground-based robots to navigate indoor spaces and other areas where GPS is not available. The same technique produces an accurate 3D map of the environment, almost as a by-product,” Stefan explained.
“The technique is called Simultaneous Localisation and Mapping or SLAM. We realised that the maps generated were useful to people, not just robots, so we commercialised a hand-held device that could be used to map building interiors and other environments.” (Zebedee, commercialised by GeoSLAM).
The next obvious step was to combine the drone autonomy work with the laser mapping work and this led to the development of Hovermap. The team used the same sensor (a laser or LiDAR) for both mapping and autonomy on a drone. It allowed the drone to detect and avoid obstacles around it, navigate without GPS and produce accurate 3D maps of the environment that it travels through. This is important because it allows drones to fly safely in challenging environments even where GPS is unavailable, to collect valuable data that is otherwise impossible to collect.
Stefan says the research involved drone autonomy and SLAM-based LiDAR mapping, and the autonomy comprised of collision avoidance, GPS-denied flight and high-level autonomous mission execution. The aim with the element of autonomy is to enable a drone to perform a mission without an expert operator telling it exactly what to do.
For example, to map and inspect a telecom tower, the user places the drone on the ground and flips a switch. The drone takes off, finds the tower and flies around it in the most optimal way, to map and inspect it. For underground mining the aim is to have a surveyor place the drone near a stope entrance and press go. The user should not need to be a skilled drone operator.
The drone autonomy research at CSIRO began in 2006 and has been ongoing for more than 10 years. The SLAM-based mapping and navigation work has been ongoing for about 8 years. The Hovermap-specific work started in 2015.
How does Hovermap work?
“Hovermap is LiDAR mapping and autonomy payload which is mounted to a drone to give the drone advanced capabilities,” Stefan explains. “It uses a rotating LiDAR (Light Detection and Ranging) which sends out 300 000 laser pulses per second in all directions. It measures the time that the pulses take to reflect off the environment and return to the receiver, using this to calculate the distance that each pulse travelled. It can measure distances up to 100m.”
“This is the same kind of device that is used in many driverless cars. We haven’t developed the LiDAR unit – this is a COTS item which we integrate into the Hovermap payload. Our focus is on the algorithms which use the LiDAR data. The payload also includes an embedded computer to process the LiDAR data and generate control commands for the drone. The LiDAR measurements are used to identify features in the environment again and again as the drone moves.”
By tracking these features, the SLAM algorithm can estimate how the drone is moving relative to them. This can be used instead of GPS to figure out how the drone is moving and allow it to navigate around these environmental obstructions.
“Because we know how the drone (and the LiDAR) is moving, we can project all the LiDAR measurements into a common coordinate frame to generate a 3D map or point cloud,” he says.
A local snapshot of this point cloud is used to generate a map of obstacles around the drone, and this obstacle map is used by the on-board flight controller to avoid collisions with the obstacles. The LiDAR data is also stored on-board and after the flight, the SLAM algorithm is used to generate a 3D point cloud from all the data that was collected.
Although they are still developing the full autonomy needed for the underground application, now the drone can fly autonomously to waypoints without using GPS and while avoiding collisions.
“This requires building a cursory map of an environment first and using this map to place the waypoints, Stefan says. “To build the cursory map, an operator flies the drone within line-of-sight, but with assistance from the Hovermap payload. Hovermap keeps the drone safe by forming a ‘Virtual Elliptical Shield’ or VESH around it. If the operator tries to fly to close to an obstacle the VESH will resist.
Hovermap also helps by providing position hold and stability even when no GPS is available. Usually a drone will drift off position without GPS if an operator is not actively controlling it. Hovermap allows the drone to hover in place without input from the operator when GPS is not available.
It’s these two features that make it possible for an operator to guide the drone around underground to build the cursory map. Once the map is built, waypoints can be placed at the map extremities in areas that are beyond where the operator can see. In the next autonomous flight, the drone can therefore travel beyond line-of-sight to map out further portions of the mine. This is an intermediate step but is still very useful.
“We are now developing the autonomous exploration capability which will remove the need for the cursory mapping flight. Instead, the drone will explore new areas autonomously until it has mapped the area completely; or is running low on battery power”.
This kind of technology is a breakthrough for the mining industry and should benefit it greatly. Hovermap will allow drones to fly autonomously into inaccessible parts of an underground mine to produce 3D maps and collect other data that would otherwise be impossible to collect. RGB cameras, thermal cameras and gas sensors could be carried by the drone into these areas.
“Mapping and surveying of underground mines can be a challenging and dangerous task. In many cases it simply isn’t possible to map certain areas because of access issues, “This can have severe and costly consequences for the mine as it leads to sub-optimal mine planning, “he says.
“An example is mapping of stopes. These are the cavities formed after blasting underground and removing the blasted material. After emptying a stope, it is necessary to map it for volumetric calculations and to compare the shape to the designed shape. This reconciliation process is needed to check if valuable ore has been left in the ground (under breaking) or if waste rock has been blasted out with the ore which dilutes the ore grade (over breaking).”
The traditional way of mapping a stope is with a CMS or Cavity Measurement System. A LiDAR attached to a boom is inserted into the stope entrance as far as possible. In many cases the CMS will only partially map the stope because of shadowing. The point density is limited and varies with distance from the scanner. These systems also require surveyors to enter hazardous areas near the stope entrance. Surveyors have been killed by falling rocks or by falling into the stope.
A near miss occurred in 2015 in WA when safety breaches saw a worker nearly reverse his ute into a 19-metre deep pit and was left teetering on the edge of the open stope for over an hour.
To overcome these types of issues, Hovermap allows the drone to fly into stopes and other inaccessible areas to explore and map them. Because the drone flies into the stope it generates a map with complete coverage and no shadowing.
“Beyond basic volumetrics, the resulting point density also enables identification of faults and other geological features. This can help the mine planners to have a better idea of the geology and to better plan the mining process.
Point clouds with this level of detail and coverage have never been available before so we’re working with the miners to understand what additional value can be extracted from them. They are extremely excited about the data samples that we’ve collected during our trials” he explains.
“At times a blockage can occur which prevents the fragmented rocks from falling down into the draw point where they are removed. It would be very useful to be able to fly in and map and image the blockage so that the cause and location are known. Plans can then be made to drill in and blast the blockage.”
By mapping the fragmented rocks in a stope or at the draw point it is possible to estimate how big the rock fragments are and what the range in fragment size is. The fragmented rocks are put through a series of crushers and mills to reduce their size and eventually produce fine material which can be processed to extract the minerals. The crushers need to be set up for certain fragment sizes so knowing before-hand what fragment sizes are on their way to the crushers can help to optimise the process. The optimisations in the ‘mine to the mill’ process can be very valuable to the mines and increase productivity.
Another use-case is exploring and mapping of old workings. These are parts of a mine that have been mined previously and have been left inactive for a long time. As mineral extraction techniques become more efficient and commodity prices rise it becomes financially viable to revisit these areas to extract more ore even if the grade is lower. In many cases the old workings are unsafe for humans to enter as they have not been stabilised for a long time. An autonomous drone could be sent in to map the old workings and collect images which can be used to assess the stability of the area before sending people in, thus increasing safety and minimising risk to human workers.
The same capability could be used for Search and Rescue operations. The drone could be sent in to search for survivors after a collapse or to check if it is safe to send rescuers in. By carrying a gas sensor, the drone could also check the air quality after an incident, or routinely after blasting. Instead of waiting an hour or so on the surface after blasting before sending miners back into a mine, the drone could fly in and check the air quality, allowing miners to return as soon as it is safe.
Any down time in a mine is extremely expensive so technology that can help reduce time is valuable to the mining industry.
Hovermap will be commercialised by CSIRO’s Data61 either through a new spin-out company or by licensing out the technology. These options are currently being explored.
“We expect to launch Hovermap commercially in Q2 of 2018. We currently have an early adopter program running with eight participants. These organisations are beta testing Hovermap for a variety of applications include mining,” Stefan says. “We’ve also been conducting underground trials in partnership with mines (South32’s Cannington mine in QLD and Northern Star Resources’ Jundee mine in WA). At Cannington last July we demonstrated the world’s first autonomous drone flight into a stope. At Jundee in September we demonstrated the world’s first beyond line-of-sight drone flight into a stope.”
Stefan says that based on the learnings from these trials and the early adopters, the team will be able to fine-tune the Hovermap hardware and the autonomy algorithms to ensure it is ready for commercialisation later this year.
Add Comment