There is no doubt that vehicle operators of heavy equipment have to accomplish difficult tasks. The increase in mine productivity has led to a proportional increase in equipment size. Most vehicles have considerable blind spots, due to scale alone. Even the most experienced operator can be challenged by multiple distractions such as variations in environmental conditions, in-vehicle management systems, navigation systems and radio communications. Despite the best efforts of operators, accidents on mine sites remain an industry-wide problem. The reasons for accidents are diverse, so too are the collision scenarios.
A special report by Nicky Guenther and Holger Salow
Safety & Health Commissioner, Stewart Bell estimates that greater than 50% of mining operations in Queensland are currently trialling or operating Proximity Detection Systems (PDT) on heavy mining vehicles. Sadly, the majority of fatalities in mines are still the result of interactions between humans and mining equipment. The varieties of accidents demand a combination of technologies to assist operators and reduce risk.
Having captured the automotive market in the recent past, advanced driver assistant systems are now on the move to conquer industrial vehicles in mining.
Advances on the road from manned to autonomous mining
Australia is in a unique situation where institutes like CSIRO, CRC Mining (The Australian Government’s Cooperative Research Centre) and the ACFR (Australian Centre for Field Robotics) do a lot of research and functional prototyping for huge automation projects like the “Mine of the future”. Mine company programmes include improvements to mining processes of extracting minerals, reducing environmental impacts and improvements in safety up to fully integrated, automated operations.
One important part of the move to greater autonomy will be the introduction of a fleet of new driverless trucks, loaders, drills, trains and remote controlled equipment. These new technologies aim to improve productivity and safety and will enable employees to work from a place which is distant from the mine site [1]. All the traffic lights are set to “green” for automation technology.
Situational Awareness
Environmental perception technology is the key to reducing the impact human factors can have. The primary goal of environmental perception technology is to provide a set of data that describes the environment sufficiently in a model. This model allows us to track objects and develop an effective strategy to avoid or mitigate the impact of collisions through impaired visibility or loss of control. Modern driver assistant systems like road departure warnings show the potential of semi-automatised driving functions.
“As mining equipment is used in very different environmental conditions and in different tasks, there is no ‘one size fits all’ solution.”
The quality of situational awareness determines the set of actions (or warnings) the system can issue with high confidence. If some conditions, e.g. weather, decrease the quality of situational awareness, the system’s reaction must be appropriate to secure a safe operator (vehicle) reaction in bad road conditions. The most important requirements on a sensor for longitudinal and lateral guidance on mobile equipment are described below. The following tables were developed in collaboration with international mine managers and industry experts during a Mine Automation Summit in South Africa
“Safety & Health Commissioner, Stewart Bell estimates that greater than 50% of mining operations in Queensland are currently trialling or operating Proximity Detection Systems (PDT) on heavy mining vehicles.”
Awareness System Overview
Passive Operator Guidance by means of mirrors and complementary cameras (reverse, side and blind spot) is important for an operator – active Proximity
Early advisory warning (obstacle monitoring zone: only visual, no audible warning)
Detection Systems have to warn when the human fails. The building blocks of any operator assistant system are a number of environmental perception sensors used cooperatively with other additional sources of information like GPS.
Established driver assistant system concepts based on local vehicle sensors have to provide the warning under the adherence of a reaction time. Recognised studies [3] define a suitable time window for “imminent crash warnings” with 700-1500ms before collision. A window of more than 1500ms can therefore be used to provide driver information (called “advisory warning” [3]) to address the potential hazard, catch the operator’s attention and establish readiness to react. Audible operator warnings are aimed to a direct operator reaction. The established timing structure can therefore be augmented in adding an early time component to enhance the effectiveness of the system.
The interpretation of measurements has to be done by the system itself, which warns and indicates possible incidents to react to or to mitigate an impact and to safeguard the persons involved. Qualified obstacle information from these sensors is utilised in a sophisticated warning strategy, which alerts the operator, via a simple user interface to take evasive action. The same technology has been widely used in passenger vehicles to automatically control the vehicle, adapt the speed to a safe distance and to avoid the collision.
Human Machine Interface (HMI)
A driver assistant system exists only with the operator “in the loop”. This fact requires special attention. The operator models’ task (Fig 1) is the situational interpretation and to derive strategies to assist safe operation. The human factor plays an important role in the development of HMI’s. Operator focus must remain on tasks at hand. Unfiltered visualisation of the environmental data inevitably leads to information overload. The operator has to filter and extract the relevant information. This task costs precious time which is not available when a collision is pending. The objective of a warning system must lie in the delivery of information that assists in the current situation.
The system has to decide on timing, modality and content of information [6], which is usually provided by means of visual display and/or audible information. Audible warnings have to be limited to imminent crash warnings. Particularly frequent activation or false alarms will lead to ignoring and even a contrary effect on the operator performance. Further importance is given [6] to the specific direction (location) and probability (risk) of the collision, which can influence the effectiveness of a warning system.
Systems fit for purpose
The operator has to be certain about the system functionality in critical situations and current system limits in case of a component failure. A system limitation has to lead to a clear indication to the operator. It is dangerous to assume that the majority of accidents occur in low speed interactions, if it is a means to obscure the fact that the avoidance and assistance technology applied cannot prevent high speed collisions. It is also dangerous to have a singular focus on high speed haul road scenarios.
The effective combination of technologies is a sensible approach, considering that fatalities and serious injury can occur in all possible scenarios from: a close proximity crush; an operator misjudgement during a precision loading or unloading process; or a microsleep whilst at the controls of a vehicle. Ultimately, people working within the mines bear the costs of such incidents.
Questions have to be asked when deciding for a certain technology, these include amongst other things:
- What are the most critical vehicle interactions in the particular site? For further information see [2] and [3]. This information is usually part of a risk analysis.
- What is involved in an installation (time and effort, service shift is usually the critical limit e.g. for a complete TPS)?
- Warning strategy applicable to a particular mine site (adjustable and dynamic warning zones)
- Does the entire fleet have to be equipped with the technology in order to be effective? Is successive installation possible?
- Layout active warning strategy (audible and visual) – operator’s focus lies on tasks at hand; operator information management (What’s comfort? What’s necessary?)
- Does it provide interface fleet management systems? (application event protocol, event logging)
- Reliability and robustness
- Availability / maintenance – is system self-diagnostics available? Ease of fault identification. Is it suited for onsite service by customer who stocks spare parts?
- Operator acceptance: False alarm and nuisance alarm rate. Is the warning strategy reasonable, annoying or a real help and pays off?
- Is a local and competent service network available?
- Are we prepared to take ownership of this new technology?
Above is an example of a system that fits for purpose with the ability to assist in situations that couldn’t be protected before.
Example SICK Excavator Protection System
Collision warning system and precise loading assist at the same time, on top = standard operation, bottom = too close for loading s Redirect!
Operator assistance in manned underground mining operations
Some of the large mining companies currently invest in the development of block cave mining and fast tunnel development technologies – which allows them to extract ore from bodies without having to dig an open pit at the surface. In order to safeguard the operation underground, tunnel collision warning systems are used to assist operations in tunnels.
The system issues warnings when the equipment operates too close to the walls. Cornering and entering intersections is automatically recognised, analysed and signalled if the operator overshoots or cuts corners. The system furthermore provides a manoeuvring assist for confined spaces. The operator screen displays a distance profile of the surroundings and the vehicle in a birds eye view – similar to modern navigation systems, but more accurate.
At the heart of the system is a small touch screen display with an integrated industrial computer, which displays relevant information dependent on the current situation, like a modern Navigation system. Data of the surroundings is delivered by a front and a rear laser scanner. All the information is processed in the display – application and warning strategies are computed. The design of the system fits perfectly to the underground haul truck from Powertrans, an Australian mine vehicle manufacturer, which operates with this technology.
The information that lies in tunnel walls
Current speed measuring devices on underground vehicles are not very accurate. Revolution counters measure the rotations of the wheel, which is used to calculate the velocity. This technique and the result are dependent on the road surface condition. A complete halt on the tachometer doesn’t necessarily mean the truck has stopped and lost traction. The load of course increases this effect. Inertia opposes the change of motion. Loss of traction must not be underestimated especially in corners and in breaking manoeuvres. Accurate speed information which is fed into the system would therefore improve algorithms and increase the overall performance dramatically.
More accurate speed information instantly adds value to the vehicle control system and improves motion estimation, driving path calculation and further algorithms in the operator assistant system. The solution has to be built of algorithms describing tunnel walls, computing free field (unobstructed areas) and predict motion. The information delivered by a laser scanner in a tunnel can be used to:
- describe the tunnel wall in polygons
- evaluate the free space (unobstructed accessible area for the vehicle)
- evaluate velocity and turning rate (yaw rate)
- estimate possible driving paths
- estimate the movement of the ego-vehicle.
Modern laser scanners produce 10 to 100 scans per second. A
scan can consist of a couple of hundred measurements with multiple characteristics like distance and intensity.
System development always has to deal with the constraint of an embedded industrial architecture. The data therefore has to be reduced to significant points. An envelope build of polygonal lines includes all these points and can be used as the barrier and thus define the drivable path. Assuming normal distribution of scan points as a certainty around measurement values is an approach to compute a safety distance and can be used to evaluate the drivable area for the vehicle. White areas are open areas for the vehicle, grey areas are undiscovered.
Extracting freely and safely accessible areas of scans or scan accumulations (over time) offers benefits to an operator assistant system. This area can therefore be used for collision avoidance and path planning. The technique that correlates between scans and this local area map is called map-matching. In many applications like mapping, road departure warning or collision warning only in the area surrounding the vehicles driving path is of interest. This knowledge can further reduce the amount of data kept in memory.
GPS is not available in underground mine sites. Inertial Measurement Platforms (IMU’s) are expensive and difficult to commission. Odometry data is therefore only available from vehicle sensors like encoders and accelerometers. Being independent of odometry data increases the field of operation for automation systems and reduces the need for additional sensors. A simplification has been made in this work that the possible driving paths consist of circles with constant radius (clothoids) and yaw rate. A straight line is therefore a circle with infinitesimal radius. The turning circle describes the yaw rate dependent on the speed. The objective is the computation of possible impact points for the vehicle at end points of these circles. Using the length of each circle leads to a good estimate of the speed and some very important information – the Time-To-Collision.
“Having captured the automotive market in the recent past, advanced driver assistant systems are now on the move to conquer industrial vehicles in mining.”
The environment in tunnels is usually stationary and not changing. This fact eases the prediction of the movement of the ego-vehicle. One scan has to be matched with the features of an earlier scan in order to detect movement. Empty corridors without features except smooth walls are therefore a major challenge for an accurate scan matching. The walls of mine sites deliver far more features. The results of this velocity estimation are very promising. The accuracy on stationary data is approx. 2kph (following a short transient oscillation time). This information adds value to the overall performance of an assistant system; for example, the length of warning zones can be automatically adjusted to speed up information.
Where to next?
Interest from mine management shouldn’t arise from pending regulation, but from the interest to overcome existing challenges, and the desire to be reputed as an industry innovator. This requires comprehensive risk assessment, followed by the implementation of robust process and the effective utilisation of suitable collision avoidance technology.
This approach means “safety first”. Acceptance of this technology requires the commitment and dedication of personnel to safeguard a successful implementation. If it is agreed with Stewart Bell that technological limitations are no longer an impediment, we are likely to see a mandate for collision avoidance and proximity detection systems.
References
[1] http://www.riotinto.com/ourapproach/17203_mine_of_the_future.asp [2] http://mines.industry.qld.gov.au/safety-and-health/coal-open-cut-safety-statistics.htm [3] http://cdc.gov/niosh/mining [4] Lenné, M. G. & Triggs, T. J. (2009). “Warning drivers of approaching hazards: The importance of location cues and multi-sensory cues.” Maastricht: Shaker Publishing. [5] Quenzel, J (2012). “Auswertung von Laserscannerdaten als Grundlage eines Assistenzsystems für Tunnelfahrzeuge” [6] Rhede, J. G., Wäller, C. & Oel, P. (2011). “Der FAS Warnbaukasten – Strategie für die systematische Entwicklung und Ausgabe von HMI-Warnungen.” Düsseldorf: VDI Verlag GmbH. [7] IQPC – “2nd Annual Mine Site Automation & Communication Africa 2012”, 23 – 25 July, 2012, Sandton, South Africa
Add Comment