Root Cause Failure Analysis in Everyday Life
Tags: maintenance and reliability
While composing this article, I am sitting in a hotel in Iceland on a business trip. Over the weekend, a colleague and I decided to take in some of the unique sites Iceland has to offer. We decided on a three-hour drive to observe the active glacier, specifically the point where it reaches the ocean.
The Arctic Circle
When asking a local man for directions to the glacier, we were advised to take a short cut. This short cut was on the map and listed as a gravel road. We were told that we could negotiate the road quite easily even though our rental car maintained a ground clearance of only four to five inches with an engine not much larger than that of a riding lawn mower.
Armed with route information, warnings and road maps, we felt fully capable of finding the way to our destination.
4X4 For a Day
To start the day, we researched the possibility of renting a vehicle that would be better suited for the type of driving we anticipated. Given the price of $400 per day to rent a four-wheel drive, we decided to continue on with our little car.
The journey along the gravel road was going well until we encountered road construction. The construction was basically rock being pulled from the levy area, placed on the road and somewhat smoothed by a loader. As we sat trying to figure out what to do, we were waved on by the dump truck driver. Making the climb uphill on the newly laid gravel, we quickly remembered the ground clearance on our car was only four to five inches. The gravel that was laid down was not our typical idea of gravel, but more like large rocks … not quite boulders, but not quite gravel. Making it to the top brought a welcome sigh of relief and a view of what was apparently more "new road construction". This construction was a bit smoother than what had just been put down, but it still required a slow speed and extra attention to negotiate.
Once we realized the road conditions and took appropriate action to avoid a catastrophic problem, we experienced a setback: the right rear tire went flat. While changing the tire, we saw at least two other cars identical to ours traveling on the road. This proved the car should be able to handle the road, so we examined more closely why the tire had gone flat. Because we are both reliability consultants, performing a quick root cause failure analysis (RCFA) came quite naturally.
Missing Tread
We noticed that the flat tire had very little tread with absolutely no tread on the outside edges (Figure 1). This meant the tire had been severely under-inflated for many kilometers; more than we'd driven over the past week. On further inspection, we noted the left rear tire had just a bit more tread than the damaged tire. We did note that at least the front tires were in suitable condition. After a short discussion and review of the map, we decided to continue on our journey. The rest of the trip went much slower because we obviously could not drive at full performance due to the low tread level of the other rear tire.
After a total of five hours, we were able to complete the "three-hour" drive to the glacier. This entire journey begged for a review which is extremely close to what we teach in seminars and consulting projects. In performing this review (RCFA), the following must be answered: What went wrong and what can be done to avoid it in the future?
Figure 2. Glacier
On the Road to a Solution
Our original plan was to do more sight-seeing the following day; an adventure that would take us on similar roads. We did not want to go through this scenario again, and were forced to take proactive action to avoid the same failure. We had to go over all of the problems, examine the solutions and choose which were the most effective with the best financial impact. In our case, the choice was obvious: we simply returned the troubled rental car and were given a new one. Learning from our first experience, we double-checked the tire conditions of the new car.
This story closely parallels a similar plant situation. Shortly after attending a training seminar on oil analysis, a reliability engineer was eager to get his program on the road. After some quick research, the plant constructed a quality lube storage room, and purchased lubrication routing software and high-quality portable filtration units. The equipment was enrolled in the oil analysis program and alarm levels were set for all parameters.
In this particular instance, the routing software can be compared to our rental car. While it did perform in the aspect of allowing for lube task routing, it certainly lacked the flexibility and durability required to run a proper lubrication program.
As sample reports began coming back showing high levels of contamination, it was noted that a failure to fully retrofit the equipment with the appropriate contamination control measures had taken place. This resulted in a massive slowdown in the overall progress of the lubrication program. Like our flat tire situation, the program was able to continue but at a much slower pace due to the limited ability to address proper contamination removal recommendations.
In the case of the plant lubrication program, while a total program failure did not occur, a review of the situation suggested several avenues for improvement. The simple routing software was discontinued. The plant is currently transitioning into a dynamic lubrication program management software tool that will help manage and track all aspects of lubrication. The addition of the appropriate contamination control measures was key to the plant's success. Figure 3 shows the overall improvement in average gearbox cleanliness over the past 24 months.
Performing failure analysis in a plant environment is like our Iceland situation. All of the potential sources of failure must be reviewed, and one should choose the conditions that can have the highest impact on possible failure and control accordingly. All failures are potential learning experiences. While some of these may end up being expensive educational lessons, failing to learn from them only sets the scene for a repeat failure. The moment you experience a second failure for the same reason, the realized cost of the first failure can be compounded exponentially.