Most of the industrial goods we buy have a pedigree that can be traced back to standards. These standards, which include verification testing, are necessary to ensure product performance and allow products to be manufactured by more than one vendor.
An example of this is the manufacture of steel. Many different steels are available for purchase. When an engineer chooses steel for an application, it is based upon the expectation that the steel will perform to meet the design requirement. Lubricants are similar, having standards and specifications in place to aid in their manufacture and ensure their performance.
Standards
In contrast, few standards are available to support the in-service lubricant test industry. Condition monitoring test services for in-service lubricants are purchased largely based upon the credibility of the laboratory providing the testing, the lab’s recommendations and price.
The tests performed by commercial labs on in-service lubricants are often modifications of American Society for Testing and Materials (ASTM) standards or proprietary methods developed by private laboratories.
The problem with this is that the use of modified test methods and lack of standards specific to in-service condition monitoring can produce data of unknown quality which introduces uncertainty in the meaning of the measurements.
It is obvious that without test measurements, oil analysis would consist only of appearance, feel, smell and apparent thickness (viscosity) - all subjective and non-comparative and with limited usefulness. It is also intuitive that test measurements in and of themselves have little meaning if they cannot be associated with a property of the oil or a failure mechanism within the machine. The collective quality of the data is less apparent but is also vitally important.
Accuracy and Precision of Data Quality
Data quality has two different but equally important parts. The first is whether the data is technically accurate. Reference materials (also called reference standards) are used to determine if a test produces the expected, or “right”, answer.
In other words, a 100 ppm iron (Fe) reference standard should produce a result of 100 ppm Fe on the instrument used for analysis. In the real world however, it is unlikely that the instrument will produce this exact value each time.
This raises two quality questions, one of which is: “how close is the actual answer to the expected answer?” and the second of which is “how good is the instrument in giving nearly the same answer each time?”
This second part of data quality concerns the precision of the data given. ASTM uses the terminology of reproducibility and repeatability of the data to address precision.
For example, if a spectrometer test is performed several times using the same instrument within the same laboratory by the same operator, what is the range or repeatability of measurements recorded? If in 20 tests, if the data ranges from 90 to 110 ppm, then the data repeatability would be ±10 percent. When many labs perform the same test, additional uncertainty is introduced.
This typically extends the variation or range of measurements and is known as data reproducibility. In the 100 ppm Fe example, this could mean measurements from 85 to 115 ppm Fe which would mean a range of ±15 percent.
The effectiveness of condition monitoring is directly proportional to the quality of the data. The saying of “garbage in, garbage out” really applies here. To the customer, the ideal scenario would be data that is perfectly accurate each time the sample is measured.
The world of testing is not that perfect. The reality is that unless standard test methods are in place and rigorously followed, the test measurements have unknown accuracy. The customer has no way of knowing how good or bad the data really are.
Cost can also become important to this discussion because different methods or standards have different laboratory test performance costs which would be reflected to the customer in the billing. The other side of the cost of testing is the level of data quality needed.
One would think that a more expensive test would result in higher data quality. In many cases this would be true, although it is a dangerous assumption to make.
The debate over the degree of data accuracy necessary to satisfy the needs of the customer is important for both customer and laboratory. Standards written to manufacture and market new products have an entirely different focus than standards intended to monitor machine condition, contaminants and oil serviceability.
Interpreting Data
The end customer needs to have sufficient understanding of the meaning of the data to grasp and act on recommendations made by a service provider as well as the level of data precision and significance needed for the application.
Using lead or tin as a wear-metal example: if 1 or 2 ppm is critical to the application (such as an indication of the wear rate of babbitt bearing material in a large turbine system), then precise and sensitive data are required. If the critical threshold of accuracy for a wear metal is much higher than 1 or 2 ppm, a less precise and sensitive level of data would be acceptable.
For example, test accuracy to 1 ppm Fe in the used oil of an operating commercial diesel engine would not be reasonable. This type of reasoning by the customer should be included in the decision to justify the cost of the test.
The accuracy associated with the rigor in verbatim performance of a test method, data repeatability and data reproducibility is a key difference between test results based upon ASTM standard test methods and ones based upon in-house methods. Modifications to the standard mean that portions of the standard were omitted or revised by the laboratory.
These differences can readily affect the cost of testing. A laboratory could modify ASTM test methods in an effort to provide a test that meets minimum customer expectation (for used oil analysis) at a lower overall cost.
Modifying a Standard
Reasons often given for modifying a standard include the following: some of the steps are not essential, the data obtained through our modification is good enough or even better, increased throughput, there isn’t an ASTM standard available but the one referenced is the closest one to what we do, it can be done cheaper my way or the ASTM method requested isn’t good enough for what is needed. In some cases, the modifications to the standards may even improve accuracy and/or repeatability.
Generally, a standard is a document developed based upon an identified need and the cumulative experience of those involved in its writing. An ASTM International Standard is a “consensus document” applicable to a material - meaning that all who participate have a say in its development. These participants are producers and users of the material as well as the instrument makers and laboratories analyzing the materials.
When differences of opinions or disagreements exist, a voting process is in place to allow the differences to be resolved and the work to go forward. Moreover, as time goes by, further improvements to the applicability, precision and meaningfulness of the method are incorporated.
Standards for Testing
ASTM D02 on petroleum products is organized to address three general types of industry needs to include the testing of physical lubricant properties, the testing of products to include specifications and specialty testing, and finally services. ASTM International develops and maintains several types of standards, three of which are relevant to this article:
A standard guide provides direction on a process, an example of this is ASTM D6439-99 Standard Guide for Cleaning, Flushing and Purification of Steam, Gas and Hydroelectric Turbine Lubrication System.
A standard practice provides test measurements but does not define accuracy; an example of this is ASTM D6224 Practice for In-Service Monitoring of Lubricating Oil for Auxiliary Power Plant Equipment.
A standard test method provides specific steps to obtain a test measurement and also describes accuracy in terms of repeatability and reproducibility. ASTM D445 Standard Test Method for Kinematic Viscosity of Transparent and Opaque Liquids is a good example of a broadly used Standard Test Method. Incidentally, this method was recently expanded to include the measurement of in-service oils.
Committee Membership
ASTM D02 membership consists of suppliers and users of petroleum products and other interested parties such as government specification bureaus and product testing laboratories.
Historically, the focus of this body has been standards aimed at new product testing. An example of this would be the subcommittee where automotive lubricant standards are developed and engine suppliers would be a key example of a general interest user within this group.
In-service lubricant condition monitoring has received little if any consideration by this or other ASTM subcommittees as the emphasis in the example of engine oils committee were requirements provided by organizations such as the American Petroleum Institute (API) or the Society of Automotive Engineers (SAE).
Over the last several years, professionals within the condition monitoring segment of the lubricant industry however have become involved in ASTM creating an ongoing and significant change to this historical focus. Standards already in place are being reviewed and revised to specifically include in-service oils and new standards are being developed specifically for in-service monitoring. Two important examples are ASTM D4378 - Practice for In-service Monitoring of Mineral Turbine Oils for Steam and Gas Turbines, and ASTM D6224 (described earlier). Both of these are highly relevant to the condition monitoring industry.
Subcommittee’s Efforts For In-Service Oils
These efforts clearly identified a need for a stand-alone subcommittee dedicated entirely to the monitoring of in-service oils. In 1999 a new subcommittee was formed to address these needs.
This subcommittee is CS96, In-Service Lubricant Testing and Condition Monitoring Services Industry Support. The subcommittee (CS96) is made up of representatives from commercial laboratories, instrument manufacturers, oil manufacturers, industrial plant end users of test measurements, and it has a broad charter that covers important aspects of condition monitoring.
This subcommittee is actively writing standards related to condition monitoring on the topics of viscosity, chemistry, FTIR, wear, particle counting and integrated testers. FTIR is a good example of a broadly used industry test that is applied and used in many different ways. New standards are presently being written to standardize this important technology.
Given that the used oil condition monitoring industry is unregulated and does not always conform to available industry standards, it is vitally important that practice-specific standards are developed which will provide the customers with a known basis for the quality of the data that is produced.
This does not mean that the data currently obtained isn’t useful, or that end customers aren’t getting value. Rather it means that the accuracy of the data is undefined and opportunities exist for unintended errors in cost of testing or evaluation of data measurements.
Additionally, the modifications that have been made by the condition monitoring industry to existing ASTM standards, and the many in-house test methods developed by commercial labs, represent significant innovation within the oil testing industry to include new product testing.
It is unfortunate that this innovation has not been captured and improved upon. Condition monitoring test measurements represent the significant percentage of all lube samples tested.
Given this fact, the in-service segment of the oil test industry has the greatest need to be responsive to market forces, and should be the driving force for innovation and quality improvement. ASTM D02 CS96 l provides a forum for this to occur.
ASTM D02 meets twice a year at various locations in the United States or Canada to write and review standards. The meetings are open to anyone interested in attending. The next meeting is planned for Toronto, Canada from June 25 through 29, 2006. It is also possible to join and take advantage of internet tools rather than coming to the meetings in person. Current committee work and minutes of meetings are available on the ASTM Web page (www.astm.org).
There are no qualifications to join other than interest in the subject. ASTM is an international organization with members from numerous countries. The CS96 committee within D02 provides the condition monitoring industry with a voice and place to improve the profession and technology. Its progress and success will largely be a result of those willing and able to participate.
