1、Designation: D8146 18 An American National StandardStandard Guide forEvaluating Test Method Capability and Fitness for Use1This standard is issued under the fixed designation D8146; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, t
2、he year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This guide covers techniques for evaluating the statisti-cal capability and fitness for use of standard tes
3、t methods usedfor measuring properties of petroleum products, liquid fuels,and lubricants. Specifically, this guide provides strategies forevaluating the capability of a test method to provide asufficiently precise estimate of the intended parameter versus agiven level or value of that parameter and
4、 for assessing, withsufficient confidence, the fitness for use of a test method fordetermining the acceptability of products versus specification,regulatory, or manufacturing limits.1.2 The assessment of capability in this guide is applicableto test methods that always yield non-negative numericalre
5、sults and have a hard, fixed zero lower limit or fixed upperlimit (for example, 100 %). Detailed knowledge of a testmethod and professional judgement may be required in deter-mining the applicability of this guide to certain test methods.1.3 In accordance with this guide, the assessment of fitnessfo
6、r use of a test method versus specification, regulatory, ormanufacturing limits is generally applicable to test methodsthat provide numerical results and have applicable reproduc-ibility precision values.1.4 This guide is intended for use by test methoddevelopers, specification setters, and laborato
7、ries selecting testmethods to evaluate products for specific purposes or versusspecific limits.1.5 This guide is not applicable to test methods that measuretemperature to determine properties such as, but not limited to,freeze point, pour point, flash point, and distillations. Thisguide is not inten
8、ded for use by laboratories for evaluatinglaboratory test method execution capability, nor is it intendedto evaluate manufacturing process capability.1.6 The expressions of capability and fitness for use derivedfrom use of this guide should not be the only criteria forselection and use of a given te
9、st method. Other factors such asthe laboratorys experience with the test method, length of timeto complete an analysis, typical product results relative to theregulatory or manufacturing specification limit, closeness ofthe results to zero, and cost factors may contribute moresignificantly to the de
10、cision to use a given test method for aspecific application.1.7 This guide draws on statistical approaches covered inother standards supporting petroleum products, liquid fuels,and lubricants, including Practices D3244, D6259, D6299,D6300, D6792 and ISO 4259.1.8 This standard does not purport to add
11、ress all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety, health, and environmental practices and deter-mine the applicability of regulatory limitations prior to use.1.9 This international standard was deve
12、loped in accor-dance with internationally recognized principles on standard-ization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recom-mendations issued by the World Trade Organization TechnicalBarriers to Trade (TBT) Committee.2. Referenced Doc
13、uments2.1 ASTM Standards:2D3244 Practice for Utilization of Test Data to DetermineConformance with SpecificationsD6259 Practice for Determination of a Pooled Limit ofQuantitation for a Test MethodD6299 Practice for Applying Statistical Quality Assuranceand Control Charting Techniques to Evaluate Ana
14、lyticalMeasurement System PerformanceD6300 Practice for Determination of Precision and BiasData for Use in Test Methods for Petroleum Products andLubricantsD6792 Practice for Quality Management Systems in Petro-leum Products, Liquid Fuels, and Lubricants TestingLaboratories1This guide is under the j
15、urisdiction of ASTM Committee D02 on PetroleumProducts, Liquid Fuels, and Lubricants and is the direct responsibility of Subcom-mittee D02.94 on Coordinating Subcommittee on Quality Assurance and Statistics.Current edition approved Oct. 1, 2018. Published November 2018. DOI:10.1520/D8146-18.2For ref
16、erenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, Wes
17、t Conshohocken, PA 19428-2959. United StatesThis international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recommendations issued by the World Tr
18、ade Organization Technical Barriers to Trade (TBT) Committee.12.2 ISO Standard:3ISO 4259 Petroleum productsDetermination and applica-tion of precision data in relation to methods of test2.3 ASTM Standards referenced only in Appendix X2,Worked Examples, are listed in X2.3.3. Terminology3.1 Definition
19、s:3.1.1 precision ratio (PR), nan estimate of relative mag-nitude of repeatability and reproducibility. The PR for a givenstandard test method can provide information on the relativesignificance between variation caused by different operatorsand laboratories compared to a single operator in a labora
20、toryperforming the standard test method. D67923.1.1.1 DiscussionThe PR for a published test methodestimates the influence that non-site-specific variations have onthe published precision; the PR metric essentially judges theadequacy of between-laboratory standardization, or the agree-ment between la
21、boratories relative to within-lab repeatabilityprecision.3.1.2 repeatability (r), nthe quantitative expression for therandom error associated with the difference between twoindependent results obtained under repeatability conditionsthat would be exceeded with an approximate probability of 5 %(1 case
22、 in 20 in the long run) in the normal and correctoperation of the test method. D63003.1.2.1 DiscussionThis definition is applicable to preci-sion values obtained following the practices described inD6300. Repeatability determined using different or modifiedpractices may not have the same statistical
23、 significance as thatinferred when following D6300.3.1.3 repeatability conditions, nconditions where inde-pendent test results are obtained with the same method onidentical test items in the same laboratory by the same operatorusing the same equipment within short intervals of time.D63003.1.4 reprod
24、ucibility (R), na quantitative expression forthe random error associated with the difference between twoindependent results obtained under reproducibility conditionswould be exceeded with an approximate probability of5%(1case in 20 in the long run) in the normal and correct operationof the test meth
25、od. D63003.1.4.1 DiscussionThis definition is applicable to preci-sion values obtained following the practices described inD6300. Repeatability determined using different or modifiedpractices may not have the same statistical significance as thatinferred when following D6300.3.1.5 reproducibility co
26、nditions, nconditions where inde-pendent test results are obtained with the same method onidentical test items in different laboratories with differentoperators using different equipment. Note that different labo-ratory by necessity means a different operator, differentequipment, and different locat
27、ion and under different supervi-sory control. D63003.1.6 site precision (R), nthe value below which theabsolute difference between two individual test results obtainedunder site precision conditions may be expected to occur witha probability of approximately 0.95 (95 %). It is calculated as2.77 time
28、s the standard deviation (R) of results obtainedunder site precision conditions. D62993.1.6.1 DiscussionSite precision is often estimated in thesetup of a quality control chart, and the TPI can be used toevaluate the performance of the test method in the laboratory asdescribed in D6792.3.1.7 site pr
29、ecision conditions, nconditions under whichtest results are obtained by one or more operators in a singlesite location practicing the same test method on a singlemeasurement system, which may comprise multipleinstruments, using test specimens taken at random from thesame sample of material over an e
30、xtended period of timespanning at least a 15-day interval. D62993.1.8 test performance index (TPI), nan approximate mea-sure of a laboratorys testing capability, defined as the ratio oftest method reproducibility to site precision. D67923.1.8.1 DiscussionThe term capability as used in thisdefinition
31、 refers to ability of the laboratory performing the testmethod relative to the published reproducibility and not to thecapability of the test method with respect to application at agiven level or value.3.1.8.2 DiscussionThe TPI term is similar to precisionration (PR = R/r), but the use of site preci
32、sion instead ofrepeatability provides a direct evaluation of the laboratorysperformance of the test method versus reproducibility. Sincethere is a general expectation thatRRr,then generally itwould be expected that TPI 4 generally indicate asignificantly large difference between repeatability andrep
33、roducibility, which means that the between-laboratory biasis the dominant contributor towards reproducibility. The im-plication is that the test method is not adequately standardizedand further work on the test method should be considered.5.4.4 When PR is small and in the range of 1 to 2 (that is,as
34、 r approaches R) then it could imply that the contribution ofbetween-laboratory bias towards reproducibility is small orinsignificant relative to the within-lab repeatability.When PR 28 % (that is,r is approaching the target value), then R is also approachingthe target value and the test method may
35、have limited useful-ness at the level of intended use.5.4.5 It is useful to compare the PR obtained for a new ormodified test method to that for existing similar test methods todetermine whether there are significant improvements.5.5 PR and TPI:5.5.1 The PR can be used in conjunction with the TestPe
36、rformance Index (TPI) as discussed in D6792. PR and TPIare used to suggest minimum QC sample testing frequencies,and the PR is used to recommend ranges for determiningacceptable versus unacceptable TPI values obtained from thesite precision standard deviation (R) generated for controlcharts.Test Per
37、formance Index 5 Test Method Reproducibility / Site Precision(4)Where site precision = 2.77 R6. Test Method Fit for Use6.1 Test method fitness for use is evaluated by comparingthe test method reproducibility to the span or range of thespecification, regulatory, or manufacturing limits.NOTE 2See simi
38、lar discussion in ISO 4259.6.2 For a test method to be considered fit for use, the lowerspecification limit (LL) should not be less than the lowest limitof the test method as listed in the scope of the test methods oras stated for the precision, or the upper specification limit (UL)should not be gre
39、ater than the upper limit of the test method aslisted in the scope or as stated for the precision.6.3 The range between specification lower and upper limitsshould satisfy the following condition: upper specificationlimit lower specification limit should not be less than thequantity (2 R evaluated at
40、 lower specification limit+2Revaluated at upper specification limit). Or simply, (ULLL) (4R).6.3.1 In situations where the value of (ULLL) is less than4R, the results obtained will be of doubtful significance indetermining whether a sample does or does not satisfy therequirements of the specificatio
41、n. According to statisticalreasoning, it is desirable for (ULLL) to be considerablygreater than 4R.6.4 For Limits Near Zero or 100 %:6.4.1 For cases where the specification limit is close to zero(or the lower test method scope limit), then a test method isconsidered fit for use when Limit (zero or l
42、ower test methodscope limit) 2R.6.4.2 For cases where the limit is near 100 % (or upperscope limit) then a test method would be considered fit for usewhen (100 or upper scope limit) Limit 2R.6.5 When the conditions of 6.2, 6.3,or6.4 are not met, thenone or both of the following courses should be con
43、sidered: thespecification limits should be examined to consider whetherthey can be changed to fit with the precision of the test method,or the test method should be examined to see whether theprecision can be improved. In some situations, an alternativetest method with a reproducibility that fits wi
44、thin the desiredspecification limits could be adopted.6.6 Many difficulties that arise in interpreting specificationsare due to test imprecision. Because of this, a true value of aproperty can never be determined exactly; and it is necessary toinfer from measured values the range within which the “t
45、ruevalue” is likely to lie. Refer to Practice D3244 for a discussionof how test imprecision should be interpreted relative tospecification values.7. Keywords7.1 capability; fitness for useTABLE 1 Evaluation of Test Method Reproducibility Using PRand Not Considering the Level of Intended Use of the T
46、estMethodPR Adequacy of Reproducibility# 1 Within-laboratory common cause variability is so large that itmay be masking the between-laboratory common causes.1 to 2 Between-laboratory common cause variability is small orinsignificant relative to the within-laboratory common causevariability.2 to 4 St
47、andardization conditions appear to adequately controlbetween-laboratory common cause variability.4 to 10 Standardization conditions to control between-laboratorycommon cause variability appear to be less than optimal.10 Between-laboratory bias appears to be the dominantcontributor towards reproducib
48、ility and that the test methoddoes not appear to be sufficiently standardized.D8146 184APPENDIXES(Nonmandatory Information)X1. STRATEGIES FOR DETERMINING TEST METHOD SCOPE LIMITSINTRODUCTIONThe following strategies are not intended to represent an exhaustive list of approaches fordetermining method
49、scope limits.X1.1 Lower Method Scope LimitX1.1.1 The lower method scope limit is determined asfollows: The larger of lowest sample mean tested in theInterlaboratory Study (ILS) or lowest achievable result + 2R,where R is evaluated at the lowest sample mean. The lowestachievable result is defined as: the point of truncation (exclud-ing minus infinity) below which a result is not defined. Anexample is the zero point for a percent concentration. For manyCommittee D02 test methods, the approach using only thelowest sample mean tested in the
