General requirements for evaluating and expressing the uncertainty of measurement results

CLAS Requirements document 5, January 2003

1.0 Introduction

  • 1.1 When reporting the result of a measurement, it is required that some indication of the quality of the result be given so that those who use it can assess its reliability. Without such an indication, measurement results cannot be compared, either among themselves, physically or in such places as in capability directories, or with reference values given in a specification or standard. It is therefore, necessary that there be a readily implemented, easily understood, and generally accepted standardized procedure for characterizing the quality of a result of a measurement, that is, for evaluating and expressing its uncertainty.
  • 1.2 This requirements document is meant to address this and is based on the Guide to the Expression of Uncertainty in Measurement (GUM), which should be consulted for more complete details.

2.0 Components of uncertainty

  • 2.1 The uncertainty of most measurement results is made up of a number of components. Some of these components are well defined and evaluated while others are based on varying degrees of knowledge and experience. No matter how these components are estimated, they may be grouped into two categories according to the method used to estimate their numerical values. These two categories include those components that have been estimated using statistical methods (Type A) and those that have been estimated using other means (Type B), with both categories expressed in the form of standard deviations and defined as standard uncertainties.
    • 2.1.1 Type A Evaluation of Standard Uncertainty
      Type A evaluation of standard uncertainty may be based on any valid statistical method for treating data. Examples include calculating the standard deviation of the mean of a series of independent observations.
    • 2.1.2 Type B Evaluation of Standard Uncertainty
      AType B evaluation of standard uncertainty is usually based on scientific judgement using all the relevant information available, which may include:
      • previous measurement data;
      • experience with, or general knowledge of, the behaviour and property of relevant equipment;
      • equipment manufacturer's specifications;
      • data provided in calibration and other certificates; and
      • uncertainties assigned to reference data taken from handbooks.
  • 2.2 Appropriate documents should be consulted to extract, for example, the standard deviation and thus the standard uncertainty for each of these areas. See the list of reference documentation.

3.0 Combined standard uncertainty

  • 3.1 The total standard uncertainty of the result of a measurement is termed combined standard uncertainty (μc). It is an estimated standard uncertainty equal to the positive square root of the total variance obtained by summing all variances (and covariances, as appropriate) as obtained from individual standard uncertainties (μi), however evaluated (Type A or B). If any of the individual uncertainties are correlated, the correlation must be taken into account by experimental evaluation or by calculation using appropriate techniques. See the GUM and/or other references for details and examples on combining standard uncertainties.

4.0 Expanded uncertainty

  • 4.1 CLAS requires that the combined standard uncertainty (uc) be expanded to produce an Expanded Uncertainty (U). The purpose is to provide an interval about the result of a measurement within which the values that could reasonably be attributed to the measurand may be expected to lie with a high degree of confidence. The expansion is done by multiplying the combined standard uncertainty by a coverage factor k = 2. This is for most practical applications equivalent to a level of confidence of approximately 95%. See the GUM and/or other references (on the subject of effective degrees of freedom) for details on the rare cases when a coverage factor other than 2 is required to achieve a level of confidence of approximately 95%. In all cases, the coverage factor, k, and the level of confidence need to be stated in the calibration certificate.

5.0 Reported uncertainty

  • CLAS recognizes three types of calibration services. The method of reporting uncertainty depends on the TYPE of service being provided.
    • 5.1 TYPE I Service
      • 5.1.1 Accredited laboratories certified to provide a TYPE I service are required to report a measurement result and an expanded uncertainty (U). The certificate must explicitly indicate that a coverage factor k = 2 was used. It must also include a probability interpretation, such as a level of confidence. When a level of confidence is given, it should be supported by a claim such as assuming a normal distribution.
      • 5.1.2 Wording such as, The reported uncertainty in this report is expanded using a coverage factor k = 2 for a level of confidence of approximately 95%, assuming a normal distribution, would be appropriate for most situations. In rare situations when the laboratory calculates a different coverage factor to account for small degrees of freedom (refer to the GUM and/or other references below for details), wording such as, The reported expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=XX which, for a t-distribution with νeff = YY effective degrees of freedom, corresponds to a level of confidence of approximately 95%.
    • 5.2 TYPE II Service
      • 5.2.1 The results of a TYPE II service are usually reported as a compliance to a tolerance or specification. When this is done, the measurement results are provided and a statement on the adequacy of the measurement system used for the calibration is made. This statement can be made in the form of a test uncertainty ratio (TUR); i.e., the ratio between the tolerance or specification of the equipment being calibrated versus the uncertainty of the measurement system. See CLAS Requirements Document 3, Minimum Requirements for Measurement Standards for Laboratory Certification. The statement can also be in the form of a measurement uncertainty, coverage factor, and level of confidence as for TYPE I services.
    • 5.3 TYPE III Service
      • 5.3.1 CLAS laboratories certified to provide a TYPE III service can report their measurement results as per the requirements of TYPE I and II services, depending on the application.

6.0 Possible components of measurement uncertainty

  • 6.1 The reported expanded uncertainty (U) of all measurements performed by CLAS certified laboratories can be separated into uncertainty components (Type A or B), which can include:
    • 6.1.1 the component associated with the NRC (or next higher echelon) laboratory, applied to the calibration of the reference or transfer standard;
    • 6.1.2 the component introduced by the transportation of the reference standard or transfer standard between NRC (or next higher echelon) laboratory and the CLAS certified laboratory;
    • 6.1.3 the component due to drift, resolution, and instability of the reference standard and instruments;
    • 6.1.4 the component associated with the measurement process (e.g., circuit loading effects, thermal emfs, layout of the apparatus, rf mismatch, cosine errors, and polarization);
    • 6.1.5 the component associated with such personal biases as those influencing the reading of analogue instruments or when deciding when to take a measurement or to terminate a measurement or to repeat a measurement or to exclude a measurement or discriminating whether an event has occurred or not;
    • 6.1.6 the component due to approximations and assumptions incorporated in the measurement process and procedure (e.g., imperfect interpolation and extrapolation of calibration data at fixed points, non-representative sampled test points, use of inexact constants, and other parameters from external sources);
    • 6.1.7 the component associated with influences of supporting equipment including ancillaries such as connecting leads;
    • 6.1.8 the component due to the behaviour of the device being measured (e.g., instability during measurement, and resolution of display);
    • 6.1.9 the component due to the condition of the device being measured (e.g., parallelism and flatness of anvils for the calibration of micrometers, magnetic susceptibility for the calibration of weights, and input impedance for the calibration of electrical equipment);
    • 6.1.10 the component due to such environmental parameters as electromagnetic interference (for electrical calibrations), buoyancy of air (for the calibration of weights), stray light (for photometric calibrations), and temperature and humidity;
    • 6.1.11 the component due to imperfect measurement or knowledge of influencing parameters (e.g., environmental); and
    • 6.1.12 the component due to variations in repeated measurements under apparent identical conditions.
  • 6.2 Some of these components might be found to be negligible, while others could be substantial, depending on various factors including the quantity being measured. Each of these components can be further separated into very specific factors, depending on the needs and the applications. It may be useful, at times, to identify whether these components arise from either random or systematic effects.
    Note: An additional allowance for the possible effects of transportation on the measured equipment, long-term stability or intended use may be included at the discretion of the laboratory. When this is done, it must be reported with adequate details.

7.0 References

  • Guide to the Expression of Uncertainty in Measurement (GUM), first edition in 1993, corrected and reprinted in 1995, International Organization for Standardization (Geneva, Switzerland).
  • The NIST Reference on Constants, Units, and Uncertainty.
  • NIST Technical Note 1297 (PDF format, 176 KB), Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results
  • Expression of the Uncertainty of Measurement in Calibration, EA-4/02 (PDF format, 1.07 MB), 1999, European co-operation for Accreditation.
Date modified: