ARCHIVED - Evaluation of the NRC Institute for National Measurement Standards

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Executive Summary

The National Research Council Act mandates NRC with "the investigation and determination of standards and methods of measurement" for Canada. This responsibility is vested in the Institute for National Measurement Standards (NRC-INMS), Canada's national metrology institute (NMI). NRC-INMS not only performs R&D that translates science into applicable technologies, but also provides a wide variety of calibration and measurement services that underpin the accuracy of millions of measurements annually in public and private sector testing laboratories.

An evaluation of NRC-INMS was undertaken in fiscal year 2007-2008. The primary reasons for conducting an evaluation of the NRC-INMS program were:

  • The last evaluation of NRC-INMS took place in 1999. Because the landscape of metrology in Canada has changed significantly since that time and to follow good management practices, NRC-INMS was selected as the site of an institute-wide evaluation study.
  • The development of a plan for metrology at NRC was underway when the evaluation work began. The evaluation of NRC-INMS was designed to provide information on the impacts and outcomes of the Institute's activities and feed into this broader planning process.

The evaluation was conducted in accordance with NRC's approved evaluation plan for 2007-08 and Treasury Board Secretariat (TBS) policies. The evaluation covers the period 2002-03 to 2007-08 inclusive and addresses issues related to relevance, success, cost-effectiveness and effectiveness of design and delivery.

The key methods used in the study included a review of documents; a review of administrative and performance data; key informant interviews; an international comparison study; case studies; a cost-benefit analysis of the socio-economic impacts of metrology in Canada; and a review of key comparisons conducted by the Institute.

Overview of the NRC Institute for National Measurement Standards

The mission of NRC-INMS is to provide "the technical and infrastructural foundation for the national measurement system and thereby strengthen Canada's innovation and competitiveness, support international trade, and advance social well being of Canadians". Its vision is "to be acknowledged globally as a model national metrology institute, and to be recognized nationally as an essential component of Canada's innovation system".

In order to fulfill its mission and achieve its objectives, NRC-INMS has three on-going programs:

  • Development of measurement standards: This program focuses on research activities to develop new methods for the realization of base SI units (International System of units) and the measurement of fundamental constants, the maintenance and improvement of existing measurement capabilities and of laboratory infrastructure, and the production of certified reference materials.
  • Dissemination of measurement science: This program uses NRC-INMS expertise to benefit Canadian industry through technology transfer activities, collaborative projects with public and private partners, calibration and measurement services, and sales and distribution of Certified Reference Materials (CRMs). Training and outreach activities include metrology workshops and courses and conference organization and participation.
  • Support for Canada's National Measurement System: NRC-INMS represents Canada on a number of international metrology and standards organizations (e.g., Bureau international des poids et mesures (BIPM) and the Inter-American Metrology System, known as SIM) and participates in the consultative committees of Comité international des poids et mesures (CIPM) Mutual Recognition Arrangement (MRA). In addition, NRC-INMS maintains the quality management system required by the CIPM MRA (ISO 17025) and coordinates with Canadian partners in the national measurement system (Measurement Canada, Standards Council of Canada, and others).

General Conclusion

The evaluation study points to the international reputation of NRC-INMS as a clear indication of the quality of its research, dissemination efforts, and support for Canada's measurement system. The Institute is highly regarded, both within the Canadian metrology community and by international organizations as a leader in its field. Specific recommendations made below focus on certain administrative processes and suggest possible areas for improvement.

Summary of Findings and Recommendations - Relevance

The extent to which the activities and mandate of NRC-INMS remain relevant to current priorities and respond to the needs of Canadians was examined based on a review of internal and external documents and through interviews with NRC-INMS representatives as well as key clients and stakeholders. The role of the Canadian federal government in measurement science appears to be supported by studies focusing on the development and maintenance of standards as a public good. In addition, the evaluation findings point to the fact that NRC-INMS continues to meet the needs of government and industry through its mandated activities in terms of enabling innovation, protecting trade interests and enhancing the quality of life of Canadians. Given the underpinning nature of metrology, the activities of NRC-INMS can be linked to the scientific priorities of the federal government. Ongoing work in new areas, such as nanotechnology and biotechnology, demonstrate the institute's commitment to making a contribution to new economic and societal issues that will increase in importance in future years.

Recommendation 1: NRC-INMS should continue to focus its efforts in developing new and emerging areas of metrology based on strategic thrusts identified by the government. However, investment of human and financial resources in these areas must be carefully weighed against its mandated responsibilities as Canada's National Metrology Institute.

Management Response and Proposed Actions: Accepted. The NRC-INMS Business Plan articulates the strategic importance of investments in human resources and in infrastructure (capital). HR investments address succession needs for critical, on-going activities, while providing flexibility to address research needs for emerging technologies. In fiscal year of 2008-09 NRC-INMS hired 5 new research associates and invested $2.6M in equipment and infrastructure. Strategic investment in these areas will continue in the upcoming years of the 3 year Business Plan.

Summary of Findings and Recommendations - Success

Development of Measurement Standards

As the main research program of NRC-INMS, the development of measurement standards can be considered a key driver in the achievement of the scientific, economic, and social impacts of the Institute. By leading research and development activities that aim to ensure that current and future Canadian needs for primary measurement standards and methods are met, NRC-INMS contributes to many areas of Canadian economic and social activity, with a direct link to the potential enhancement of the quality of life of Canadians.

Although the broader scientific, economic, and social impacts of the research conducted by NRC-INMS could be identified through various means in the evaluation study, the more concrete contributions of the Institute were difficult to pinpoint due to a lack of information on the outcomes associated with specific projects. The performance measures collected by the Institute provided important and useful information regarding scientific and administrative outputs stemming from the activities conducted by Institute staff. However, details regarding the immediate and intermediate outcomes were difficult to obtain from interviews and other research methods and thus limit the analysis presented here.

Dissemination of Measurement Science

The dissemination of measurement science involves an outward-looking set of activities aimed at sharing research findings with the scientific community and industrial partners. Overall, clients of NRC-INMS are highly satisfied with the calibration and measurement services received from Institute staff, although some administrative issues remain to be resolved. These stem mostly from recent changes in the fee-for-service structure and should be relatively simple to improve in the future. The findings of the evaluation also point to an issue related to a perception of the metrology community in regards to the CLAS program. In brief, this issue speaks to how the CLAS certification is perceived outside of Canada given a certain lack of awareness of international agreements such as the International Laboratory Accreditation Cooperation (ILAC). There were also misconceptions associated with the role of CLAS laboratories in providing routine measurements to clients and the role of NRC-INMS in providing similar services. Finally, the evaluation was only able to identify anecdotal findings related to the course offerings of the Institute. A more systematic approach to performance measurement should be implemented in order to obtain management information on course selection, delivery, and outcomes.

Recommendation 2: The administrative mechanisms associated with the changes in the fee-for-service structure and the contracting process should be streamlined in order to reduce the burden on the partners and clients of the Institute. Further, a tracking system for education and outreach activities should be developed to enable the Institute to capitalize on participant interests and needs and to maximize the potential benefits of these activities.

Management Response and Proposed Actions: Accepted. A variety of communications means will be employed to reach partners and clients in a timely manner regarding fee increases. This will include customized mailings for key clients before changes take effect; meetings with partners; enhanced, interactive internet communications and contract administration.

Recommendation 3: The misconceptions held by some members of the measurement and calibration community regarding the duplication in the calibration services provided by NRC-INMS and the CLAS laboratories should be corrected through the communication of internal analyses conducted by NRC-INMS and other means deemed appropriate by the Institute. NRC-INMS should also pursue its ongoing efforts in rectifying issues associated with the international recognition of the CLAS certification by requesting specific information from CLAS laboratories on inappropriate requests and by continuing to participate in international fora on this question.

Management Response and Proposed Actions: Accepted. A communications strategy that addresses the misconceptions will be developed. INMS will market the information content and search functionality of the CLAS database to provide up-to-date information to potential clients regarding the capabilities of CLAS laboratories. CLAS, through its outreach program, will proactively solicit information regarding the acceptance of CLAS certification internationally and represent the CLAS laboratories' interest in international bodies such as ILAC, IAAC, APLAC, NUPIC, etc.

Support to Canada's National Measurement System

The role of NRC-INMS in supporting Canada's national metrology system takes different forms, from its participation in international key comparisons to ensuring that Canada's interests are well represented through committee work. Overall, the findings of the evaluation point to the fact that the Institute carries out this mandate well and is touted as a leader within the international metrology community.

Summary of Findings and Recommendations - Cost-Effectiveness

Collaborative Activities

The collaborative activities undertaken by NRC-INMS with other NMIs are highly regarded in the international metrology community, where the Institute has clearly established its role as that of leader. Opinions within Canada were more mixed, however. It appears as though the definition of collaborative research is different for NRC-INMS than it is for its partners, who typically perceive the Institute as a service provider rather than collaborator. In addition, very little data were available on the concrete outcomes of these partnerships, especially from collaborators.

Greater linkages within Canada in coming years may result in the identification of problems or issues of relevance to OGDs and Canadian industry. Working with partners on resolving these issues through measurement science will raise the profile of the Institute while protecting Canadian trade interests, enabling innovation, and enhancing Canadians' quality of life.

Recommendation 4: The Institute has made some headway in recent years in terms of actively pursuing research collaborations and of increasing the awareness of potential collaborators who support/have a stake in the Institute's mandate. These activities should be encouraged further, not only at the executive level, but for all scientific staff of the Institute. This should include a greater level of involvement from the Business Development Office in the framing of research agreements in order to encourage the development of collaborative activities.

Management Response and Proposed Actions: Accepted. Our natural partners for collaborations related to new primary standards for physical measurements and certified reference materials for chemical measurement are other national metrology institutes (NMIs). Many of our scientific staff are engaged in such collaborations. Many of them are not formalized in collaborative R&D agreements, but they are a vital means by which we execute our mandate. Collaborations with other NRC institutes are also increasing, in particular in the area of nanotechnology. Selection criteria for new research projects will favor collaborative projects. The Business Development Office (BDO) will actively participate in the development and negotiation of research agreements. The BDO will also ensure that agreements are worded in accordance with NRC's directives, appropriate levels of signatures are obtained and the administration of each project reflects the principles of good project management.

Resource Allocation

NRC-INMS derives its resources from continuing funding as well as from revenue obtained for services rendered. When compared to other NMIs, the evaluation showed that it is achieving important results on the international stage with fewer resources than most of the major players.

However, the current allocation levels will become increasingly insufficient in future years due to the increase in activities. The Institute may experience difficulty in maintaining its core activities as Canada's NMI given the need to conduct R&D in emerging areas of science.

Summary of Findings - Effectiveness of Design and Delivery

Recent changes in the administrative structure of the Institute are generally perceived as positive and have had some impact on increasing collegiality within NRC-INMS. The fact that the Institute is divided into four separate buildings is a cause for concern among some researchers, although it does not currently seem to have a significant impact on intra-Institute communications and management. Other notable administrative achievements include the implementation of the quality management system (QMS) and the standardization of the fee structure. A comparison to other organizational structures used in different NMIs revealed that the current model is most appropriate for NRC-INMS given its size, resources, and mandate.

Date modified: