ARCHIVED - Audit of Research Facilities Management - November 2014

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

ARCHIVED - Audit of Research Facilities Management - November 2014 (PDF, 541 KB)

Executive Summary and Conclusion

Background

This audit report presents the findings of the National Research Council Canada's (NRC) Audit of Research Facilities Management.

Audit Objective

The objective of this audit was to provide assurance that an adequate research facilities management control framework is in place to enable the delivery of NRC's research programs in a manner that is compliant with applicable Government of Canada and NRC policies, procedures and regulations.

Raison d'être

Research facilities management has major strategic, operational, financial, and risk implications for NRC and ensures that the optimal environment exists to support the management, operation and maintenance of research facilities for the benefit of Canadians. Effective research facilities management is codified in the NRC 2013-18 Strategy and is a key pillar to achieving NRC's vision as one of the top five Research Technology Organizations (RTO) in the world. Research facilities management was identified as a high audit priority area in the NRC 2014-15 to 2016-17 Risk-Based Internal Audit Plan.

Stewardship of research facilities is driven by long-term strategic planning and has significant downstream impact on operational plans, directly feeding into NRC's Investment Plan and portfolio annual operating plans. Across NRC, research facilities and equipment comprise a historical investment of more than $600M in assets directly supporting research and revenue generating activities. NRC's research infrastructure includes assets found in leading private industry labs as well as one-of-a-kind, unique to Canada, instruments.

Audit Opinion and Conclusion

Overall, we found that NRC manages its research facilities in a manner generally compliant with the expectations and key principles outlined in Government of Canada policies and regulations including the Policy on Management of Material, Treasury Board Accounting Standards, the Policy on Investment Planning – Assets and Acquired Services and the Policy on Management of Real Property. The audit noted that the current research facilities management framework requires improvements to better support the delivery of NRC's research programs and to achieve NRC's vision as a leading RTO.

We found that the decentralized approach, unclear standards and expectations for planning, costing and changes to corporate and portfolio level internal business processes contributed to an inconsistent implementation of NRC's research facilities management framework. Within the current transformation process, internal management practices are approaching a steady state and structures have been put in place to support more robust research facilities management practices. However, we found that research facility managers are still addressing research facilities issues in a reactive manner. Current facilities management practices are not sustainable in the long term and do not support effective management from an NRC-wide perspective.

We noted numerous initiatives in-progress to address some of the issues we identified through the course of the audit. NRC has defined a costing model and a general framework with which to manage research facilities that adequately reflects the maturity of the organization. Strengths noted by the audit include system capabilities for cost tracking and resource management, financial performance reporting capabilities, systems integration capabilities between NRC's financial and project management systems, a well-defined real property management framework and base building (non-research real property) management approach that supports synergies with research facilities management, and knowledgeable staff and expertise.

Our recommendations address the root cause of findings taking into consideration the interrelationship between research facilities planning, costing, and operations. Harmonization of processes and practices across portfolios and between research functions and corporate services is vital to sustain NRC's transformation and ensure that NRC research facilities meet current and future research needs for the benefit of Canadians.

Table 1 below summarizes the audit conclusions by line of enquiry. Following this table is a list of related recommendations designed to improve current business processes.

Table 1: Summary of Audit Conclusions

Line of Enquiry Assessment Associated
Recommendation(s)
1. Strategic and sustainable planning Needs Improvement Footnote 1, Footnote 2, Footnote 5
2. Costing Needs Improvement Footnote 3
3. Operations Adequate Footnote 4
Overall Audit Conclusion Needs Improvement

Summary of Recommendations

Footnote 1

Recommendation 1: NRC Senior Executive Committee (SEC) should appoint a champion to spearhead key initiatives aimed at implementing the research facilities management framework to better support the achievement of strategic objectives and delivery of research programs.
[Priority: HIGH]

Return to tablenote 1 referrer

Table note 2

Recommendation 2: The NRC SEC Champion for research facilities management, with the support of NRC Planning and Reporting Services (PRS), should ensure structured research facilities planning practices through the development and consistent application of guidelines, templates and tools, as well as the provision of additional training on the use of new and existing tools.
[Priority: HIGH]

Return to tablenote 2 referrer

Table note 3

Recommendation 3: The NRC SEC Champion for research facilities management, with the support of NRC Finance Branch (FB), should ensure consistent application of costing methodologies and tools to provide reliable information for decision-making.
[Priority: HIGH]

Return to tablenote 3 referrer

Table note 4

Recommendation 4: The NRC SEC Champion for research facilities management, with the support of NRC Knowledge Management (KM), should ensure that NRC develops a research facilities marketplace for more effective and efficient resource planning and utilization.
[Priority: MODERATE]

Return to tablenote 4 referrer

Table note 5

Recommendation 5: The NRC SEC Champion for research facilities management, with the support of NRC Planning and Reporting Services (PRS) should align research facilities performance management across corporate, divisional and portfolio levels to ensure that operational plans enact management strategies and encourage desired research facilities management behaviour.
[Priority: HIGH]

Return to tablenote 5 referrer

Statement of Conformance

In my professional judgment as the Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the audit opinion and conclusion. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Alexandra Dagger, Chief Audit and Evaluation Executive

NRC Audit Team Members:
Irina Nikolova, FCCA, CIA, CISA
Andy Lang, CIA
Julien Dussault, BCom
Jon Byford-Harvey, BComm

1.0 Introduction

This audit report presents the findings of the Audit of Research Facilities Management. The decision to conduct this audit was approved by the President following the recommendation of the Senior Executive Committee and thereafter by the Departmental Audit Committee on June 26, 2013 as part of the NRC 2013-14 to 2015-16 Risk-Based Audit Plan (RBAP).

1.1. Background

NRC research facilities management operates in a complex environment of Treasury Board (TBS) and NRC policies, directives and legislative requirements. Concurrently, the diversity of research activities across NRC requires a broad asset base to support program needs. NRC currently has more than 600 research facilities and equipment work centers (portfolio-defined groupings of research assets) representing in excess of 8500 unique assets within its resource management system.

As illustrated in Figure 1 below, asset management is a joint activity between Administrative Services and Property Management (ASPM), managing NRC's base building (real property) assets, and research portfolios responsible for research assets.

Figure 1: Research facilities management responsibilities at NRC

Figure 1: Research facilities management responsibilities at NRC

Research facilities management within NRC is a cross-functional activity requiring oversight, integration, and cooperation between resource users (NRC programs), research managers and owners (NRC portfolios) and NRC's common services functions. Division Heads have delegated research facilities management and stewardship responsibilities to General Managers.

Research facilities may comprise standalone structures (ex. wind tunnels, standalone test chambers, etc) or integrated components of base buildings (ex. clean rooms, labs or test chambers found within a building). Capital improvements, utilization and relevant certification and registration with accrediting bodies fall under the purview of General Managers with support from ASPM. Maintenance and repairs on research facilities and equipment are part of routine operating activities and are planned, budgeted, and managed by portfolios.

ASPM is steward of NRC's real property and base building infrastructure such as building envelopes (ex. building shell, roof, etc) and other infrastructure not directly related to research activities. ASPM coordinates construction activities and supports portfolios in the development, construction, and maintenance and upkeep of research facilities through its procurement and engineering functions.

NRC Finance Branch (FB) has defined an organization-wide costing framework with supporting templates and tools. Finance Branch is responsible for the implementation of the costing model including providing training and guidance with respect to financial management of research facilities.

NRC Planning and Reporting Services (PRS) supports portfolio and program planning activities and manages NRC's programs, projects, and departmental Investment Plan. PRS also coordinates program, portfolio and branch activities, providing templates and tools for strategic and operational planning.

Figure 2: NRC research facilities management framework

Figure 2: NRC research facilities management framework

Adequate management and oversight over NRC's research facilities is vital to ensure that NRC is able to deliver on its objectives, attract leading talent to achieve its vision, and maintain the trust of Canadian taxpayers, clients and other stakeholders in the stewardship of Canadian research infrastructure. Legacy and inadequate research facilities present challenges with respect to adequate maintenance funding, access to spare parts and vendor support, and the ability to deliver meaningful outcomes and perform leading edge research activities. As well, inadequately maintained facilities can have direct and indirect impacts on the safety, health, and well-being of staff, visitors and clients.

1.2. Context

Our audit conclusions were developed within the context of changes occurring across NRC including, but not limited to, maturing corporate, divisional and portfolio business practices, Government of Canada mandates as well as evolving corporate operating expectations and performance management standards. Numerous forces of change have introduced additional complexities to NRC's research facilities management framework. Audit findings and recommendations were developed within the context of this environment of continuing change to strengthen NRC's research facilities management framework.

The research facilities management framework must be viewed holistically to appreciate the cross-functional nature of the topic and the need for expertise and collaboration from a variety of stakeholders. As a core element of NRC's raison d'être, ensuring that current and future research facilities needs are satisfied is a challenging balancing act of human and financial resources.

1.3. About the Audit

Objective

The objective of this audit was to provide assurance that an adequate research facilities management control framework is in place to enable the delivery of NRC's research programs in a manner that is compliant with applicable Government of Canada and NRC policies, procedures and regulations.

Scope

For this audit (as presented in Figure 3 below), research facilities include all assets, buildings and structures directly supporting research activities. It may include components such as a lab within a base building (structure generally built for use as an office) or a standalone structure such as a wind tunnel that may contain limited office space.

The audit assessed the management framework in place to support research facilities management in fiscal year 2013-14 taking into consideration structures, processes and procedures planned for implementation in following fiscal years. Four portfolios were selected across three provinces with representation from each of NRC's three research divisions.

The audit did not assess the adequacy of NRC's frameworks for costing or pricing of research facilities. The audit scope excludes physical and operational security of research facilities, occupational safety and health, real property management, and infrastructure and processes related to the management of the building envelope or infrastructural components unless directly related or contributing to research activities.

Audit fieldwork took place between March and July 2014.

Figure 3: Audit scope

Figure 3: Audit scope

Approach and Methodology

The audit was conducted in accordance with generally accepted professional auditing standards of the Institute of Internal Auditors (the IIA) and the standards and requirements set out in the Treasury Board Policy on Internal Audit. The audit criteria, presented in Appendix A, were primarily derived from the TB Policy on Management of Real Property, TBS Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (2011) and, as applicable, the NRC Financial Management Manual and the NRC Cost Accounting Guide. Criteria were discussed with senior management in advance of the audit.

The audit addressed the audit criteria as they existed at the time of examination.  Audit recommendations take into account ongoing internal initiatives that may impact research facilities management processes and functions such as ongoing initiatives to transform NRC's common services. The audit methodologies were selected to ensure that the root cause of findings was identified and to ensure recommendations add value for NRC. Methodologies are detailed in Figure 4 below.

Figure 4: Overview of audit methodologies

  • Review of relevant facilities management documentation and records including but not limited to: corporate, divisional and portfolio strategic and operational business plans, internal process documents and maps, research facilities costing templates, tools and guidance
  • Trend analysis of corporate performance data
  • Interviews with Division Heads, General Managers, Research and Operations Directors, facility managers and operational level staff in the National Capital Region (NCR) and at regional locations
  • Site visits across six NRC research campus locations in Quebec, Ontario and British Columbia
  • Review and analysis of results of corporate monitoring activities
  • Data mining of raw SAP project, transactional and financial data
  • Research on best practices and processes of other research organizations and from academia
  • Relevant findings from current portfolio evaluations
  • Application of substantive verification procedures for a sample of 16 research facility work centers

2.0 Audit Findings

This section below presents the detailed audit findings with accompanying assessments. Suggested management priorities for implementation of action plans to address risks are identified as high, moderate or low.

Audit Findings Assessment
Line of Enquiry 1: Strategic and sustainable planning Needs Improvement

Criterion 1.1: Strategic and operational planning guide facilities management activities and demonstrate the whole-of-life concept for effective stewardship of assets

Effective and integrated planning are key elements to ensuring that NRC's current and future research facilities needs are addressed within an environment of constrained resources. Adequate planning supporting investment and divestment decisions, maintenance, and renewal maximize the utility and benefits of research assets, which underpin a key pillar in NRC's Strategy to support access to national scientific infrastructure for benefits to Canadians.

Strategic and operational planning activities are inconsistent across portfolios and do not provide an NRC-wide perspective with respect to current or future research facilities needs. The lack of consistent planning information precludes effective optimization of resources to meet strategic and operational targets.

Planning structures and processes have been implemented and whole-of-life planning concepts are reflected in planning templates; additional training and guidance is necessary to ingrain these practices as part of ongoing planning activities. Opportunities exist to integrate planning and costing templates for internal efficiency.

Research facilities organization

Research facilities are managed in a decentralized fashion where Division Heads have delegated responsibility to individual portfolios and General Managers. NRC is continuing to refine its business planning approach following an internal transformation that began in fiscal year 2013. Our review of Divisional strategic plans identified inconsistent research facilities management guidelines. While some Divisional strategic plans outlined minimum standards and divisional approaches, others only specified roles and responsibilities, delegating full responsibility to General Managers.

We found that the decentralized approach and the absence of definitive guidelines to portfolio strategic and operational planning resulted in application of inconsistent assumptions and irreconcilable planning information. We noted that while current planning templates provided a general outline of future needs, completed planning documents varied in depth and quality. We noted that similar planning information across portfolios were dispersed inconsistently between strategic and operational plans. The absence of standardized planning practices across NRC complicate operational efforts to streamline and develop common internal business practices. Differing planning approaches also precludes the ability to compare and share best practices and assess facilities performance from an NRC perspective.

NRC has a diverse portfolio of large scale, capital-intensive research facilities that require significant lead-time in concept development and strategic planning to ensure that NRC is adequately positioned for relevance in leading research and development activities. While portfolio strategic plans have a five-year planning period, a more advanced outlook may be required to develop, build, and operationalize leading edge research facilities. We also noted a lack of integration between portfolio strategic plans with ASPM planning activities. Misalignment of long-term planning between research facilities, base-building infrastructure and accommodation planning precludes the development of an integrated, and comprehensive asset management plan for NRC to demonstrate integrated planning and stewardship of resources.

A noted practice among institutions with large real property footprints is the development of a campus master plan. A campus master plan brings together the guiding principles of property and asset management with a long-term outlook of organizational needs to support the development of an optimal environment to deliver on research program objectives.

We found that while five-year strategic plans for research portfolios and ASPM adequately reflect short to medium term planning requirements, a longer term outlook may be warranted to integrate research facilities planning, base-building infrastructure and accommodation planning activities. Within the context of NRC, a campus master plan would integrate internal services optimization targets with portfolio and program growth objectives and bring transparency across real property and research facilities planning.

The Vice-President, Corporate Management and the Executive Vice-President are currently engaged in an exercise to determine NRC's long-term potential growth and to identify research facility needs over a 15-year or more time horizon.

The absence of consistent planning assumptions and approaches precludes a holistic assessment of research facilities needs and increases the difficulty of obtaining an organizational wide perspective of resource requirements. At the portfolio level, we did not identify common or consistent criteria to support the prioritization of research facilities for investment or renewal needs. While strategic and operational plans identified resources, they lacked sufficiently detailed information on the condition, strategic fit and utility of existing facilities to support decision-making.

A reconciliation process to match research facilities resource needs with available portfolio resources does not exist nor do guidelines exist to implement such a system. We found that NRC common services, such as Finance Branch (FB), Administrative Services and Property Management (ASPM) and Human Resources Branch (HRB) were not able to consistently in a streamlined fashion identify the resourcing needs of research groups due to differing assumptions, approaches and methods used in resource planning and definition.

We could not identify a common or consistent approach for portfolios to validate their resourcing assumptions and needs. This has downstream impacts on internal performance measurement as facility activity rates are used to generate internal charge-backs to portfolios based on defined resource needs identified in operational plans. The lack of resourcing validation increases the risk that activity rates are derived from unrepresentative data resulting in an under or over-allocation of internal costs to facility users and leads to inconsistent performance data at the portfolio level.

At minimum, portfolio level strategic and operational plans should have defined basic parameters to facilitate comparison of research facilities needs and to prioritize investment requirements. Ranking facilities would help risk manage an aging asset base, highlighting the state of current research facilities infrastructure, expected investment requirements for major and minor capital, and support prioritization of limited resources to investments of greatest program impact.

Planning templates should also consider more in-depth definition of resource needs from NRC common services such as ASPM procurement, construction and project management needs and hiring services from HRB. As well, a process should be defined to validate resource assumptions and needs to match facilities costing requirements with overall portfolio financial plans.

We noted progress in the development of consistent strategic and operational planning templates for the FY2015-16 planning cycle.

See Recommendation 2.

Planning and systems integration for collaboration and resource sharing

SAP Project System (SAP PS) is NRC's corporate project management system. The system includes a comprehensive database of defined resources (work centers) which include individual employees as well as research facilities. While SAP PS provides a listing of research facilities resources, descriptions often reflect the business activities and preferences of individual portfolios making it difficult to identify specific resources capabilities outside the circle of the work center owner. Research facilities data is entered into SAP with no standard nomenclature and naming standards to identify technical specifications. The inconsistent use of SAP data fields precludes a comprehensive listing of available research facilities for effective planning and sharing of resources.

The absence of an integrated NRC resource marketplace to support integrated planning, greater inter-portfolio sharing of resources and funding to areas of priority represents a limitation of the current research facilities management framework. An opportunity exists for NRC to consider making the research marketplace accessible to other government departments and/or collaborators for better utilization of existing resources and potential economies of scale in purchasing.

See Recommendation 4.

Lifecycle planning and considerations

We found that investment planning templates have been updated requiring investment submissions to define expected utilization, in empirical terms, across different programs. We noted that planning templates do not define expected asset operating capacity to better illustrate the performance attributes of the investment; defining capacity during the planning process would streamline the internal costing process and support post investment performance validation (See Line of Enquiry 2).

In a sample review of approved investment plans, we noted limited consideration of research facilities disposal with no consideration of disposal costs in financial analyses. In one investment submission to replace legacy equipment, a portfolio maintained the legacy research facility to cannibalize for spare parts. Storage and associated opportunity costs of not disposing the decommissioned facility in a timely manner were not considered. Nor were considerations made for how to remove the equipment, given its size, once it was stripped of available spare components.

In one instance, we identified an investment submission to increase facility capacity four-fold and thereby reduce the activity rate by approximately 24% excluding consideration of benefits associated with higher reliability and lower operating and maintenance costs. At the time of the audit, the out of service facility was idle awaiting vendor support. The submission was not approved due to priority investments but the analysis would have provided empirical information to support decision-making and investment ranking. The addition of a capacity and utilization analysis would improve decision-making through improved quality of information and provide a benchmark for cost-benefit analysis assessing overall benefit between repair costs and lost business opportunities versus reinvestment in the facility.

NRC's investment plan decision-making and integration of cross-functional processes would be strengthened by including in investment planning templates:

  • Asset capacity analysis tools to assess varying levels of utilization to demonstrate return on investment and utility;
  • The Finance Branch facility costing template as part of project close-out activities to streamline close-out and costing processes and operationalize new capital acquisitions in an efficient manner;
  • Cost-benefit analysis tools to drive decision-making with regard to maintenance and repairs versus investment or reinvestment to ensure that NRC can deliver quality services to clients at an acceptable cost to the organization; and
  • Facility retirement guidelines to address the potential need to handle and dispose of hazardous materials such as biological, chemical or radioactive components of research facilities or their by-products

See Recommendation 2.

ASPM's Role in Facilities Management

ASPM manages base building infrastructure and space use. ASPM plays an integral role in research facilities management, providing support and coordination services to research facility managers. Consistently, across sampled portfolios we noted difficulties in engagement and communication between research portfolios and ASPM, adversely impacting planning and operational activities.

Further improvements are required to the interface and liaison activities between ASPM building coordinators (individuals responsible for the day-to-day operations of base building) and research portfolios. In one instance, we noted that the resident portfolio of a building was not informed in a timely manner of a new building tenant. The new portfolio tenant was working with biological materials, which have safety and health as well as research quality assurance implications from a space planning perspective. As well, co-locating research groups dealing with biological, chemical, radioactive and other dangerous substances may require adjustments to existing building exhaust, ventilation and utility systems to maintain health and safety standards.

ASPM has undertaken efforts to build linkages with research portfolios through the creation of a Client Services Group as a single point of contact for portfolios requiring building related support including building requirements impacted by or impacting research facilities management. We also found that the ASPM Client Services Group is developing an NRC-wide project tracking system to provide visibility to all projects requiring ASPM support or impacting ASPM resources. Data integrity for the database requires improvement; we found the database to be incomplete due to a lack of consistent communication with portfolios and PRS who oversees NRC's investment planning program and maintains investment planning information. Effective and timely communication between ASPM, related common services, and research functions is vital to ensure that capital projects are executed in an efficient manner to support program delivery. A complete and integrated project database ensures that existing base building infrastructure can support facility additions and upgrades, projects comply with health and safety requirements, and limited resources are prioritized or options to contract out are explored where necessary.

Numerous initiatives are underway to improve information flow and increase communication between ASPM and research groups for more effective research facilities management practices including:

  • The creation of a Client Services Group as the point of contact for portfolio-ASPM engagement;
  • The creation of an NRC-wide project database to support more robust ASPM resource planning and allocation;
  • An NRC-wide space inventory project to identify space use, needs and gaps to support portfolio and program needs; and
  • Engagement of PRS to ensure that program investment plan submissions receive adequate ASPM support to reduce impediments and improve the flow of project review and approval.

A resource marketplace would support visibility to available building space for portfolio and program needs to support resource sharing and optimization.

See Recommendation 4.

Streamlined planning templates would support better definition of portfolio resource needs from common services such as ASPM to facilitate more effective resource planning and prioritization.

See Recommendation 2.

Criterion 1.2: Plans are in place to support current and future facilities management human resource needs

Research facilities rely on skilled and trained personnel for effective maintenance and upkeep to maximize the utility and return of these assets for the benefit of Canadians. Ensuring that adequate human resources are available to plan the use and maintenance of research facilities demonstrates effective stewardship.

We found that portfolios have taken unique approaches to research facilities management. Some capital-intensive portfolios with large-scale research facilities and equipment have dedicated technical staff for maintenance and operations while other portfolios push facilities ownership, such as maintenance and care, to researchers. We noted diverse approaches to research facilities maintenance including in-house, on the job training, specialized training by vendors, vendor support contracts as well as outsourced management. At the operational level, we noted that portfolios are exploring alternative work arrangements such as shift-based work schedules and adding shifts to maximize the capacity and use of research facilities.

Division and portfolio level strategic and operational plans provide sufficient detail in identifying human resource related risks and related mitigation plans. Divisions and portfolios have generally identified anticipated resource needs and challenges highlighting areas of concern to senior management. The majority of portfolio level strategic and operational plans provided analyses of personnel requirements and contingencies in anticipation of retirements and staff departures; some portfolios also provided general catalogues of their capabilities and skills sets to identify their human capital requirements in relation to program activities as well as research facilities management. While current resources and future needs were defined, we noted that plans were incomplete with respect to classifications of new hires, timelines for hiring and expected resource needs from HRB to facilitate hiring actions. We found elements of succession planning in strategic and operational documents, which varied in detail and quality.

Approximately 400 staff have been trained in the use of NRC's enterprise project management system, SAP Project System. We found inconsistent levels of understanding and use of the resource management system for research facilities planning and operations precluding NRC's ability to reap the benefits of a structured and consistent approach to schedule, optimize capacity and integrate facilities management activities with NRC's financial management and reporting systems (See Line of Enquiry 3 – 3.2).

Portfolio plans in place are still inconsistent in their definition and depth of identifying human resource requirements without a consistent base of assumptions to align strategic and operational direction.

Line of Enquiry 2: Costing Needs Improvement

Criterion 2.1: Facilities management includes clear consideration of cost information with a view of achieving organizational objectives

Effective costing processes are vital to ensure that management has access to adequate and representative information on which to make strategic and operational decisions. Sufficient support, guidance, and tools for research facilities costing ensures that NRC's costing model is implemented as intended and is balanced to reflect the unique operations of research portfolios and the operational consistency required for effective financial management.

We found a defined costing model and framework in place to support research facilities costing including a research facilities costing template to support consistent collection of information. NRC FB has defined an organizational standard for costing through the NRC Finance Branch Cost Accounting Guide. Additional support is provided through the NRC Financial Management Manual (FMM) and its associated directives. Effective June 2014, following a reorganization of NRC Finance Branch, research facilities costing templates and tools are being reviewed.

NRC has defined a costing model for research facilities commensurate with the maturity of the organization. Implementation of the model has been challenged by a lack of consistency in support, guidance and standards. There is a risk of being unable to focus activities towards common objectives.

Defining units of analysis

NRC has three defined work center types in SAP PS, facilities, equipment and services. Facilities and equipment are discrete resources that may represent individual assets or groupings of assets and have unique activity rates. Services are work centers with defined activity rates that are organized around capabilities or specific deliverables. A service may encompass a standard task such as sample testing where costs to deliver are generally known and estimable (firm-fixed cost type work). The unit of measurement for services may also vary. We noted denominators on a per sample basis, per test basis, per square feet per day basis, and a per unit cost basis among others. Service work centers are associated in NRC's financial system to a facility or equipment. Utilization reporting would take into account time and costs charged via facility or equipment rates as well as for services to unique work centers for a comprehensive view of facility usage.

Portfolios are responsible for determining research facilities activity rates. We noted that the delegation of costing responsibilities to portfolios provided a wide degree of latitude in the interpretation of NRC's costing principles. Portfolios took varied approaches to organizing facilities for costing and definition in SAP PS. The organizational approaches we observed included: by workflow where related assets that had to be used in sequence to achieve an outcome were grouped as a single work center; by a variation of workflow where each asset is a unique work center; by business line, consolidating assets into a "toolbox" where individual assets may be used independently of each other but share a common activity rate; and a hybrid approach where common equipment is consolidated together and unique assets as separate units.

One portfolio took the approach of consolidating general facility and laboratory equipment into a generic facility chargeable based on an allocation formula of staff hours. Where a facility provided a unique functionality or capability, it was costed as a standalone work center. We found that the lack of guidance in determining units of analysis resulted in one portfolio having more than 100 unique facility work centers and another having just six.

A decentralized approach to research facilities definition increases the potential for resource duplication due to inconsistent naming conventions and the lack of an easily accessible listing. In the absence of a comprehensive research facilities database, existing capacity of NRC resources is not effectively utilized.

As a general practice, we noted that academic institutions and other research and technical service providers define research facilities at a granular level for equipment with specialized functionalities. Facilities are defined at the individual asset level and are costed and administered as unique pieces of equipment. In administering small-scale equipment such as microscopes, centrifuges and analysis instruments found in general wet / dry labs, we noted practices of consolidating assets and charging a standard rate to ease administration.

An opportunity for improvement exists for NRC to develop additional guidance and clarification on the appropriate unit of analysis for portfolios to plan, cost, and use their research facility holdings to ease administration and oversight of research assets. Principles could include:

  • Specialized facilities or equipment are accounted for individually as unique work centers with unique activity rates. Where the facility is related to other work centers or pieces of equipment from a business process perspective, they should be associated in NRC's financial system to a cost center for reporting purposes; and
  • General research facility and lab equipment should be consolidated into general research facilities that may use an allocation formula or other portfolio defined measure for utilization to simplify administration.

See Recommendation 3.

Theoretical capacity is the maximum theoretically possible output for a facility in the absence of any planned or unplanned downtime. Practical capacity is the available output of a facility taking into account planned maintenance activities and planned capacity is the output budgeted for a specific period based on client demand.

We found inconsistent definitions of practical capacity, the baseline value for determining planned capacity, due to insufficient guidance with respect to costing approaches and facility performance objectives. NRC's performance management framework includes an indicator of facility utilization to assess the efficiency with which the organization leverages its facility resources. We noted diverse definitions of practical capacity including; capacity based on a 24-hour, 365 day work schedule (8760 hours); capacity based on a 37.5 hour work week (1950 hours); capacity based on operator availabilities (1450 hours); and other portfolio defined variations.

Efforts have been made to address inconsistencies in practical capacity definition and to define a corporate standard for facilities utilization. Different working groups have been formed to assess and define a corporate measure for research facilities utilization as of August 2014.

Additional guidance on practical capacity for consistent interpretations of baseline capacity would support assessments of research facility utilization and efficiency of resource use. Defining utilization benchmarks for different facilities would ensure that the unique operating characteristics of NRC's vast facilities holdings are reflected in utilization assessment.

Use of planned capacity as the denominator in deriving research facility activity rates for all facility cost categories would support internal performance assessment.

See Recommendation 3.

Use of Costing Templates

Finance Branch uses manual spreadsheet tools to derive research facility activity rates. Our review of a sample of work centers noted that costing workbooks did not always contain an audit trail or substantiation for values used such as for the number of assets within the work center, the amount of depreciation to be charged annually and the square footage used to generate a Building Cost Assessment (BCA) allocation. In some instances, we noted that modifications had been made to templates by individual research facility managers to provide additional clarification to defined activity rates. The template modifications disrupted FB activities to collect and consolidate costing workbook information and precluded the ability to compare and benchmark facilities across portfolios and in some instances, across geographical offices of the same portfolio.

We found that costing templates spread labour costs, operations and maintenance expenses, common services support costs and only one of three components of BCA (utility costs over planned capacity). Research facility related depreciation, base building related maintenance, PILT and base building amortization are spread across a research facility's practical capacity in lieu of planned capacity contrary to budgeting guidance provided by Finance Branch. Unless a facility is 100% efficient, based on planned capacity being equal to practical capacity, internal research facility costs will not be recovered.

In one instance, we noted that the facility activity rate was under-representative of defined costs due to the use of actual square footage in lieu of chargeable square footage mandated by ASPM. The research facility's base building footprint was understated resulting in an under-representative activity rate. The lack of costing information validation increases the risk that activity rates are not representative of defined operational costs with implications on financial and resource performance management.

NRC's space management system, ARCHIBUS, provides data using the metric system (square meters) while the FB facility costing template requires building space data for BCA using imperial units. The requirement for measurement conversion resulted in minor deviations in building space definition due to round-off error. We noted that space information, for BCA purposes, is not updated in a timely manner and ASPM places reliance on portfolios and branches to report and validate space use changes.

There are a number of inconsistencies in the use of research facilities costing templates across portfolios. As a result, research facility activity rates do not effectively operationalize NRC's defined costing model and rates for similar facilities are not consistent nor comparable.

The use of costing templates would be facilitated if NRC's costing model is implemented as defined in existing costing guidance documents by:

  • Validating completed costing templates; and
  • Aligning costing templates with information sources (use of square meters in lieu of square feet for ARCHIBUS data)

See Recommendation 3.

Research facilities costing model monitoring

The FB facilities costing process map outlines a bi-annual review cycle for research facilities activity rates. We did not identify further guidance on updating activity rates when changes are made to a research facility. The absence of a mechanism to review activity rates outside of the bi-annual cycle introduces the risk of facility and equipment rates not being updated in a timely manner to reflect additions or retirements of assets, changes in labour requirements for maintenance and upkeep and general expenses related to facility or equipment operations. Concurrently, general increases in utility costs and PILT would not be reflected in a timely manner, directly impacting internal performance measurement and assessment of program activities, and portfolio operations. At minimum, internal cost rates should be updated on an annual basis or if changes are made to a research facility's cost structure.

A review cycle for research facility activity rates (at minimum, on an annual basis) as well as a process to update rates when facility cost structures change would allow facility and equipment rates to be updated in a timely manner. Consideration could also be given to integrating the annual FB asset verification exercise with costing activities to validate the existence of assets, reassess their condition and take appropriate action to update NRC's asset inventory. This could be done in conjunction with portfolio analyses of facilities operations and maintenance costs for a comprehensive activity rate update.

See Recommendation 3.

Change within the FB Costing Group due to internal reorganization, staff attrition, and new reporting relationships resulted in inconsistent support and guidance to portfolios and unclear roles and responsibilities between FB costing group staff, FB comptrollers, and portfolio facility managers.

Validation of costing information was difficult due to the lack of a defined validation process, the lack of historical information for comparison purposes, inconsistent tracking methodologies across portfolios, and difficulty in validating the correct use of information provided by ASPM and other common service groups. While some portfolios included assumptions and background information to support costing activities, others provided little in audit trail. A lack of costing template controls precluded consistent application of defined costing principles.

We noted that FB comptroller roles and responsibilities with respect to facilities resource validation were unclear and that completed costing templates were reviewed and validated in an inconsistent fashion. The lack of consistent guidelines and direction resulted in inconsistent interpretations of costing principles and variations in assumptions used to drive facilities costing. Portfolio management priorities to implement programs compounded delays in costing. Concurrently, internal reorganization and new processes within ASPM resulted in inconsistent BCA information being provided to portfolios for costing purposes. BCA data was continually updated as changes and details were confirmed and portfolios received different baseline data sets as they developed internal processes to support research facilities costing precluding the ability to reconcile BCA allocations to research facilities across NRC.

More direct involvement in research facilities costing activities by comptrollers, as the key portfolio financial management experts, would improve the application of the NRC costing model. Clarity around roles and responsibilities with respect to facilities costing between NRC's costing group, portfolio comptrollers, and staff is needed to ensure the consistent application of NRC's costing model.

See Recommendation 3.

Formalization of the process to confirm building space utilization and occupancy ensures that ARCHIBUS data is up to date and adequately reflects building space use on an ongoing basis. ARCHIBUS and SAP asset module information should be reconciled at least annually to verify assets and their location.

See Recommendation 3.

We noted that facility activity rates are manually entered in SAP PS and are not split out in their component elements of labour, operations and maintenance, depreciation, common services settlements, and BCA. Research facilities are planned and managed manually through spreadsheet tools precluding the ability to reconcile facility resource requirements in aggregate at the portfolio operational budget level.

We did not identify monitoring mechanisms to ensure the integrity of costing information such as labour requirements, maintenance costs or application of BCA.

Defining a standardized cycle to review and update facilities activity rates and standardizing templates to match NRC's space planning system would facilitate reconciliation of space use, support the proper reflection of building footprint in NRC's research facilities, and reduce the burden on research functions for costing activities.

The development of a process to verify under or over applied BCA as part of a comprehensive research facilities variance analysis process would support the refinement of NRC's research facilities costing activities.

See Recommendation 3.

Tracking utility costs

We noted that in general, large-scale research facilities do not have separate utility meters to capture granular utility expense data and NRC's legacy base building infrastructure further complicate efforts to capture information. Where facilities lack independent utility meters, utility consumption is subsumed in the BCA charge creating potential issues with cost attribution from an internal costing perspective. The lack of unique operating data for large scale facilities that may consume significant utilities limits the ability to properly cost research facility activities and operate efficiently and effectively for the benefit of Canadians. We noted that ASPM considers the addition of metering infrastructure as part of recapitalization activities but no formal decision-making criteria have been defined and the process is ad-hoc. Efforts to capture more granular costing data are challenged by legacy utility systems, the lack of incentives for more granular costing information; and the cost to meet regulatory requirements such as revenue grade utility meters when reselling electricity as part of testing activities. Inconsistent definition of units of analysis impact the placement of utility meters and therefore the ability to collect and analyze consumption data (See Line of Enquiry 2 – Unit of analysis).

A review of all completed portfolio operating plans illustrated inconsistent assumptions and approaches to research facilities planning. Four plans were found to use a two-percent growth rate in BCA while three of 12 portfolios did not account for any inflationary increases. Another three of 12 portfolios had BCA growth projections between five and ten percent. The lack of consistent assumptions to BCA impacts overhead allocation to facility activity rates and internal performance assessment of program activities. As well, BCA is a cost allocation and is not controllable by portfolios but unrepresentative allocations may adversely affect portfolio financial performance through their statement of operations.

There are opportunities to explore greater use of utility metering tools for more accurate assessments of utilities consumption as ASPM undertakes recapitalization projects across NRC for the maintenance and upkeep of general base building infrastructure as well as part of research facilities recapitalization and upgrade initiatives. More refined and granular utility usage data introduces increased accountability at the end-user / facility manager level who generally have the greatest influence on operating costs.

As ASPM is responsible for base-building management, it should define a standard growth rate for BCA for portfolio planning purposes for harmonization of assumptions. More representative BCA projections and cost allocation to portfolios would be supported by ASPM's initiative to inventory portfolio space holdings.

See Recommendation 2.

Ongoing efforts to improve NRC's costing model

In recognition of the higher utility requirements of labs and research facilities relative to general office and administrative functions, ASPM is planning to introduce a new formula for base building cost allocation. Effective communication of the new formula is vital to ensure transparency, understanding and buy-in from research functions.

Integral to the new BCA cost allocation formula is a full inventory of NRC's real property holdings and space footprint. As of Q1 of fiscal year 2015, ASPM is validating the building footprints of all portfolios and concurrently documenting the location and space use of research facilities within those base buildings. We noted that while the exercise will provide an updated space inventory, ASPM is relying on portfolios to self-report space use and holdings. There is an inherent risk of under-reporting space for research activities due to the higher cost attributed to research space which has implications on portfolio financial performance within NRC's performance measurement framework.

Finance Branch began an initiative to develop building based BCA allocations where research facilities would be allocated a proportion of general building expenses based on their location. The cost allocation approach has the potential to penalize research facilities housed in older base building structures where operating costs and maintenance expenses may be higher due to base building age.

Line of Enquiry 3: Operations Adequate

Criterion 3.1: NRC is compliant with applicable Treasury Board, NRC and stakeholder policies, directives and related regulatory requirements relating to facilities management

NRC research facilities operate in a complex environment of Government of Canada and NRC policies and directives as well as regulatory requirements from relevant oversight bodies.

Applicable Treasury Board policies and directives reflect general good practices demonstrating stewardship and adequate oversight such as material management, financial stewardship, safety and health, and investment planning requirements. Internally, NRC has defined real property requirements and practices under its NRC Real Property Management Framework and NRC Real Property Policy Suite and has defined effective financial management practices through the NRC Financial Management Manual. The NRC investment planning program supports the planning and development of new and or replacement research facilities and the NRC Material Management Manual governs procurement and ongoing management of assets. In addressing health and safety, NRC has a comprehensive suite of policies, directives and standards governing health and safety which impact research facilities management. For example, the Directive on Biosafety requires that all NRC labs dealing with biological materials be operated according to the Human Pathogens and Toxins Act and the Health of Animals Act, as well as all applicable, related regulations, standards and guidelines. The NRC Directive on Laser Safety clearly delineates an operating standard for the use of lasers in the workplace. The Directive on Ionizing Radiation and Consolidated Nuclear Substances outlines the requirements defined by the Canadian Nuclear Safety Commission for working with and managing materials and devices that emit ionizing radiation. We also found the necessary licenses for managing dangerous and or controlled substances such as radioisotopes and related nuclear materials.

NRC also has a Policy for Research Involving Human Subjects, a Policy on Research Involving Animal Subjects and a NRC Animal Care Committee that provides oversight and defines specific requirements that impact the organization and operation of research facilities to ensure ethical practices in the delivery of research activities and services.

NRC is generally compliant with policies, directives and regulatory requirements associated with research facilities management. NRC has the framework and processes in place to obtain the permits, licenses and related documentation required to operate research facilities in compliance with regulatory and policy requirements spanning the breadth of NRC research activities.

Reflective of the diverse research activities undertaken across NRC, our sample of scoped-in portfolios included permits, licenses, and oversight requirements to address the handling of chemicals and radioactive materials, the caring and treatment of animals for scientific testing, and the handling of infectious and other biological materials. Sampled portfolios were found to have sufficient documentation in place to support compliance with applicable regulatory requirements including up to date permits and licences for bio-confinement as prescribed by the Canadian Biosafety Standards and Guidelines and nuclear safety, which is stipulated in NRC's Radiation Safety Manual.

We did not find a centralized authority or repository for permits, licenses or oversight requirements either at the corporate or portfolio levels. We noted that regulatory documentation is kept at the portfolio level, often with specific individuals. The lack of a centralized authority responsible for coordinating unique health, safety and regulatory requirements increases business continuity risk and timely access to relevant information should key information holders not be available.

An inventory of oversight requirements centrally held at the portfolio level may support streamlined management practices by ensuring that relevant and necessary regulatory and oversight documentation is accessible, that oversight requirements are documented, and that research facilities activities are coordinated and there are no lapses in adherence to laws, policies, directives and or other operating requirements.

Following the reorganization of NRC research functions, safety and health functions were impacted. Insufficient staff and the need to provide specialized training to address unique health and safety requirements have slowed the re-establishment of capable and effective emergency response functions. Consistent across in-scope portfolios, existing emergency plans and documentation were found to be outdated. We noted that NRC is in the process of implementing an emergency management program. An NRC wide emergency management plan was approved by the Senior Executive in October 2013 with training and local emergency response plan templates to be distributed by July 2014. Concurrently, research facilities Business Continuity Planning (BCP) requires further guidance and updating to ensure that NRC's research facilities continue to meet Treasury Board standards and that contingency plans are continuously updated to prevent adverse impacts to client service delivery.

Portfolios should consider ranking their major research facilities in terms of impact on programs and general operating condition for more effective risk management and proactive business continuity and contingency planning. As well, a portfolio level ranking would support investment or divestment decisions based on the strategic outlook of the portfolio. The ranking and related planning activities should feed into NRC's business continuity program and help portfolios risk manage their essential research infrastructure to meet internal and external client expectations.

See Recommendation 2.

Criterion 3.2: Facility conditions are regularly assessed and maintenance and repair expenses are analyzed on a regular basis

Timely and preventative maintenance are vital to ensure that research facilities are safe to use, properly maintained, and deliver on NRC's program activities. Access to repairs and maintenance information facilitates decision-making in terms of investment, reinvestment or divestment of research facilities and supports a comprehensive review of the efficiency of research facilities operations for effective planning and budgeting.

Portfolios have taken unique approaches to monitoring and managing research facilities maintenance. While portfolio internal processes are in their infancy, repairs and maintenance expenses have been tracked and logged to support trend analyses. Data collection processes are not integrated with NRC's project management system to enable streamlined resource management and capacity scaling.

We found that research facilities maintenance is a decentralized activity under the responsibility of portfolios. The maturity of internal processes and systems used to support maintenance and repair tracking and analysis vary across the organization. We found that portfolios have set-up internal orders to track maintenance and repair expenses but the level of detail of internal orders vary across portfolios. For example, one portfolio summarized repairs and maintenance at an operating location level, which would require further analysis to identify trends for specific facilities, while another portfolio took a granular approach by setting up tracking methods for individual research facilities and types of maintenance such as emergency, reactive and preventative maintenance. No corporate approach or standards to research facilities maintenance have been defined. While structures have been developed to support maintenance and repair expense analysis, no process has been defined to validate annual maintenance expenses and integrate analyses with budgeting assumptions and costing activities.

NRC's enterprise project management system requires research facilities maintenance and downtime to be planned as components of projects or as unique activities. The use of internal orders (outside of SAP PS module) precludes the ability for the system to scale capacity and provide an accurate assessment of resource utilization. We also observed ad-hoc or manual tools being used to plan and schedule operations and maintenance activities; unsustainable practices as portfolios increase efforts to share facilities and program activities ramp-up.

ASPM uses SAP Plant Maintenance (PM) to manage base building repairs and maintenance. The system directly links to NRC's financial management and enterprise project management systems. No portfolio currently uses SAP PM to manage repairs and maintenance of their research facilities. While one portfolio explored the option of implementing the system, NRC's transformation and other priority initiatives delayed the trial. We did not identify corporate guidance or standards for maintenance trend analyses. The lack of an integrated maintenance management and information system between ASPM and portfolios inhibits the ability to gather an NRC-wide perspective on recapitalization requirements that integrates base building and research facility needs.

NRC should consider reducing its reliance on manual tools by increasing adoption of existing automated planning and management systems to support research facilities maintenance including:

  • Targeted training for facility managers and staff to better leverage SAP PS capabilities;
  • Increased use of SAP PS to plan projects and scale capacity; and
  • Increased use of SAP PM for facilities planning, maintenance and performance analysis

Variations in research facilities maintenance practices

The diversity of research facility assets across NRC is reflected in the diverse approaches to facilities maintenance. We found select portfolios have extensive vendor support contracts while others have sourced local expertise or invested in extensive staff training to handle maintenance needs and the risk of equipment failure. We found that research facilities and equipment warranty information is managed in a decentralized fashion within portfolios increasing the risk of the inability to acquire timely vendor support should key information holders not be present or available.

An NRC-wide research facility and equipment database that consolidates warranties would help identify opportunities for economies of scale when entering into contracts covering multiple facilities or pieces of equipment to manage the risk of facilities and equipment failure.

See Recommendation 4.

Criterion 3.3: Surplus space capacity/assets are considered within the context of shortages in other parts of the organization or are disposed of in a timely manner

Sharing of resources in an environment of constrained funding demonstrates stewardship of public assets and ensures that NRC maximizes the value and benefits to Canadians from its research facilities holdings. The timely disposal of surplus assets ensures that NRC receives fair value or repurposes resources for priority initiatives in areas of need.

NRC has a defined asset disposal process where portfolios are responsible for asset management, and identifying surplus assets while ASPM is responsible for transferring surplus equipment to the Public Works and Government Services Canada (PWGSC) Crown Assets Distribution Directorate (CADD) for disposal. NRC retains proceeds from the disposal of assets centrally for reinvestment into program activities. We found a consistent understanding of NRC's research facilities disposal process despite the significant changes to internal business processes and practices. We noted that some portfolios have defined unique internal processes to dispose of surplus research facilities and equipment prior to engaging ASPM and CADD.

We noted through site visits that NRC has significant holdings of dormant assets, at times held in older structures with poor FCI ratings. The disposal of dormant assets releases capital and space for reallocation to program needs or for divestment via disposal or sale thereby reducing NRC's financial commitments to maintain storage space. We identified concerns with disposing of assets due to the cyclical nature of select industries. The poor state of repair of NRC's general storage structures, the pace of technological change, and a defined needs-based departmental investment planning program support timely disposal of surplus assets to ensure that NRC receives the maximum value for its surplus assets and that disposal proceeds can be reinvested into program activities.

The geographically dispersed nature of NRC and of individual portfolios complicates resource-sharing initiatives. Portfolios have developed unique responses to share resources internally and across NRC but an enterprise level initiative to optimize resource use is lacking. An NRC marketplace would facilitate sharing and optimization of unused or underused capacity, supporting allocation of funds to priority initiatives and maximizes existing resource capacity.

See Recommendation 4.

Beginning in Q2 of fiscal year 2015, the Vice-President of Corporate Management began an initiative to purge NRC's holdings of dormant assets, freeing up space and capital for reinvestment into NRC's programs.

There are opportunities to improve integrated planning by using existing resources such as the ASPM Surplus Disposal Unit to help assess opportunities to dispose of unneeded assets and provide estimates of recoveries for inclusion in investment plan submissions.

Individual portfolios are addressing surplus capacity through unique initiatives. At one geographically dispersed portfolio, management has set-up a research facility and equipment inventory and reservation system to share existing resources across the portfolio. The system is complemented by a project to define and delineate roles and responsibilities to reinforce research facilities accountabilities. We found another portfolio is exploring the deployment of an in-house booking and logging system to manage research facilities usage and to track sample processing across all operating locations until a permanent solution can be procured. In one instance, we noted portfolio plans to transport research samples across geographical locations to maximize existing facility capacity.

Across scoped-in portfolios, we found that informal systems and mechanisms are being employed to book research facilities in the form of manual sign-up sheets, paper logs and negotiation between facilities managers and project staff. The continued reliance on informal and manual systems is unsustainable. The expected increase in program activities and demand for research facilities coupled with existing resource sharing initiatives is expected to put pressure on research facility capacity and increase the risk of scheduling and workflow conflicts. Proactively addressing capacity scaling and workflow issues with electronic work tools and scheduling standards and guidelines ensures that NRC is prepared and able to increase utilization of existing resources and foster organization-wide sharing.

An organization-wide marketplace would support resource sharing and optimization of existing capacity. A corporate approach ensures that the system is integrated with existing corporate resources and initiatives, is developed to support cross-portfolio sharing, and is developed with consistent standards and guidelines.

Supporting resource efficiency and optimization, NRC should consider implementing tools and processes to automate facility reservations for proactive workflow management and resource sharing.

See Recommendation 4.

In one instance we identified an inter-portfolio initiative to document and analyze capacity of commonly required research equipment such as nuclear magnetic resonance apparatus, mass spectrometers and advanced microscopy instruments. The initiative was part of an analysis of duplicate investment plan submissions in fiscal year 2013 that increased utilization of existing capacity and avoided investments estimated at $2.3M. The initiative was an ad-hoc effort driven by portfolio resources. It is not sustainable to have portfolio resources address coordination and resource sharing without additional corporate support. The lack of visibility with regard to existing research facilities across NRC increases the complexity of program resource planning, diverts management attention from priority initiatives, restricts investment funding and results in duplicative investment submissions.

Academic institutions have set-up resource sharing databases in an environment of reduced government funding to maximize return on research assets. We noted success in the adoption of Commercial-Off-The-Shelf (COTS) software that provides a database functionality to share research facilities and equipment. Examples of COTS and internally developed software solutions were found in use at Canadian and United States based academic institutions and non-profit and private research entities providing detailed research facility information including but not limited to; internal and external charge rates; technical capabilities; operating hours; instructions to book equipment; and in some cases, utilization data.

A research facilities marketplace accessible across the organization would facilitate resource sharing and support the reduction of duplicative investment submissions for already existing research capabilities. The marketplace could be made available to other government departments to increase intergovernmental resource sharing and synergies.

See Recommendation 4.

Externally accessible research asset databases could provide benchmarks in developing cost-benefit analyses, costing benchmarks, and as options for alternative modes of service delivery. This would support investment plan business cases and recommendations.

Information Technology and Security Services (ITSS) is responsible for the management of NRC information technology resources including the procurement, coordination, and management of common corporate productivity software. Research portfolios are responsible for the management of research software. ITSS undertook a survey to explore the consolidation of commonly used research software to reduce costs and streamline management in January 2014. The survey noted resistance to a single software system due to backward compatibility issues, effort required to re-train staff, and the ability to work with both internal and external clients. Due to the geographical dispersion of NRC research activities, opportunities exist to streamline research software procurement and management with floating licenses allowing numerous, potentially cross-Canada installations, with limited active users. As noted at one portfolio operating in three geographical time zones, the ability to share software assets has the potential to realize significant software license related cost savings.

Software management is expected to be a key component of a rebuilt "Secure NRC" electronic working environment following the July 2014 cyber intrusion.

In the absence of defined corporate standards for categorizing, costing, and managing digital research facilities, portfolios should consider managing software based on workflow or business processes. Where research software is integral to a research activity and must be used with other software, facilities or equipment, it should be consolidated. Portfolios should consider categorizing research software that can be used independently as a unique facility and cost it accordingly.

NRC ITSS should consider exploring different licensing options for research software to rationalize assets and encourage sharing and collaboration based on NRC's unique operating characteristics. Opportunities also exist to use software monitoring or metering tools to track actual usage demonstrating the need for said software and to improve tracking for project costing purposes.

See Recommendation 4.

Criterion 3.4: Performance indicators guide decision making and facilities management activities

Research facilities performance indicators support empirically based decision-making including investment, reinvestment or divestment decisions. An effective research facilities performance management system ensures that desired behaviours are encouraged to achieve established management goals and objectives and that research facilities are used in a manner that maximizes value for Canadians.

In general, we found that a performance management system for research facilities has been defined but elements are incomplete to fully and effectively support empirically based decision-making.

Corporate and research performance metrics integration

NRC has defined metrics and performance data requirements at a corporate level to assess the efficiency with which it leverages research facility resources. Metrics are focused on facilities utilization and the degree to which costs are internally recovered from program related research activities. At the portfolio level, General Managers have defined internal performance metrics to assess operational activities and resource use in support of programs such as throughput, cycle time, availability, and downtime. We also noted the absence of independent validation of portfolio defined performance metrics.

Research facilities performance metrics are integrated elements of program and portfolio performance measurement systems. NRC's research program cycle has active programs assessed on a tri-annual cycle on predefined performance metrics, which include an assessment of the effectiveness of resource management. Program leaders are responsible for developing appropriate indicators for their programs, which may include resource use indicators such as research facilities utilization. The first program review is expected to be undertaken in the Winter 2015.

We noted that one portfolio has defined a metric of research facilities utilization relative to costs recovered to operate facilities. The metric relies on defined activity rates. We identified concerns with the robustness of defined facility activity rates due to the inconsistent application of costing principles and guidance which would directly impact the application of the defined metric (See Line of Enquiry 2).

At the corporate level, while NRC has defined research facilities performance expectations, understanding, acceptance, and implementation of activities to achieve corporately defined metrics is inconsistent. Portfolios have identified operational indicators reflective of their unique operating requirements. Better alignment between operational and corporate measures would improve organizational alignment and support an integrated facilities performance management framework.

As of August 2014, strategic and operational planning templates are being revised with additional guidance in areas pertaining to performance management to support better alignment and definition of performance indicators across NRC at Divisional and portfolio levels. NRC should consider reinforcing research facilities guidance and standards to ensure that facilities utilization can be analyzed and compared at the corporate level. Defining a program for research facilities utilization variance analysis would allow for a systematic approach to assessing performance and determining variance from budgets for timely remedial action where necessary. A defined standard for utilization variance analysis would reinforce costing practices and support continuous refinement of facilities activity rates.

See Recommendation 5.

Resource Efficiency

Utilization efficiency is tracked and measured through SAP PS, NRC's enterprise project management system. The system records personnel and research facilities time commitments to projects. While employees record time electronically via a self-service portal, research facility postings are a manual process using a spreadsheet tool. We noted that portfolios have taken unique approaches to roles and responsibilities for research facilities time management based on internal organizational structures and business practices. Adequately documented roles and responsibilities for facilities time management is vital to ensure that postings are reflective of time used, are attributed to the correct facility, and are reviewed and validated independently for proper segregation of duties. Adequate roles and responsibilities definition is also vital to ensure that facilities postings are properly reflected in portfolio internal financial performance assessments.

PRS generates monthly and quarterly utilization reports. We noted that reports were inconsistent and did not provide useful information for decision-making due to the lack of consistency in defining practical capacity and inconsistent understanding and use of NRC's project management system. We found that utilization reports and information were being generated using differing capacities between FB cost template defined values and those found in SAP PS. Consistently we noted that practical capacity in SAP PS was higher than defined in costing workbooks resulting in unrepresentative utilization figures. A preliminary analysis of research facilities utilization adjusted to include only time-denominated activity rates resulted in increases to facilities utilization of more than 90% for one division and exceeding 200% for another. SAP PS requires that all facility activities be planned in the system including routine maintenance and other planned downtime. The lack of comprehensive planning in SAP PS precludes the ability to leverage automatic capacity levelling and utilization reporting capabilities and skews utilization reporting. Concurrently, the lack of a consistent baseline definition of capacity precluded a full reconciliation of facilities utilization. Not all facilities practical capacity was defined in temporal units, some were organized on a sample or testing unit basis.

Effective reporting using SAP PS requires a systematic process of planning and timely recording of research facilities usage. The lack of guidance and consistent standards to support the definition of research facilities practical capacity directly impacts the activity rate charged to internal users, the denominator value used to calculate utilization and performance reporting to support decision-making.

NRC should consider leveraging the capabilities of SAP PS to better integrate systems with business processes. SAP allows for defined facilities activity rates on a time-measured basis as well as for fixed fee services. A comprehensive capacity assessment within SAP PS by planning all activities including project usage and planned maintenance downtime in the system could provide a more representative view of research facilities capacity for planning, usage, and reporting purposes. As well, it would reduce the level of effort required for reporting purposes, leverage complete data, and reallocate resources to more value-added activities.

Increased training for research facilities management staff in the use of SAP PS with a focus on planning would enable automated reporting capabilities.

NRC complements the facilities utilization performance indicator with a measure of internal direct to indirect costs. The ratio encourages investment of resources in program activities and maintaining overhead efficiency. NRC direct costs are expenses that can be directly traced to program activities including labour, materials, and capital expenditures. Indirect expenses include overhead to run operations and any costs not recoverable from program activities including any uncharged research facilities expenses from idle capacity, maintenance and repairs, and emergency downtime. Research facilities maintenance costs are only fully recovered where facilities are used at 100% of practical capacity; utilization at any level less than total practical capacity would result in residual facilities expenses (overhead expenses would increase). Continued reliance on manual budgeting and planning processes completed outside of SAP PS limits the ability to assess the ratio of direct to indirect expenses at the individual facility level.

Increasing efforts to share existing resources and divesting unneeded assets has benefits for NRC overall in terms of resource efficiency that could reflect positively on portfolio performance under NRC's performance management framework; divesting assets and using shared resources directly reduces indirect costs such as facilities maintenance expenses.

As stated earlier, the development of an enterprise marketplace for research facilities would enable the effective use of existing resources, better management of NRC's asset base and decrease the likelihood of resource duplication. The marketplace would support greater linkages between portfolios and common services and has performance assessment implications should portfolios decide to divest asset holdings.

See Recommendation 4.

Real Property Related Metrics

NRC undertook a comprehensive assessment exercise in fiscal year 2009 to assess the condition of its real property (base building) holdings to generate a Facility Condition Index (FCI) score for each building. FCI provides a financial assessment of deferred maintenance required to bring a building to a state of good repair. Within NRC's Real Property Management Framework, ASPM noted efforts to gather information to develop a utilization standard for laboratory space based on research activities being undertaken. NRC has defined an office space utilization target of 17 square meters per person and is working towards defining a utilization standard for laboratory space based on research activities. A similar exercise has not been undertaken in relation to research facilities.

NRC's larger, stand-alone research facilities, such as wind tunnels, wave tanks, etc, may benefit from comprehensive condition assessments to avoid piecemeal investment and or reinvestment requests and to demonstrate long-term planning and consideration of NRC's resources.

At the beginning of fiscal year 2015, ASPM began a project to inventory space across NRC. The definition of a laboratory space utilization target is in progress.

Investment planning indicators

Effective and timely expenditure of capital funding is vital to ensure that NRC can support program investment and renewal needs, demonstrates financial stewardship, and capitalizes on the seasonal construction cycle. Program capital asset requirements are addressed via NRC's investment planning process for major capital needs above $250K or through a minor capital management process at the Division Head level for expenditures below $250K. We found that in fiscal year 2013-14, NRC spent $32.4M through the departmental investment planning process representing 55% of its major capital budget and $27.9M through Divisional capital allocations representing 91% of its minor capital budget.

A review of portfolio plans identified varying levels of targeted investment and reinvestment in research facilities as well as inconsistent financial planning for research facilities related support from ASPM. The majority of scoped-in portfolios had defined an investment target of 2% of their historical minor capital (under $250K) research facilities asset base. Major capital is prioritized from a corporate perspective based on program needs. Two portfolios were found to have performed an analysis of their research asset base identifying nearly 40% of their assets as fully depreciated or more than 10 years old. The portfolios also noted that they operated in a capital-intensive industry where continual investment is vital to maintain relevance.

We noted that Parks Canada has defined investment targets of 1% annually of the current replacement value of their asset base for operating expenditures, 2% for maintenance and repairs to sustain their asset base and 2% for capital expenditures to improve functionality and or capacity of their assets which includes purchasing new or replacing assets and addressing its renewal backlog. Segregating capital investment targets between growth (new asset investment) and maintenance (investment to maintain existing asset base) would provide a benchmark from which NRC could assess organizational growth relative to program activities and demonstrate benefits to Canadians. The segregation of capital investment also provides a benchmark for portfolios to assess investment relative to their operating industries to maintain relevance.

NRC's performance management framework for research facilities should be clearly defined to reinforce and re-communicate research facilities performance related behaviours and expectations.

NRC has defined a structured and integrated approach to investment in capital assets aligned with program needs and NRC's budgeting cycle. Opportunities for improvement exist around:

  • Integrating portfolio level capital investment targets within its performance measurement framework to ensure that sufficient investment and reinvestment is made to research infrastructure to maintain relevance in rapidly evolving industries; and
  • Defining a performance indicator for the level of legacy assets currently in use and a target to either renew or divest legacy assets to ensure that NRC invests sufficiently in renewing its asset base to maintain technological relevance

An up-to-date research asset base ensures that NRC delivers services to Canadians with state-of-the-art facilities and equipment.

See Recommendation 5.

Research Asset Base

NRC's capital asset base includes a significant proportion of assets that have reached or exceeded their accounting lives. Within our sample of four portfolios, the ratio of total book value of assets relative to the historical cost of all assets in service spanned between 18% and 27%. While accounting life is not a precise measurement of the economic benefits extractable from a capital asset, the continued use of legacy assets increases the risk of escalating maintenance and repair expenses to support older, less efficient and or less effective instruments. We found that in some instances, research facilities or their components, such as controlling software or mechanical components, lack vendor support and do not have an active market for replacement parts. In one instance we found that a facility was kept operational through the cannibalization of parts from retired equipment. An older scientific asset base increases the likelihood of breakdowns increasing the risk of being unable to deliver on program and client requirements and thus, benefits to Canadians. Legacy assets do not effectively support NRC's mission to develop and deploy solutions to meet Canada's current and future industrial and societal needs.

Asset Related Performance Measures

We found that portfolio operational plans consistently highlighted concerns with the age of current research facilities and both their utility and sustainability in relation to NRC's objective to deliver leading edge research and development services. We noted that portfolio level performance indicators were developed by portfolios in agreement with respective Division Heads and are at times driven by the performance indicators defined by programs. No consistent performance indicators were defined at the portfolio or corporate levels, to address the age of research facilities although we found numerous strategic and or operational plans that highlighted and provided analysis of required renewal targets to address the issue.

We noted that Atomic Energy Canada Limited (AECL), in its annual report identifies a historical cost of $400M in fully depreciated assets that are still in use to illustrate its backlog in recapitalization. NRC may benefit from a similar disclosure on an annual basis at the research portfolio level to provide executive level visibility to expected resource requirements to demonstrate stewardship and sustainability.

Appendix A: Audit Criteria

Line of Enquiry Intended Audit Outcomes / Value Added Audit Criteria

1.0 Strategic and sustainable planning

1.1 Achieving strategic outcomes by:

  1. Effective planning of limited resources
  2. Aligning processes and activities
  3. Having the right people and skills

Strategic and operational planning guide facilities management activities and demonstrate the whole-of-life concept for effective stewardship of assets

1.2 Ensuring sustainability of infrastructure / business operations

Plans are in place to support current and future facilities management human resource needs

2.0 Costing

2.1 Effective management of resources

Facilities management includes clear consideration of cost information with a view of achieving organizational objectives

2.2 Recovery of NRC investment

3.0 Operations

3.1 Compliance with policies, directives, regulatory requirements

NRC is compliant with applicable Treasury Board, NRC and stakeholder policies, directives and related regulatory requirements relating to facilities management

3.2 Allocating / re-allocating limited resources in an effective manner to deliver greatest impact and value

Facility conditions are regularly assessed and maintenance repair expenses are analyzed on a regular basis

3.3 Defined metrics to drive planning and targeted performance

Surplus space capacity/assets are considered within the context of shortages in other parts of the organization or are disposed of in a timely manner

3.4 Empirically supported decision making

Performance indicators guide decision-making and facilities management activities

Appendix B: Potential Overall Ratings

Management Attention Required

There are significant weaknesses in the design and/or effectiveness of the selected key management controls that require management's attention. Critical practices / processes do not meet the expectations and or key principles described in Government of Canada and NRC regulations, policies and directives. There are significant opportunities for development.

Needs Improvement

The design and/or effectiveness of the selected key management controls needs improvement. Some areas of practice / processes meet the expectations and or key principles described in Government of Canada and NRC regulations, policies and directives. There are several opportunities for improvement.

Adequate

The design and/or effectiveness of the selected key management controls needs improvement. Some areas of practice / processes meet the expectations and or key principles described in Government of Canada and NRC regulations, policies and directives. There are several opportunities for improvement. 

Strong

The design and/or effectiveness of the selected key management controls is strong. All areas of practice / processes meet the expectations and or key principles described in Government of Canada and NRC regulations, policies and directives. No areas for improvement were identified.

Appendix C: Management Action Plan

Definition of Priority of Recommendations

High
Implementation is recommended within six months to reduce the risk of potential high likelihood and/or high impact events that may adversely affect the integrity of NRC's governance, risk management and control processes.
Moderate
Implementation is recommended within one year to reduce the risk of potential events that may adversely affect the integrity of NRC's governance, risk management and control processes.
Low
Implementation is recommended within one year to adopt best practices and/or strengthen the integrity of NRC's governance, risk management and control processes.
Recommendation Corrective Management Action Plan Expected Implementation Date and Responsible NRC Contact

Recommendation 1: NRC Senior Executive Committee (SEC) should appoint a champion to spearhead key initiatives aimed at implementing a research facilities management framework to better support the achievement of strategic objectives and delivery of research programs.
[Priority: HIGH]

Management agrees with the recommendation. NRC Vice-President, Emerging Technologies is appointed as NRC SEC champion to provide strategic direction for the successful implementation of an improved research facilities management framework. Within the current transformation environment, this role will ensure effective linkages with key corporate functions and research portfolio general managers.

Date: Completed

Contact: Senior Executive Committee

Recommendation 2: The NRC SEC Champion for research facilities management, with the support of NRC Planning and Reporting Services (PRS), should provide structure to research facilities planning practices through the development and consistent application of guidelines, templates and tools, as well as the provision of additional training on the use of new and existing tools. [Priority: HIGH]

Management agrees with the recommendation.

Date: June 30, 2015

Contact: Vice-President, Emerging Technologies

Recommendation 3: The NRC SEC Champion for research facilities management, with the support of NRC Finance Branch (FB), should ensure consistent application of costing methodologies and tools to provide reliable information for decision-making. [Priority: HIGH]

Management agrees with the recommendation.

Date: June 30, 2015

Contact: Vice-President, Emerging Technologies

Recommendation 4: The NRC SEC Champion for research facilities management, with the support of NRC Knowledge Management (KM), should ensure that NRC develops a research facilities marketplace for more effective and efficient resource planning and utilization. [Priority: MODERATE]

Management agrees with the recommendation.

Date: June 30, 2015

Contact: Vice-President, Emerging Technologies

Recommendation 5: The NRC SEC Champion for research facilities management, with the support of NRC Planning and Reporting Services (PRS), should align research facilities performance management across corporate, divisional and portfolio levels to ensure that operational plans enact management strategies and encourage desired research facilities management behaviour. [Priority: HIGH]

Management agrees with the recommendation.

Date: June 30, 2015

Contact: Vice-President, Emerging Technologies

Date modified: