ARCHIVED - Audit of Investment Planning

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

ARCHIVED - Audit of Investment Planning (PDF, 516 KB)

1.0 Executive Summary and Conclusion

Audit objective

The objective of this audit is to provide NRC senior management with assurance that an adequate management control framework is in place to effectively support investment planning in a manner that is compliant with applicable Government of Canada and NRC policies, procedures and regulations.

Raison d'être

The Treasury Board Secretariat (TBS) Policy on Investment Planning – Assets and Acquired Services (2009) stipulates that resources should be allocated and reallocated to new and existing assets and acquired services essential to program delivery in a manner that ensures sound stewardship and achievement of value for money. Investment planning has major strategic, policy, operational, risk and financial implications for a department and serves as the basis for which investments of the highest priority and risk to the government are managed. Effective investment planning should ensure a diligent and rational manner of resource allocation for both existing and new assets, and for acquired services, within departmental reference levels.

Audit Opinion and Conclusion

The Office of Audit and Evaluation (OAE) found that NRC has established an adequate management control framework for investment planning in compliance with Government of Canada policies and directives. Further, we found that NRC is continually reviewing the investment planning framework and related activities with a view to achieving a steady state which addresses opportunities for streamlining and integration as identified by management. Particular strengths in the framework were identified in respect of the governance structure for investment planning and the accessibility of information and related systems in support of informed decision making. The audit highlights some opportunities of moderate priority for strengthening the investment planning framework.

The table below summarizes the assessment of the investment planning framework relative to each in-scope area. Following this table is a list of related recommendations.

Lines of Enquiry
Overall audit conclusion Adequate
Governance Structure Strong
Roles and Responsibilities Adequate
Investment Planning Processes Adequate
Training, Capacity and Competencies Adequate
Systems and Information for Decision Making Strong
Monitoring Adequate

Summary of Recommendations

  1. Program and Project Services (PPS) should continue working toward integrating investment planning processes, program planning activities and NRC’s budget cycle and continue to support initiatives to leverage tools and capabilities that will maximize the use of existing resources prior to consideration of new and/or additional investments. [Priority: MODERATE]
  2. PPS should consider establishing a timeline for developing and implementing training and competency targets for Project Managers in recognition of the multifaceted nature of investment planning activities. [Priority: MODERATE]
  3. Integral to its mandate to manage and oversee the departmental investment plan, PPS should consider implementing a comprehensive monitoring framework to measure overall performance of the investment plan and NRC’s performance as a department in the allocation and reallocation of resources. The framework should incorporate all elements of investment planning managed at the PPS and Division Head levels, including acquired services, and should include quantitative performance metrics to ensure continuous improvement. [Priority: MODERATE]

Statement of Conformance

In my professional judgment as the Chief Audit Executive, the audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Irina Nikolova, FCCA, CIA, CISA, Acting Chief Audit Executive

NRC Audit Team Member:
Andy Lang, CIA, Auditor

2.0 Introduction

This audit report presents the findings of the National Research Council (NRC) Canada’s Audit of Investment Planning.

The audit was undertaken by the Office of Audit and Evaluation in accordance with NRC’s 2012-2013 update of the NRC 2011-12 to 2013-14 Risk-Based Internal Audit Plan presented at the June 2012 Departmental Audit Committee meeting.

2.1 Background

Treasury Board defines departmental investment planning as the function of allocating and reallocating resources to new and existing assets and acquired services that are essential to program delivery. Effective investment planning activities and oversight should ensure a diligent and rational manner of resource allocation for existing and new assets, and for acquired services, within departmental reference levels. As a high impact horizontal activity, investment planning is a major enabler in NRC’s transformation to a Research Technology Organization.

Investment planning is primarily governed by the Treasury Board Policy on Investment Planning – Assets and Acquired Services and the Policy on the Management of Projects, both of which became effective in December 2009. Additional Treasury Board requirements include the Standard for Organizational Project Management Capacity (OPMCA); and the Standard for Project Complexity and Risk (PCRA). NRC completed and submitted its OPMCA along with its first Investment Plan to Treasury Board in 2011, resulting in a Level 2 (“Tactical” Footnote1) rating. This rating provides NRC with the authority to undertake individual investment projects, with a Project Complexity and Risk Assessment rating of up to 2, without further TBS approval. According to the submitted Investment Plan, a total of $815M in investments is planned over the 5 year period (2011-12 through 2015-16 inclusive) on major and minor capital investment projectsFootnote2, acquired services in support of assets and operations, and on contaminated sites.

Chart 1 – Five Year NRC Investment Plan Allocation

Investment in assets over $250K
Chart 1 – Five Year NRC Investment Plan Allocation

The NRC investment planning governance structure has adapted to NRC’s transformation to a program-based management model, maintaining the structures that were created at the inception of investment planning activities at NRC. The Senior Executive Committee approves the prioritized list of projects that form the NRC Investment plan, providing funding and resources and overseeing the investment plan. The Investment Management Committee manages investment planning, reviewing, prioritizing and recommending a list of NRC-wide, cross-asset projects for approval by the Senior Executive. The Asset Management Boards (Scientific equipment and laboratories; Real property and vehicles; Information Management/Information Technology) review, rank and endorse projects within their asset categories. Division Heads are responsible for submitting a prioritized list of cross-portfolio projects to the respective asset management boards and for ongoing management of projects and acquired services. Program and Project Services (PPS) provides NRC-wide coordination for investment planning, project support services and fulfills the secretariat role within the investment planning governance framework.

NRC Investment Planning Governance Structure

NRC has defined investment projects within the context of supporting key existing research infrastructure, investing in new technology to support NRC’s new programs and supporting lifecycle management of real property. Criteria have been developed to define projects from a NRC perspective within the three categories of scientific equipment, machinery and laboratories, real property, facilities and vehicles and information management and technology. Corporate or strategic projects are classified as strategic elements or enablers. Investments outside the defined categories may still be classified as investment projects where they align with the definition of a project in the NRC Project Management Framework. Projects within NRC are defined as any effort to develop or acquire new, or to replace, refresh, upgrade or otherwise significantly improve existing, capabilities. Program research activities, regular and routine operations and maintenance costs are excluded from the scope oof investment plan projects.

Following the introduction of the new Treasury Board requirements for Investment Planning, NRC designed and implemented a complementary framework comprised of NRC-specific policies, tools, templates and guides. This framework includes investment planning-specific elements, and leverages relevant pre-existing guidance related to investment planning including the NRC Financial Management Manual, the NRC Project Approval Authority Directive, the NRC Project Management Framework, and the NRC Guide to Costing of Investment Projects. PPS has primary responsibility for ensuring NRC’s approach to investment planning reflects compliance with policy requirements while serving to allocate resources in a manner that reflects NRC’s particular environment and unique objectives.

2.2 Context

In planning and conducting this audit, OAE considered the context of investment planning in terms of NRC’s operating environment and departmental objectives. Successful implementation of investment planning activities and oversight requires effective change management. Further, a new investment planning framework is one of a number of change initiatives that are impacting the activities and accountabilities of managers and executives across NRC. The audit opinions, conclusion and recommendations included herein have been developed having taken into account the implications of this context.

Since the introduction of the Treasury Board requirements for investment planning in 2009-10, NRC has been in the process of implementing a sustainable investment planning framework. Given the nature and breadth of requirements for investment planning, and including the expectation for consistent and risk-based practices across the organization, implementing this framework has comprised a variety of individually significant elements including those within the scope of this audit. Managing stakeholders, communicating, providing training, and other tools/tactics are critical to the success of this cultural and procedural shift. Moreover, while compliance with Treasury Board requirements is an immediate priority, the achievement of an effective and efficient investment planning framework is more of an evolution than an end point.

Further, it is important that implementation of the investment planning framework has not occurred in isolation. There have been a number of related, and significant, change initiatives associated with NRC’s business model which have direct implications on investment planning including but not limited to changes in roles and responsibilities, processes, use of financial tools/systems and monitoring and oversight activities.

2.3 About the Audit

Objective

The objective of this audit was to provide NRC senior management with assurance that an adequate management control framework is in place to effectively support investment planning in a manner that is compliant with applicable Government of Canada and NRC policies, procedures and regulations.

Overall budgeting and financial forecasting were not included in the scope of the audit. Costing practices were reviewed to the extent that they impact investment plan projects.

Scope

Based on the risk assessment conducted during the planning stage, the scope of the audit comprised the following elements of the NRC control environment around investment planning:

  • Governance model (structure and roles/responsibilities);
  • Investment planning process and tools (including project complexity and risk assessments);
  • Training, capacity and competencies;
  • Systems and information for decision making; and
  • Monitoring.

The audit was carried out within the National Capital Region with a national scope. The application of the framework was examined within a sample of eleven (11) investment projects within the National Capital Region as well as in regional offices. The examination period was from July to November 2012.

Approach and methodology

The audit was conducted in accordance with generally accepted professional auditing standards and the standards and requirements set out in the Treasury Board Policy on Internal Audit. The audit criteria (as detailed in Appendix A) were primarily derived from the Treasury Board Policy on Investment Planning – Assets and Acquired Servicesand Policy on Management of Projects, and the Office of the Comptroller General’s Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011). The Treasury Board Standard for Organizational Project Management Capacity Assessmentsand Standard for Project Complexity and Risk Assessments were also referenced. Consideration was also given to relevant policies, standards, and relevant directives within the Treasury Board Policy Suite related to investment planning, management of assets, material and projects and planning. NRC frameworks and directives considered included the NRC Project Management Frameworkand the NRC Project Approval Authority Directive.

The audit procedures addressed the audit objective as conditions existed at the time of examination and any future plans as applicable. The audit methodologies were tailored to ensure that the root cause of findings was identified to support the development of value-added recommendations. The audit approach included:

  • Conducting interviews with Division Heads, relevant Corporate Directors General, managers and staff involved in investment planning and the investment control framework at the Corporate and Division Head levels;
  • Reviewing relevant NRC investment planning documentation such as framework documents, policies, guidelines, manuals, performance data, periodic reports and submissions to NRC governance committees and Treasury Board;
  • Reviewing the minutes and records of decision of NRC governance committees;
  • Conducting a literature review to obtain leading practices, potential benchmarks and governance models as they pertain to investment planning;
  • Conducting data mining and analysis;
  • Identifying and reviewing key information systems in place in support of investment planning and investment functions; and
  • Reviewing supporting documentation for a sample of investment projects selected.

Follow-up interviews were conducted to ensure that the auditors had complete information upon which to base their opinion.

3.0 Audit Findings

The following table presents the detailed audit findings with accompanying assessments of risks. Suggested management priorities for addressing risks are identified as high or moderate.

Audit Criteria Audit Findings Assessment

1. Governance Structure: An effective governance structure and related governance mechanisms are in place to support investment planning.

Strong

NRC has put in place an investment planning framework that is supported by sound governance structures and mechanisms for investment planning that reflect Government of Canada requirements and adequately support the investment planning process within NRC. The agendas and minutes of NRC's investment planning governance bodies demonstrate NRC management's involvement in the oversight of the investment planning process in support of the development and implementation of a departmental investment plan. NRC has provided complementary support and direction through policies, directives and guidance which provide a departmental context to Treasury Board requirements. The governance structure is subject to regular review as evidenced by the adjustments made to the asset management boards in response to changes in the operating environment and revisions to the Terms of Reference for the Senior Executive Committee to reflect their investment planning responsibilities.

No opportunities for improvement were identified.

1.1 Roles & Responsibilities: Roles and responsibilities, including those of external parties, are clearly defined and effectively communicated, understood and acknowledged.

Adequate

The roles and responsibilities of committees and boards involved in the investment planning framework are clearly defined, communicated and acknowledged in accordance with their respective mandates, goals and priorities.

Program and Project Services supports the horizontal nature of investment planning in their role as the central coordination body for all investment plan activities above $250K in value. PPS facilitates communication and relationship building between project managers and relevant stakeholders within the organization for procurement, property and asset management, financial advice, project management support and information technology (IT)-related support and guidance. Finance Branch supports project managers in the financial planning and management of projects. Relations with external organizations are managed by NRC’s internal common services; Administrative Services and Property Management is responsible for procurement for NRC and has a defined process when interacting with Public Works and Government Services Canada for procurement of goods and services above NRC’s authorities. With respect to information technology related projects, roles and responsibilities between NRC and Shared Services Canada are currently being defined and developed.

NRC’s reorganization and concurrent changes to the financial and operating model have created challenges to the continued clarity of investment planning roles and responsibilities, particularly as they relate to monitoring the financial progress and performance of the investment plan and individual investment projects. The lack of defined roles and responsibilities for monitoring of financial progress and performance of the investment plan and individual projects has the potential for duplicated efforts, inconsistent reporting to support decision making and an inability to take corrective action in a timely and effective manner.

See Recommendation 3

An opportunity exists to clarify and confirm the roles and responsibilities for financial monitoring of the investment plan between PPS, Finance Branch and portfolio Comptrollers to ensure that funds are effectively and efficiently allocated and that the financial progress of projects is monitored and reported to management in a timely and consistent manner.

As this opportunity is primarily related to monitoring activities, the related audit recommendation is included in Section 5.1(b) of this Table.

2. Investment planning processes: There are clearly communicated policies, directives, guidelines and standards which define investment planning processes across NRC.

Adequate

2.1 Policies, directives, guidelines and standards on investment planning are identified, available and accessible to employees for reference and use in the course of their investment planning activities. The PPS intranet page provides direct links to Treasury Board investment planning and project management materials as well as related internal guidance such as the NRC Project Management Framework, NRC Project Approval Authority Directive, NRC Project Complexity and Risk Assessment (PCRA) Guide, Guide to Costing of Investment Plan Projects and related tools and templates. Finance Branch provides the Cost Accounting Guide to support consistent costing practices across NRC.

No opportunities for improvement were identified.

2.1a/b/c/d NRC’s investment planning processes adequately reflect the requirements of the Treasury Board Policy on Investment Planning – Assets and Acquired Services with regard to a representative and value-added departmental investment plan. Senior management has mandated the adoption of SAP Project System as the departmental project management information system in support of standardized project management methodology and processes and to centrally plan, budget and account for resource allocations.

Following significant changes to NRC’s operating and financial models, PPS has been continuously identifying opportunities to improve processes and tools while maintaining their alignment with departmental investment planning objectives. PPS has made incremental changes to tools and templates as opportunities for improvement have been identified; the most recent changes made in Fall 2012 incorporated more detailed instructions with examples to help project managers complete templates with greater efficiency and consistency. Asset lifecycle related information, linkages to NRC’s programs, cost and revenue forecasts and the requirement for financial metrics, such as return on investment and payback period, were also incorporated into template updates. Templates are comprehensive and support project managers through the phases of the NRC project lifecycle.

Investment planning processes and activities have been communicated both formally and informally through management communiqués, the PPS intranet and departmental newsletters.

In January 2012, an independent consultant was contracted to undertake an investment planning and project management streamlining exercise to support continuous improvement of processes and practices. PPS is working towards implementing the streamlining exercise recommendations concurrent with the renewal of the departmental investment plan and to align practices with NRC’s program-based management model and financial model.

2.2 Investment planning processes were generally found to support appropriate financial stewardship and risk management for consistent decision making.

See 2.2 (a, b and d) for identified opportunities for improvement.

2.2a Investment planning processes are generally linked, in an appropriate manner, to related practices in strategic, business and financial planning and program and project management. Oversight of the linkages is supported by the Asset Management Boards and the Investment Management Committee whom are tasked with ensuring that project submissions are aligned with NRC’s Strategy by reviewing all projects in the investment plan, at minimum, on an annual basis. Moreover, processes for corrective action have been defined where non-aligned projects are identified or where new opportunities and requirements emerge requiring reprioritization of investment projects.

NRC’s shift to a program-based model occurred following the development of investment planning processes resulting in program and investment planning processes being less than fully aligned. Misalignment of the two processes may result in timing differences between approved programs and investment projects contributing to delays in program activities and potential financial and reputational repercussions where programs are collaborative in nature. Furthermore, program approval does not automatically delegate authority to undertake investment plan projects and delays can be compounded by procurement considerations such as lead time and Contracting Policy requirements. Maximizing the use and value of NRC’s investment funding requires that investment planning be coordinated with budgeting activities to ensure proper allocation and reallocation of resources as operating conditions change. PPS is working towards integrating and aligning program and investment planning processes as part of the investment planning streamlining exercise and renewal of the NRC Investment Plan.

Further, the investment planning framework is not supported by a widely accessible database of scientific infrastructure and/or equipment to provide visibility to existing and/or planned resources and investments that would encourage horizontal information sharing and collaboration and could reduce efforts required to develop business cases and plans for already existing or underutilized capabilities. The Scientific Equipment and Labs Asset Management Board is taking steps to address this opportunity and have identified existing capabilities, including functionalities within SAP that can be better leveraged to communicate available equipment and resources across NRC.

See Recommendation 1

An opportunity exists to strengthen the alignment of investment planning activities with program planning and budgeting processes to ensure that NRC resources are efficiently and effectively deployed and redeployed in support of NRC’s Strategy. Greater synergies may be realized through strengthened alignment of investment planning tools and templates with NRC’s program management model as well as consolidated approvals of programs and related investments. Furthermore, integrating investment planning activities with the overall NRC planning cycle would facilitate the development of a more representative and coordinated financial roadmap to drive decision making.

2.2b/c While costing and budgeting processes generally reflect NRC and Government of Canada requirements, file reviews demonstrated inconsistent methodology in the development of project documentation. Business cases did not consistently assess proposed solutions against the baseline state. Moreover, it was difficult to substantiate that business case and project planning documentation incorporated all relevant NRC policies and practices on budgeting and costing as PPS project planning templates do not require project managers to identify the source information relied on to develop estimates and assumptions.

Two separate documents, the Cost Accounting Guide and the Guide to Costing of Investment Plan Projects, have been developed by Finance Branch and PPS which establish NRC costing standards and practices. The Guide to Costing Investment Plan Projects focuses directly on investment planning related costing practices, in greater depth and detail than the Finance Branch Cost Accounting Guide. The Cost Accounting Guide provides a holistic view of project costing from a program-model perspective. There are opportunities to better align the two documents for greater consistency and to realize potential synergies between PPS and Finance Branch in supporting NRC’s program and project management communities.

PPS tools and templates require the identification of project risks and their incorporation in project planning documentation with appropriate mitigation plans. The tools and templates incorporate elements of NRC’s risk management methodology; PPS should consider updating the tools and templates when NRC’s updated risk management framework is approved by management. The majority of projects reviewed budgeted a contingency that was in line with costing guidance.

In the development of project management plans, project managers were inconsistent in their risk management practices with risk registers focusing on procurement related risks and less emphasis on risks related to adequate staffing and resources to execute the project, integration of new and old systems, training and issues related to set-up and start-up and interruption to operations among others. Inadequate and inconsistent risk management practices may adversely impact the execution of investment projects resulting in delays and cost overruns. As well, proper risk management may enable project managers to recognize and act on positive risks.

See Recommendation 2

The opportunities identified are primarily related to training and not process weaknesses with resulting recommendations included in Section 3.1 of this Table.

Areas for improvement were identified with regard to:

  • Consistent comparison of proposed solutions with the baseline state in business cases to demonstrate the value of the project as well as implications of not undertaking any action;
  • Identification of information sources used to generate key underlying assumptions and estimates in the project business case including assumptions for ongoing maintenance and lifecycle related costs to address Treasury Board’s whole-of-life cost concept; and
  • Consistent risk management practices and methodology to guide decision making, support prioritization of actions, improve understanding of threats and opportunities and effectively allocate resources.

2.2d Formal and informal continuous processes are in place to challenge assumptions and variances identified in the investment planning process at different levels in the governance structure for investment planning. Oversight structures and mechanisms are in place to challenge the basic, underlying assumptions of project submissions, plans, budgets and estimates.

While the role and responsibility of Finance Branch Comptrollers for monitoring the financial progress of capital investments within their respective portfolios is understood, no process has been defined to communicate and resolve identified issues and variances with related stakeholders including PPS and portfolio management. The lack of a defined communication process between FB Comptrollers and key stakeholders increases the risk of duplicated efforts by different monitoring bodies, inconsistent information for decision making and unsatisfactory resolution of identified issues.

See Recommendation 3

This issue relates primarily to monitoring and is addressed by the audit recommendation included in Section 5.1 of this Table.

2.2e Projects are being adequately assessed for risk and complexity according to their scope and objectives through the use of the Treasury Board Project Complexity and Risk Assessment Tool. PPS has also developed a NRC Project Complexity and Risk Assessment Guide tailored to support NRC project managers in the use of the tool.

No opportunities for improvement were identified.

Recommendation 1: PPS should continue working toward integrating investment planning processes, program planning activities and NRC’s planning cycle and continue to support initiatives to leverage tools and capabilities that will maximize the use of existing resources prior to consideration of new and/or additional investments. [Priority: Moderate]

3. Training, capacities and competencies: Formal training is provided to appropriate employees in support of developing relevant investment planning competencies.

Adequate

3.1 Formal training is provided to employees involved in the investment planning process and related activities to support the development of investment planning competencies. At the inception of the investment planning framework, PPS provided comprehensive training including instruction in the use of NRC’s approved tools and templates and project management methodology. PPS has also provided training to project managers with focus on project planning and project management within SAP Project System. PPS has noted its intention to develop competency profiles for project managers to foster consistency in knowledge and practices within the community.

See Recommendation 2

With PPS plans to develop training and competency targets for investment planning, there is an opportunity to establish a timeframe to address investment planning competencies via project manager competency profiles. Competency profiles ensure consistency in development and knowledge as well as facilitating easier identification of training requirements and succession planning.

Furthermore, there are opportunities to focus training by:

  • Providing targeted training in costing and financial management competencies and performance management (e.g. focused on quantitative performance metrics, such as return on investment) and development of value statements to demonstrate financial stewardship and performance measurement.
  • As noted in 2.2(b), while there is evidence of improvement in more recent investment project files, there were still documentation weaknesses that can be addressed through improved training and monitoring.

3.1a/b Training and materials related to the Project Management Framework and methodology are generally adequate to support employees in delivering on their investment planning responsibilities. Given the changes to the operating environment surrounding investment planning, training on the use of investment planning templates and tools were targeted at project managers with approved projects only. General training sessions were discontinued until investment planning processes and activities are streamlined and integrated with NRC planning and program activities as part of the renewal of the departmental investment plan.

Systems related training has focused on the use of SAP Project System. Project managers and employees have been provided general training on financial management principles and concepts as part of SAP Project System training to create, manage, report on and monitor projects. Finance officers and individuals assigned to the Finance Branch Costing Group have received targeted training for Project Accounting within Project System.

3.1c PPS facilitates ongoing development and continuous improvement of investment planning skills and knowledge with standardized templates and a project close-out process that requires project managers to document their lessons learned and best practices to be shared with the general investment planning and project management community. Microsoft SharePoint is leveraged as a collaborative resource, serving as a forum for sharing information between PPS and project managers. Project managers noted inconsistent feedback from PPS and management in relation to submitted investment plan submissions. The lack of feedback precludes project managers from tailoring their level of effort to investment planning activities and developing a general understanding of requirements and expectations from PPS and the investment planning framework.

A Project Management User Community was recently created in an effort to facilitate knowledge sharing, development of organizational expertise as well as bi-directional feedback between project managers and the NRC project management authority.

No opportunities for improvement were identified

3.2 NRC leverages SAP Project System to support managers in the identification of resources, their availability, commitments and utilization levels reflecting NRC's new operating and accounting models. Training activities are on-going to develop the necessary competencies, knowledge and skills set across NRC in basic project management and use of SAP Project System.

No opportunities for improvement were identified.

Recommendation 2: PPS should consider establishing a timeline for developing and implementing training and competency targets for Project Managers. These targets should address specific areas including costing, development of assumptions, project management, performance management (including use of key performance measures such as ROI), financial management concepts, budgeting, and documentation of investment project files. [Priority: Moderate]

4. Systems and information for decision making: Information and systems are available to support decision making across NRC.

Strong

4.1 Information is readily available and sufficient for decision making purposes for management at varying levels within the organization. NRC’s adoption of SAP Project System, implementation of business intelligence software and internal initiatives to develop workflow procedures to manage investment planning documentation facilitates information access and dissemination to support rational resource allocation and oversight of investment planning activities.

No opportunities for improvement were identified.

4.1 a/b/c/d Information provided to decision makers for investment planning purposes is sufficient, timely and reliable. Project managers are also required to provide semi-annual status reports to PPS to communicate changes, emerging risks and general progress in a structured manner. PPS also provides financial progress reports detailing spending against the budgeted investment plan for major capital spending (projects above $250K in value only) to the Senior Executive Committee. Finance Branch introduced a new financial monitoring process for branch and portfolio comptrollers to provide monthly financial progress reports, which integrate investment plan projects, to General Managers and Directors General to support management monitoring and oversight. NRC has recently adopted quarterly performance reviews that incorporate analysis of investment plan progress to keep management informed and reactive to changing conditions.

SAP Project System, Business Explorer and related business intelligence software support capturing of, analysis and reporting on relevant information for decision making purposes related to investment planning activities. The enterprise wide adoption of SAP Project System has facilitated collaboration and flexibility, enabling project managers to oversee resources across NRC contributing to individual projects in a structured and consistent manner. Integration of Project System with the financial management, controlling, sales and distribution, human resources and material management modules within SAP provides NRC with a holistic view of NRC’s activities facilitating more efficient and effective resource allocation, analysis of cost drivers and areas for cost reduction, and monitoring of revenue.

Collection and measurement of performance indicators for investment projects including financial metrics is enabled through SAP Project System. Project managers are provided the ability to organize projects by work breakdown structures with specific financial targets for defined activities. PPS also provides performance reports to the Senior Executive Committee on a monthly basis reporting on the elements of time, budget and scope through a manual project tracking process.

5. Monitoring: A monitoring framework is in place and monitoring activities are undertaken to ensure effectiveness and compliance with applicable Government of Canada and NRC policies, directives and processes.

Adequate

5.1 NRC has the necessary elements to support a monitoring framework to ensure effectiveness and compliance of NRC investment planning activities with applicable Government of Canada and NRC policies, directives and processes. Monitoring and oversight activities are occurring at different levels of management to provide the Deputy Head with assurance that the NRC investment planning process is effective, efficient and demonstrates value and stewardship of public funds. As noted in 1.2, there is a need to better define the body responsible for monitoring the investment planning framework and process within NRC to ensure its effectiveness and continued compliance with Treasury Board requirements.

See Recommendation 3

Related opportunities for improvement are provided in connection with audit criteria 1.2.

There are opportunities to better ensure the completeness of projects included within the investment planning process, the consistency of practices related to all investing activities and the effective monitoring of investment planning, at both the tactical and strategic level by:

  • Implementing a monitoring process to ensure that all real property, IT, scientific equipment and strategic enabler-type projects that meet the risk and complexity criteria of an investment plan project are captured within the investment planning framework and subject to the rigor and oversight of the framework; and
  • Assigning responsibility to help ensure that investments falling outside of the investment planning process (e.g. projects under $250,000 in value) are managed by respective Divisions according to their risk and complexity and in a manner that is consistent across NRC.
  • Integrating the monitoring elements to create a comprehensive monitoring framework to support investment planning within NRC. Assigning responsibility to measure the performance of the departmental investment plan as a whole by integrating the monitoring activities of PPS (major capital activities), Division Heads (minor capital activities) and Finance Branch (financial activities) would provide management with a holistic assessment of NRC's effectiveness and compliance in the allocation and reallocation of funding to achieve departmental strategic outcomes in accordance with Government of Canada and NRC policies, directives and processes.

5.1a/b/c The effectiveness of the investment planning framework was reviewed by an external consultant to identify opportunities for streamlining and simplification in January 2011. Implementation of the streamlining exercise recommendations is ongoing as PPS seeks to align the investment planning and project management framework with NRC’s program-based management model and related financial and operational processes and activities. Other priorities and changes to NRC’s operating environment have delayed the development of a schedule for third party reviews of the investment planning and project management framework to support continuous improvement.

Project progress is monitored by the Senior Executive Committee through monthly project status and financial reporting, and quarterly budgeting and performance reviews. Finance Branch Comptrollers have been tasked with providing financial analysis and monitoring of investment projects within their branches and portfolios as part of recently adopted financial oversight practices.

There is an opportunity to ensure risks continue to be effectively managed by clarifying and formalizing the roles and responsibilities among monitoring bodies to ensure a coordinated approach between PPS and Finance Branch to avoid duplication of efforts and inconsistent reporting, escalation and resolution of identified issues.

Investment planning monitoring and oversight is performed at varying levels of the organization. Projects over $250,000 in value are managed according to the NRC Project Management Framework and NRC investment planning process. Projects under the threshold are managed at the Division level by respective Division Heads. PPS has no direct responsibility for oversight of minor capital investments.

Following changes to the operating environment, responsibility and accountability for ensuring that the NRC Investment Plan incorporates all investments of commensurate risk and complexity that align with the definition of a project within the NRC Project Management Framework is unclear. Ensuring that no investments bypass or are excluded from the investment planning process demonstrates an effective investment planning process and framework and may have implications on the NRC OPMCA rating and consequently, departmental delegated authorities from Treasury Board. As well, NRC currently lacks a body mandated with responsibility to verify that activities undertaken deliver on approved business cases and plans with concrete deliverables and quantifiable results.

5.1d PPS collects information on best practices and lessons learned from completed projects, incorporating them into continuous improvement efforts on templates, tools and guides. The recent creation of a Project Management User Community for bi-directional feedback between the NRC project management authority and project managers represents a new forum for communication and knowledge sharing. SharePoint is also leveraged as a resource to convey best practices and lessons learned to the overall community.

No opportunities for improvement were identified.

5.2 On an annual basis, NRC reviews the alignment of its funded and unfunded investment plan projects against the department's strategic direction; projects deemed to not directly contribute to the achievement of NRC’s Strategy are removed and the remaining projects are reprioritized. Investment plan projects overseen by PPS are reviewed on a periodic basis to assess their progress against the approved budget, schedule and scope and corrective action is taken where necessary.

Management efforts to align the investment planning, program management and budgeting processes would facilitate the review of investment and project alignment with NRC's strategic direction; the approval or termination of a research program would have direct implications on associated investment projects and facilitate decision making in the allocation or reallocation of funds to projects of greater strategic priority.

See Recommendation 3

Related opportunities for improvement are provided in connection with audit criteria 1.2 and 5.1(c).

The framework for monitoring the performance of investments is an area where a number of opportunities were identified:

  • Improving guidance and practices to demonstrate the link between investment projects and NRC’s overall objectives;
  • Improving the monitoring of quantifiable performance metrics for investment projects (with a view to improving comparability and objectivity of investment decisions);
  • Implementing a framework to support the measurement of project outcomes with objectives and assertions identified in the project business case and project management plan;
  • Implementing a framework that measures the performance of NRC’s investment planning activities holistically, including all elements of major and minor capital and acquired services.

5.2a Project performance characteristics such as budget, schedule and scope are measured on a periodic basis and monitored by the Senior Executive Committee. Performance measurement efforts have improved with updates to investment planning templates to ensure consistency in the use of quantitative metrics for investment decisions.

While PPS and Division Heads are responsible for managing their respective elements of the departmental investment plan, no single entity has been mandated with measuring the performance of the NRC Investment Plan as a whole. The lack of performance measurement of the departmental investment plan as a whole precludes NRC’s ability to assess departmental performance in the allocation and reallocation of resources to new and existing assets and acquired services to achieve NRC’s desired strategic outcomes.

Recommendation 3: Integral to its mandate to manage and oversee the departmental investment plan, PPS should consider implementing a comprehensive monitoring framework to measure overall performance of the investment plan and NRC’s performance as a department in the allocation and reallocation of resources. The framework should incorporate all elements of investment planning managed at the PPS and Division Head Levels, including acquired services, and should include quantitative performance metrics to ensure continuous improvement. [Priority: Moderate]

Appendix A: Audit Criteria

No.

Lines of Enquiry

1.

Governance & Roles and Responsibilities

1.1

  1. An effective governance structure and related governance mechanisms are in place.
    1. Management is actively involved in the oversight of the investment planning process to support the development of a departmental investment plan.
    2. The governance structure and mechanisms reflect Government of Canada accepted practices and support alignment of investments with NRC strategy
    3. The governance structure is regularly reviewed for appropriateness in light of changes within, or external to, NRC

1.2

  1. Roles and responsibilities, including those of external parties, are clearly defined and effectively communicated, understood and acknowledged.
    1. Roles and responsibilities support the horizontal nature of the investment plan and supports allocation of resources in a strategic manner (supports the overall organizational strategy, direction and vision)

2.

Investment Planning Processes

2.1

  1. There are clearly communicated policies, directives, guidelines and standards which define investment planning processes across NRC.
    1. NRC investment planning processes reflect Treasury Board requirements and/or guidance and support NRC’s OPMCA rating
    2. Processes and related tools are complete, aligned and support NRC’s investment planning objectives
    3. Investment planning policies, directives and standards are communicated to, understood and acknowledged by NRC staff
    4. Process and other changes resulting from the streamlining initiative by Program and Project Services support NRC’s investment planning framework and the associated recommendations are progressing as per approved management action plans

2.2

  1. NRC’s investment planning processes support rational, informed and supported financial stewardship and risk management through a standardized and methodical approach to investment planning.
    1. Investment planning processes are appropriately linked with, and aligned to, related processes such as; costing, budgeting and forecasting; strategic, business and financial planning; and program and project management
    2. Investment costing and budgeting processes account for total project costs and reflect: analysis of investment options, risk information/contingencies, lifecycle management requirements and NRC/Treasury Board guidance
    3. Planning/budgets incorporate risk mitigation plans
    4. There is a formal, continuous and structured review and challenge of assumptions and variances
    5. Project complexity and risk assessments (PCRA) are effectively and appropriately used

3.

Training, Capacity and Competencies

3.1

  1. Formal training is provided to appropriate employees in support of developing relevant investment planning competencies.
    1. Training and materials support project managers and employees on key investment planning processes, systems and related tools
    2. Project managers and employees are provided with appropriate training in financial management principles and concepts to effectively develop budgets, forecasts and informed assumptions
    3. A continuous improvement program supports the ongoing development of investment planning skills and knowledge (e.g. Lessons learned and project results are communicated across the organization to share best practices)

3.2

  1. There is a process to support Managers in identifying the availability of resources reflecting the new departmental accounting model, new information technology resources and processes and cross-government fiscal restraint.

4.

Systems and Information for Decision Making

4.1

  1. Information and systems are available to support decision-making across NRC.
    1. Information provided to decision-makers regarding investment projects is sufficient, reliable and timely
    2. Systems support the effective capture, synthesis, analysis and reporting of information.
    3. Systems enable collaboration, flexibility and responsiveness given the departmental focus on revenue generation.
    4. Systems track and collect performance information of projects including financial metrics

5.

Monitoring

5.1

  1. A monitoring framework is in place and monitoring activities are undertaken to ensure effectiveness and compliance with applicable Government of Canada and NRC policies, directives and processes.
    1. There is a process for monitoring the effectiveness of the investment planning framework
    2. Project progress is monitored and corrective action is escalated/taken until issues are resolved or the project is terminated
    3. There is a process to ensure all investment projects are identified and tracked (e.g. split projects/ funded projects, and under-costed projects are identified)
    4. Monitoring supports the continuous improvement of investment planning practices and processes

5.2

  1. Investments and projects are reviewed on a periodic basis to monitor progress and reassess alignment with NRC’s Strategy.
    1. The performance of investments is monitored through an effective framework to define, collect, measure, assess and report the effectiveness of investment decisions (e.g. performance against those metrics which are aligned with NRC’s investment objectives)

Appendix B: Potential Overall Ratings

Management Attention Required

Significant issues exist that require management’s attention. Resources devoted to investment planning activities are being managed without due regard for efficiency, economy and effectiveness.

Needs Improvement

Some areas of practice / processes are in compliance with Government of Canada and NRC policies, directives and regulations pertaining to investment planning but many deficiencies exist. The deficiencies identified, whether taken individually or collectively, may compromise the control objectives and / or sound management of investment planning activities.

Adequate

Most of the areas of practices / processes are in compliance with Government of Canada and NRC policies, directives and regulations pertaining to investment planning with opportunities for continuous improvement and any identified deficiencies, whether taken individually or collectively, do not compromise the overall adequacy of the management control framework for investment planning. 

Strong

All areas of practices / processes are in compliance with Government of Canada and NRC laws, regulations, policies and directives pertaining to investment planning. No material weaknesses were identified in the overall control framework. The principles of continuous improvement are being practiced.

Appendix C: Management Action Plan

Priority of Recommendations

High

Implementation is recommended within six months to reduce the exposure to high likelihood, high impact events that may adversely affect the integrity of NRC’s governance, risk management and control processes.

Moderate

Implementation is recommended within one year to reduce the exposure to events that may adversely affect the integrity of NRC’s governance, risk management and control processes.

Low

Implementation is recommended within one year to adopt best practices and/or strengthen the integrity of NRC’s governance, risk management and control processes.

Recommendations

1.

PPS should continue working toward integrating investment planning processes, program planning activities and NRC’s budget cycle and continue to support initiatives to leverage tools and capabilities that will maximize the use of existing resources prior to consideration of new and/or additional investments. [Priority: MODERATE]

Corrective Management Action Plan

Program and Project Services (PPS), which is now responsible for the administration of Investment Planning, has outlined in its 2013-14 Operational Plan, a priority activity to continue its efforts to more fully integrate and align investment plans (and the timing of annual budget process) with program plans. This will include implementing changes already in progress that will require submission documents to clearly justify the expected demand or need (by programs and clients, or as part of maintenance, regulatory requirements, etc.), the value of the investment (in terms of utilization, revenue), and a full economic analysis showing cost-benefits that indicate life-cycle costs, payback period, etc. This change will be accomplished through the formal integration of investment plan business cases as part of annual operating plans for portfolios and programs.

Expected Implementation Date & Responsible NRC Contact

Contact: Executive Director, Program and Project Services
Date: 31 March 2014

2.

PPS should consider establishing a timeline for developing and implementing training and competency targets for Project Managers. These targets should address specific areas including costing, development of assumptions, project management, performance management (including use of key performance measures such as ROI), financial management concepts, budgeting, and documentation of investment project files. [Priority: MODERATE]

Corrective Management Action Plan

The development and implementation of training and competency targets for projects managers is being undertaken as part of a larger PPS priority activity for program and project management competency building that is being undertaken in collaboration with Human Resources Branch. Generic competency targets will be included for all project managers, with additional targets identified for those managers that lead investment projects, including items such as building business cases (e.g. ROI analysis) and documentation of investment project files.

Expected Implementation Date & Responsible NRC Contact

Contact: Executive Director, Program and Project Services
Date: 31 March 2015 for full implementation

3.

Integral to its mandate to manage and oversee the departmental investment plan, PPS should consider implementing a comprehensive monitoring framework to measure overall performance of the investment plan and NRC’s performance as a department in the allocation and reallocation of resources. The framework should incorporate all elements of investment planning managed at the PPS and Division Head Levels, including acquired services, and should include quantitative performance metrics to ensure continuous improvement. [Priority: MODERATE]

Corrective Management Action Plan

PPS in collaboration with Finance Branch has already assembled a roles and responsibilities document for financial support that is provided for the financial monitoring of investment projects. This document will be formally reviewed, finalized and signed-off between the two parties. Additionally, a new investment project status report template has been created (ready for roll-in first quarter of 2013-14) for use by PPS staff, project managers, and comptrollers that will better delineate the different roles and responsibilities from a reporting perspective. Quantitative metrics to ensure continuous improvement have been identified as a priority activity in the PPS 2013-14 Operational Plan, and will be supplemented by new Treasury Board (TB) performance requirements being developed as part of the current TB submission to renew the NRC Investment Plan.

PPS will also develop, in consultation with the Division Heads (VPs), a proposal for an annual monitoring exercise for projects <$250k.

Expected Implementation Date & Responsible NRC Contact

Contact: Executive Director, Program and Project Services
Date: 31 March 2014

Footnotes

Footnote 1

As per the Treasury Board Guide to Using the Organizational Project Management Capacity Assessment Tool, management self-assesses their project management capacity classification level – i.e. limited (0), sustaining (1), tactical (2), evolutionary (3), or transformational (4).

Return to footnote 1 referrer

Footnote 2

Major capital projects within the context of NRC are defined as investments greater than or equal to $250K whereas minor capital refers to projects under $250K in value.

Return to footnote 2 referrer

Date modified: