ARCHIVED - Evaluation of Central and Western Cluster Initiatives - NRC Institute for Fuel Cell Innovation
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.
Table of contents
In 2001, NRC was allocated resources to target a number of emerging research and technology fields that were identified by local partners in consultations with NRC. Under the central and western technology cluster initiatives (CWI) six initiatives were funded, each with an identified technology focus. These included:
- aluminum transformation in the Saguenay;
- photonics fabrication in Ottawa;
- biomedical in Winnipeg;
- nutraceuticals and functional foods in Saskatoon;
- nanotechnology in Edmonton; and
- fuel cells in Vancouver.
In 2005, an evaluation of these Round II technology cluster initiatives was launched and this report presents the key findings, conclusions and recommendations related to the NRC Institute for Fuel Cell Innovation.
The primary reasons for conducting an evaluation of the initiative at this time are as follows:
- to collect information on the progress of the initiative to date, including lessons learned and novel practices, as a means of supporting NRC's strategic direction in contributing to the socio-economic sustainability of Canada's communities, through technology clusters;
- to provide an opportunity to communicate with initiative stakeholders in the communities; and
- to provide information on NRC's performance to date, to be used to facilitate decision-making around funding renewal of the cluster initiatives, which expires at the end of a five year funding cycle (2006-2007).
NRC Senior Executive Committee approved the Terms of Reference for this evaluation in September 2005.
The evaluation covered the period 2002-2003 to 2005-2006 inclusive. It addressed issues related to relevance; early outcomes and impacts; design, delivery and cost-effectiveness, and lessons learned and novel practices. The federal government's Expenditure Review Committee questions were also taken into account in the development of the evaluation issues.
The key methodologies used to address the evaluation issues included a review of documents; a review of administrative and performance data; key informant interviews, and a cluster measurement study.
Fuel Cell Technology Cluster Initiative Overview
NRC launched its fuel cell technology cluster initiative in 2002 as part of a second round of funding dedicated to technology clustering. Lead delivery of the initiative is the responsibility of NRC's Institute for Fuel Cell Innovation (NRC-IFCI). NRC's Canada Institute for Scientific and Technical Information (NRC-CISTI) and NRC's Industrial Research Assistance Program (NRC-IRAP), along with a number of regional partners,Footnote 1 also play roles in the initiative.
NRC-IFCI received funding to enable the NRC-Innovation Centre to become a full-fledged research Institute in fuel cells and hydrogen, in addition to developing its core focus on fuel cell testing and demonstration, and to become the "flagship" for NRC's fuel cell research in Canada and internationally. Overall, the mission of NRC-IFCI is to support the growth of British Columbia's fuel cell cluster and the leadership of the Canadian fuel cell and hydrogen industry through excellence in relevant research, innovation, partnerships and cluster building.
A total of $20M in federal resources was allocated to the fuel cell technology cluster initiative in its 2001 Budget. All NRC resources in support of the technology cluster initiative have been directed at the NRC-IFCI. None were allocated to NRC-CISTI or NRC-IRAP.
Based on the evaluation study conducted from October 2005 to June 2006, the following key evaluation findings and recommendations were identified.
NRC's overarching rationale for launching technology cluster initiatives is to support the economic prosperity of communities. The evaluation found that the fuel cells cluster initiative is a relevant initiative that fits with the needs and strengths of the greater Vancouver region and Canada. The creation of a full-fledged research Institute with testing and demonstration capability that is dedicated to supporting the hydrogen and fuel cell industries is viewed by both the region and the sector as being of preeminent importance. While Canada led the development of fuel cell technologies, Canada is at risk of falling behind as other countries increase their investments in fuel cells and the industry struggles to commercialize in the near-term.
Interviewees consistently pointed out that NRC-IFCI started with virtually no capabilities in fuel cells, a limited budget, little flexibility in staffing, and a building that was not intended for hydrogen and fuel cell research. The predominant message from interviewees was that given these constraints and challenges, and in the time they have had, NRC-IFCI has come a long way and done a good job in developing capacity. This is born out by the performance data, which shows increased levels of research activity, publications, collaborations, and presentations. As well, NRC-IRAP has played a critical role in fostering the cluster's firms, and NRC-CISTI has adapted its services to meet the needs of researchers. Novel arrangements between NRC-IFCI and NRC-IRAP are increasing their cooperation and the quality of services provided to firms.
NRC-IFCI has embraced a partnership approach to all aspects of its activity, including:
- in research, by seconding researchers from the universities to build expertise;
- in facilities and equipment, by working with Western Economic Diversification Canada (WD), and Fuel Cells Canada (FCC) to develop hydrogen safe labs and the Hydrogen Technologies Environmental Chamber;
- in testing, with the Vancouver Fuel Cell Vehicle Program; and
- in demonstration, with the Hydrogen Highway.
In so doing, NRC-IFCI has contributed significantly to: increased communication and collaboration throughout the cluster; increased profile for the cluster internationally; greater leverage on resources; and greater complementarity between cluster actors in the region.
Summary of Findings and Recommendations – Relevance
The evaluation found that the launch of NRC's fuel cell technology cluster initiative was consistent with federal government and NRC priorities established in 2002 through the Innovation Strategy. The initiative continues to be aligned with federal priorities, and in particular, with NRC's new Strategy. The evaluation also found that there is a legitimate role for NRC in fostering the development of clusters, and in supporting the development of secure, clean energy sources. Overwhelmingly stakeholders consider that there is a legitimate and necessary role for the NRC to sustain fuel cell research, to keep Canada on the leading edge, and to facilitate commercialization.
NRC-IFCI is making progress in meeting stakeholder needs. Most interviewees felt that NRC-IFCI is "core to the cluster", and an important resource for an industry that is struggling to generate revenue. They note that NRC-IFCI has fostered collaboration and cooperation among stakeholders. Some noted that the Institute has been challenged in clearly identifying stakeholder needs given the industry's focus on short term commercialization and its rapidly changing needs.
NRC-IFCI must articulate a value proposition to industry to ensure its research and development (R&D) complements industry rather than potentially placing itself in competition with industry.
Management Response and Proposed Actions: Accepted. We will regularly review, with our partners, our research directions against short, medium, and long term needs of the fuel cell cluster, and make adjustments in order to ensure that we maximize our impact in meeting the needs of industry through the unique provision of technology breakthroughs and expertise, and core-competencies.
Our Cluster Business Plan is based on industry consultations, an independent survey of their needs, and the Round Table, and as such has captured key parts of their recommendations:
- We will communicate our cluster plan to the cluster members, advisory board and other stakeholders, including NRC-IFCI staff.
- The cluster funding will be dedicated to working with industry to meet the short and medium term needs. We will also use a small portion of the funding to collaborate with universities to address next generation technologies that will enable the industry to sustain its global leadership.
- We will use our A-base funds to develop relevant core competencies that will allow us to become world leaders enabling us to meet industry needs, and will dedicate a small portion to exploratory work.
- We will link the cluster with universities and global research networks to bring in expertise and knowledge.
Summary of Findings and Recommendations – Early Outcomes and Impacts
Access to Research Facilities: NRC-IFCI has significantly increased the infrastructure for fuel cells and hydrogen research. The Institute offers hydrogen safe laboratories; test stations which can test fuel cells of varying wattage and their components, while controlling for critical parameters. Through a joint initiative with WD and FCC, NRC-IFCI has established a Hydrogen Technology Environmental Chamber (HTEC) which can test fuel cell systems of varying size under a broad range of environmental conditions. However, despite support from industry, use of the HTEC has been slower than anticipated.
NRC-IFCI should focus on ways to increase usage of the HTEC, including the implementation of targeted marketing efforts.
Management Response and Proposed Actions: Accepted.
- NRC-IFCI will continue to implement the marketing plan for the HTEC. We will allocate 20% of the Business Development Officer's (BDO's) time as well as 40% of Technical Officer (TO) time towards developing projects for HTEC.
- We will continue to work with H2FCC to increase utilization of HTEC by the BC fuel cell cluster.
- We will establish a partnership with Powertech Labs and CTC to market the HTEC to other industry sectors such as oil and gas, mining and military.
Attraction and Development of Highly Qualified Personnel: NRC-IFCI has been successful in attracting and developing HQP, despite having started with virtually no capability in the technology area and with limited flexibility in staffing. The Institute has made good use of arrangements such as secondments and has successfully reoriented a number of internal teams. With 101 staff in 2004-2005, the Institute is on track with initial staffing plans. Interviewees cautioned against expecting too much too soon, considering that it will take five to ten years to build significant depth and expertise. There is some concern that NRC-IFCI will not be able to retain HQP over the long term if the market turns around.
NRC-IFCI must continue to focus on building internal research capabilities.
Management Response and Proposed Actions: Accepted. Building core competencies and research leadership skills is critical in ensuring the sustainability of the institute and is therefore a top priority. In addition to training research leaders, we plan to increase the number of research positions through increased revenue generation.
- NRC-IFCI will allocate all its science and technology (S&T) A-base funding towards building internal research capabilities (core competencies) in low temperature and high temperature fuel cell technologies and in doing exploratory research.
- We will focus on the core competencies that are relevant to the industry with direction from the industry & an insight from market & technology intelligence studies to ensure our uniqueness.
- We will develop & implement staff training plan.
- We will put in place portfolio management process to ensure focus & sustainability.
Level of Research Activity: NRC-IFCI is making good progress in its fuel cell research program. All measures of research outputs (publications in peer-reviewed journals, technical reports, and patent applications) have increased since 2002. The Institute has signed 19 memoranda of understanding (MOUs) resulting in 53 projects, and has a further 15 collaborative research projects underway. However, interviewees felt that the focus of NRC-IFCI research – covering PEM, SOFC and hydrogen – appears to be very broad given resources levels. Further, the value proposition of the Institute's R&D program is not clear to all stakeholders.
NRC-IFCI must ensure its research plan is focused and is within its resources, and clearly communicate it to stakeholders.
Management Response and Proposed Actions: Accepted.
- NRC-IFCI will continue to focus by reducing the number of technology platforms for its internal R&D program. We will do this in consultation with industry technical committee and our university partners.
- We will continue to focus on building world-class expertise in fuel cell materials, design, and fabrication of fuel cells.
- Quarterly communication of progress to the Advisory Board & key stakeholders.
Access to NRC Technology and Industry Support Resources: NRC-CISTI and NRC-IRAP are well integrated with NRC-IFCI and are making positive contributions to the cluster initiative. NRC-CISTI has built a library collection oriented to fuel cells and is evolving its services to meet researcher requirements for more lengthy, complex and in-depth literature and patent searches. Prior to the launch of the fuel cell technology cluster initiative, NRC-IRAP played a catalyzing role in consulting industry on the role NRC could play. The Institute and NRC-IRAP have established mechanisms for close collaboration, and industry is strongly supportive of the support provided by NRC-IRAP.
Cluster Networking and Industry Development: The evaluation found that the Institute has played an extremely important role in facilitating cluster networking. Interviewees noted that there was very little interaction between players prior to NRC-IFCI and consider that it has been instrumental in bringing the universities together, connecting the universities with industry, and in providing firms with opportunities to network.
NRC-IFCI is contributing to industry development; however it has faced a number of challenges: the industry entered a downturn as NRC-IFCI was established and firms have few resources for collaborative research. Nonetheless, the Institute's hydrogen safe labs have been extremely useful to industry, and the number of incubating firms has increased. NRC-IFCI is but one component of a successful innovation system that industry will require to succeed.
Incremental Benefits: NRC-IFCI has contributed to a number of additional benefits to the cluster. In particular, NRC-IFCI has raised the profile of the province and of Canadainternationally. Interviewees noted that the Institute provides a focal point for foreign delegations, and that these visits have increased. As well, NRC-IFCI has been instrumental in launching the Hydrogen Highway, a coordinated, large-scale demonstration and deployment program to accelerate the commercialization of hydrogen and fuel cell technologies which is intended to be operational for the Winter Olympics in 2010. Finally, an unanticipated benefit resulting from the requirement to move to a new facility (discussed in Section 5.1 – Initiative Implementation) is that the Institute now has one of the greenest buildings in Canadaand a Leadership in Energy and Environment Design (LEED) certification from the U.S. Green Building Council.
Summary of Findings and Recommendations – Design, Delivery and Cost-Effectiveness
Initiative Implementation: The implementation of the Initiative was driven by the 2002-2007 NRC Fuel Cell Program Strategic Plan, as well as business plans, a technology plan and a human resources (HR) plan. Despite the decision by the University of British Columbia (UBC) to have the Institute move to a new building at a different location on campus, theevaluation found that the implementation of the fuel cell technology cluster initiative has largely progressed according to these plans.
Effectiveness of Management Practices
NRC-IFCI has established an Advisory Board which provides strategic advice on program and policy matters relevant to the Institute, and consists of representatives mostly from private industry and academia. A review of documents suggests that the Advisory Board meetings have been held sporadically – in one year, three meetings were held while in another, there were no meetings. The review of documents suggests that there are numerous potentially conflicting perspectives from the Advisory Board as to whether the Institute should focus on short or long term research.
Strategy and Planning
In a short period of time, NRC-IFCI underwent rapid change and expansion in roles from an Innovation Centre supporting manufacturing innovation broadly, to undertaking testing and demonstration of fuel cells as part of NRC's horizontal fuel cell program, to becoming a full fledged fuel cells Institute that was intended to be the "flagship" for NRC's fuel cell research. The Institute has embraced its new roles and has developed numerous plans and partnerships towards realizing these. However, there is concern regarding the balance between planning and implementation of plans, and there is concern that the Institute is pursuing too many objectives and spreading resources too thinly across these.
Management and Administrative Processes
NRC-IFCI is beginning to implement the processes and systems necessary to effectively manage the Institute. While some of these are relatively recent, the evaluation found several reasons including: the Institute only began to ramp up in 2003, there was turnover in key research and administration positions, and the fact that the management team was new to NRC and distant from corporate headquarters resulted in some challenges in implementing systems.
The biggest challenge in transforming the Innovation Centre into the Institute for Fuel Cell Innovation was human resources. The Institute was not able to consider workforce adjustments at the outset of the initiative and had limited resources for new hires. The approach taken was to hire several experts and have them train existing staff on a range of projects, some of which were internally funded and some of which were for industry. It generated mixed results: some researchers left, some have very successfully adapted to fuel cell applications, and some have found it difficult to adjust. However, the Institute is now on track, having recently implemented a competency based workplan, resulting in some workforce adjustments.
NRC-IFCI must focus on managing resources and implementing plans, and continue the development of management practices and processes to deliver against objectives.
Management Response and Proposed Actions: Accepted.
- NRC-IFCI has developed a project review and planning process that allows us to manage the resources, review progress, and make decisions on new initiatives.
- We will continue to improve the process to maximize resource utilization.
- We will establish an Innovation Program Committee, consisting of group leaders and management, to regularly review progress, needs, and priorities.
NRC should consider all options, including work force adjustments, when making a significant shift in research focus.
Management Response and Proposed Actions: Accepted.
- We are in the process of finalizing the restructuring that will include work force adjustment. The initial strategy was to allow time and resources for retraining the staff to meet the needs of the new mandate. Over the last two years we have been going through the process of realigning which will result in reorganization and work force adjustment.
- We will align our resources to deliver on the technology and business plan.
Role of NRC Institutes, Programs and Branches: Other Institutes, programs and branches have been directly involved in fuel cells R&D with NRC-IFCI since the launch of the Fuel Cell Program in 1999. This collaboration has continued to improve as NRC-IFCI has gained expertise in fuel cell R&D. While NRC-IFCI has been positioned as the "flagship" Institute for fuel cell research at NRC, internal interviewees noted that there is some confusion as to the meaning of a "flagship" Institute.
NRC needs to clearly articulate what is meant by "flagship" Institutes and what roles and responsibilities are related to this.
Management Response and Proposed Actions: The industry is requesting NRC to provide one portal, which coordinates & focuses resources within NRC. In our view NRC-IFCI can be the portal for fuel cell R&D because NRC-IFCI is an applied Institute, located in the centre of the largest fuel cell cluster and has become the centre of activities for the fuel cell industry as a whole. It has the largest FC research group in Canada and works closely with its university partners to focus on meeting the needs of the industry. NRC-IFCI has become the hub that coordinates the fuel cell R&D community in Canada and interfaces with industry. Hydrogen & Fuel Cells Canada, the national industry association, is headquartered at IFCI and regularly communicates industry-involving needs. We have built strong relationships with industry and the universities and established strong links with international research organizations. In addition, NRC-IFCI, with its Industrial Partnership Facility (IPF)/testing/demo facilities, industry led Advisory Board, and strong links with the NRC-IRAP ITA community, is very much aware of market & industry needs. So, as the portal, NRC-IFCI, on one side, can communicate those needs to the NRC FC Horizontal Program. The Horizontal Program will use that information to establish criteria for project selection that enables development of critical and yet novel knowledge platforms to be used by the applied institutes to support industry partners. And/or on the other side, NRC-IFCI can address multiple needs/requests from industry by linking them and/or their projects/consortia to appropriate NRC wide skills & expertise.
Partner Contributions: The evaluation found that NRC-IFCI has played a significant role in driving and establishing partnerships with other government departments, other levels of government, industry associations, and the private sector. As one intervieweenoted, coordination is effective because stakeholders in the region co-invest in programs.
Level of Resources: The Innovation Centre had an existing A-base budget of approximately $4M per year, and was allocated an additional $4M per year ($20M over 5 years) to transform itself into a full fledged research Institute focused on fuel cells. The evaluation found that the Institute has made good use of the resources available, and has leveraged additional resources through partnerships. However, the objectives for the fuel cell initiative are ambitious when compared to the funding received and NRC-IFCI must, in the absence of additional resources, ensure that its planned results are achievable within their resource levels.
Alternatives: NRC-IFCI made very effective use of resources to achieve its objectives by establishing cross appointments, secondments, and joint arrangements with multiple organizations. However, a significant number of interviewees were of the view that a more efficient use of resources could have been made if the Institute had been able to undertake a workforce adjustment early on in the launch of the Institute.
Lessons Learned and Novel Practices
The final area examined by the evaluation was lessons learned and good practices. The findings highlight key lessons learned that could be applied or adopted by the central and western technology cluster initiatives as well as other initiatives launched by NRC.
- Reorienting the research focus of an Institute is a slow and challenging process, and clear strategies for managing the transformation of core competencies are critical.
- Institutes building new areas of research have to carefully consider their capacity and expertise when approaching industry to develop collaborative research projects.
- New Institute management, located away from NRC headquarters in Ottawa, would benefit from more support in learning NRC systems and culture.
- Joint arrangements and engagement with cluster stakeholders are important tools in building trust and collaboration and in meeting cluster initiative objectives.
NRC-IRAP and NRC-IFCI have implemented a number of steps to foster closer collaboration and increase each other's value added:
- The NRC-IRAP and NRC-IFCI shared Industrial Technology Analyst / Business Development Officer position helps link NRC-IFCI and NRC-IRAP, as well as NRC-IFCI with companies, and provides technical expertise to NRC-IRAP in assessing potential projects.
- An arrangement where, when NRC-IRAP meets with energy companies, a team of ITAs and the NRC-IFCI business development officer is present. This helps NRC-IRAP in its funding decisions, it provides value-added to the firms in terms of technical advice, it lets NRC-IFCI see what industry is doing, and puts a face on NRC-IFCI for the companies.
- Footnote 1
Use of the term 'partner' and 'partnership' throughout this report should not be interpreted in the context of a legal partnership where risks are shared, but rather as initiatives where parties have simply jointly agreed to contribute resources (financial, in-kind) towards particular projects.
- Date modified: