Progress on performance management

In November 2016, a new framework for performance management was approved by CGIAR’s System Council, which recognized the complexity, risk, multiple objectives and long timeframes inherent in agricultural research for development (AR4D). Drawing on a framework originally developed for Canada’s International Development Research Centreⁱ, it conceptualized the results of AR4D as falling into three concentric spheres (Figure 1). These are: the Sphere of Control, the direct products of CGIAR research; the Sphere of Influence, where CGIAR may have some input but cannot control the outcome (such as contribution to a policy decision); and the Sphere of Interest, where CGIAR has very little control and which may happen many years after the research, such as adoption by farmers of a technology at large scale and at their own cost.

This decision was followed by the approval at the end of 2017 of a new results reporting system, following consultations with Centers, Programs and Funders. The principles underlying the new system included:

• Reporting credible, robust data based on checkable evidence.
• Aiming to report through Management Information Systems, not as a separate exercise.
• Not using the results mechanistically to compare research programs or in decision-making on funding.
• A representative range of reporting information and indicators sufficient to demonstrate progress in the spheres of control, influence and interest of CGIAR.
• Choosing indicators that would be relevant to all parts of the CGIAR System, which produces very diverse outputs (for example, ‘innovations’ rather than ‘varieties released’)
• Parsimony: minimizing the information required to that needed for accountability and decision-making, as reporting has a high cost.

Figure 1. Integrated performance management framework for CGIAR research¹

Components of the reporting system included the introduction of revised planning and reporting templates collecting new information from Programs (for example on the use of pooled ‘Window 1/2’ funding) and a significantly revised and streamlined set of common reporting indicators with detailed guidance. An annual system-level performance report (this report) would then draw from these. Plans were also made to scale up existing Management Information Systems, at the time used only in a minority of CGIAR Research Programs, and then link them to a central reporting dashboard.

Implementing such a reporting system takes time, especially in a diverse system like CGIAR. However, there are distinct signs of progress. This report reflects the first year of reporting using the new templates and indicators. There is a mass of rich material to draw on, some of which is in the databases linked to this report. Data or supporting evidence is still incomplete in places, but the templates and guidance are being revised based on the 2017 experience and user feedback, and the improvements should be visible in future reports. In the meantime, adoption of new Management Information Systems has proceeded rapidly, and by 2019 they should cover all CGIAR Research Programs. An interactive results dashboard fed by these ‘interoperable’ systems is under construction.

Monitoring and reporting form only one part of a performance system. At the moment, work is underway on a series of other important areas, including reform of program appraisal and evaluation processes, and agreeing new program performance standards, expected to be approved in November 2018. These will be covered in more depth in future CGIAR System annual performance reports.

Monitoring progress in program implementation

Each of the CRPs and Platforms develops an annual Plan of Work and Budget against a standard template, that sets out key Research and Development milestones² for the year along each of the impact pathways. Typically, a CRP may have 20-30 annual milestones.

Reported progress in 2017 for each CRP against achievement of each planned milestone is shown in the evidence table available in Achievement of planned milestones in 2017, together with evidence of achievement (if complete) or an explanation (if incomplete, extended for a further year or canceled). Overall progress is shown in Table 1³.

Table 1. Achievement of planned R4D milestones in 2017

Source: CRP Annual Reports 2017

Each Program and Platform also submits a detailed annual report on progress. The reports include a description of major areas of variance from planned program, and any changes to the overall impact pathway. In 2017, programs reported some areas of expansion and some areas which were cut back. Expansion mostly resulted from funder and partner demand to support new areas of work which respond to topics of current policy interest.

Examples of these expanding areas of demand include:

• Urban food systems and food safety in the informal sector in Africa (Reported by A4NH),
• Sustainable rice straw management to avoid straw burning (Reported by RICE),
• Linkages between ecosystem health, food production or systems and human wellbeing in areas such as synthetic proteins and water-related diseases (Reported by WLE).

Most cuts resulted from funding cuts (for example in bilateral projects), or reluctance to invest in the face of budget uncertainty in the case of CGIAR Fund Window 1 and 2 (W1/2), and occasionally from lack of specialist staff, although one program (CCAFS) mentioned cutting back lower-performing research. This is an area CGIAR hopes to be able to better report on in 2018.

Use of pooled funding (W1/2)

The CGIAR Trust Fund provides for two types of pooled funding channel for programming carried out by CGIAR Programs and Platforms themselves, through the so-called Funding Windows 1 and 2 (W1/2). Platforms are almost entirely funded through W1/2.

Research Programs used W1/2 in slightly different ways, depending on the nature of their research, relative proportion of project funding (‘Window 3/bilateral’) they received and whether project funding was able to cover certain key areas of work (e.g. gender integration, M&E) or not. The detailed results are shown in this linked table (Main areas of CGIAR Fund Window 1/2 expenditure for 2017). Nevertheless, there were some common patterns.

In general, Window 1 and 2 (W1/2) funding provided the backbone of RICE and catalyzed impact through strategic investments along the whole impact pathway, from upstream research to downstream development of business models and multi-stakeholder partnerships for innovation and scaling out. …The long-term nature of W1/2 funding provides the continuity to the program, and guarantees not only short-term impacts (as derived from most bilateral projects) but also long-term impacts on 5- to 10-year time scales. Most W1/2 funds were used to support …key monitoring, evaluation and learning activities across all projects, gender analyses and mainstreaming, capacity development and partnership building for scaling out and achieving impact at scale, and new initiatives (such as farm diversification, value-chain analyses).
Introduction to the response from RICE CRP

Some examples of W1/2 use in 2017 include:

(a) Start-up investment on emerging research priorities, leading to later funding of W3/bilateral projects, e.g. on fall armyworm (a serious new pest) or precision agriculture (a cutting-edge approach). (Reported by MAIZE)

(b) Competitive allocation to projects selected under a ‘Scaling Fund’ to foster the scaling of the most promising RTB innovations and generate an evidence base on scaling approaches. (Reported by RTB)

(c) Supporting new collaborative work on seed policies with several centers and CRPs. (Reported by PIM)

(d) Developing innovative frameworks e.g. to evaluate the environmental impacts of productivity enhancing technologies. (Reported by LIVESTOCK)

(e) Financing a review of equity issues in research, which is set to inform new equity research across A4NH from 2018. (Reported by A4NH)

(f) Financing international policy engagement to better leverage research results, for example work on soil carbon feeding into the UNFCCC Koronivia Decision on Agricultureⁱⁱ deliberations. (see box) (Reported by WLE)

(g) Building and maintaining external partnerships, including start-up costs and maintaining and continuing work between projects. (Reported by several CRPs)

(h) Developing novel tools and approaches, for example, new analytical approaches for land restoration planning and rapid soil analysis. (Reported by WLE)

(i) Financing capacity development, for example developing curricula and training modules based on research results, supporting participation of national partners in key events, supporting a breeding community of practice. (Reported by several CRPs, including WLE and RTB)

(j) Supporting integration of gender across the program. (Reported by most CRPs)

(k) Funding key research management and communications functions. (Reported by several CRPs)

(l) Financing core research, for example, CCAFS uses W1/2 to fund the core elements of its strategy and only accepts additional project funding to complement this. (Reported by CCAFS)

(m) Adding value to project funding, e.g. extending scaling work to additional countries or to expand a promising area of research. (Reported by several CRPs including RICE, LIVESTOCK)

(n) Supporting key ex ante and ex post studies to determine research priorities. (Reported by WHEAT and PIM)

(o) Supporting other monitoring and learning activities across the portfolio, including investment in a new Management Information System. (Reported by several CRPs)

Footnotes

¹ See conceptual framework in SC3-03, 17 Nov 2016 http://www.cgiar.org/wp-content/uploads/2016/11/SC3-03_Towards-PerformanceMgmtSystem_17Nov2016.pdf

² A milestone can be, for example, the completion of a significant activity, such as completion of a set of trials, or a major survey; the release of a particular technology onto the market; or the production of significant new evidence about the effects of a policy or variety. Milestones may also mark important decision points in the research, for example ‘Results of trial X will inform decision on whether to continue developing technology Y or take a new direction’.

³The table does not show comparative achievement data for different CRPs, because there is strong evidence both at the international level and from previous experience in CGIAR that simplistic comparison of percentage achievement, especially when it informs funding allocation, quickly leads to goal displacement and a reluctance to take risks which are essential for high-payoff research. However, the management and independent governance bodies of each Program, which include experts in the relevant field of research and acquainted with the realities of each line of work, are expected to closely scrutinize progress and achievements against each planned milestone, and the level of scrutiny will in its turn be evaluated by system bodies and in independent evaluations.

ⁱ Z. Ofir et al., “Research Quality Plus (RQ+): A Holistic Approach to Evaluating Research,” 2016, https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/56528/IDL-56528.pdf.

ⁱⁱ UNFCCC secretariat, “Decision -/CP.23; Koronivia Joint Work on Agriculture: Advance Unedited Version,” 2017.

Photo by N. Palmer/CIAT.