CGIAR Programs depend on integrated monitoring, evaluation, learning, and impact assessment to test their assumptions, learn, and improve their work. A list of relevant activities reported in 2017 is available in the Status of evaluations, Impact Assessments and learning exercises in 2017. There was significant under-reporting, and this will be remedied in future. However, based on a preliminary analysis, the main types of activities reported by CRPs included:

• Studies seeking to understand impact pathways and test assumptions (Reported by 6 out of 9 CRPs) for example, studying different business models for the Happy Seeder technology in India (Reported by CCAFS), or understanding the role of input subsidies on the adoption of natural resource management technologies. (Reported by MAIZE)
• Ex post Impact Assessments (6 CRPs), for example an impact and adoption study on fertilizer tree systems in Malawi. (Reported by FTA)
• Adoption surveys (5 CRPs), for example an assessment of GIFT tilapia dissemination in Bangladesh and the Abbassa strain of Nile tilapia in Egypt. (Reported by FISH)
• Qualitative impact studies (6 CRPs), for example an assessment of uses of Statistics on Public Expenditure for Economic Development. (Reported by PIM)
• Ex ante analyses (4 CRPs), for example an ex-ante analysis of economic benefits and returns on investments in yam seed systems. (Reported by RTB)
• Learning workshops to discuss findings and feed these into programming (5 CRPs), for example an annual monitoring, evaluation, impact, and gender learning workshop. (Reported by RICE)
• Other activities, including systems development and training, were also reported by most CRPs.

The cross-CGIAR Monitoring, Evaluation and Learning Community of Practice (MELCOP), and the Evaluation Community of Practice regularly meet to share experience and improve Monitoring, Evaluation and Learning across the system. Their joint October 2017 workshop made an important contribution to the development of common CGIAR reporting indicators and planning and reporting templates and a CGIAR glossary of key terms, as well as holding learning events on approaches and methods. Standing Panel on Impact Assessment (SPIA¹), also convenes regular meetings of Impact Assessment Focal Points from centers and CRPs to share information, exchange feedback with each other and with SPIA, and identify opportunities for collaboration.

In 2017, SPIA and PIM co-sponsored a technical conference on the impacts of international agricultural research with 180 conference participants, about half from CGIAR and one-third women. The conference discussed evidence from recent rigorous studies on how and how much agricultural research has contributed to development outcomes and considered the implications for how CGIAR and others set research priorities and conduct impact assessments.

CGIAR has both the potential and responsibility to bring in new methods and to think really carefully about how to change the practice of impact assessment… so that we are not ignoring the questions that need to be answered in pursuit of the things we know how to answer.
Doug Gollin, Oxford University and Chair of SPIA

An important lesson from the conference was that: “there is often a contradiction between donor [Funder] demand for rigorous evidence and for evidence of impact at scale, especially in the short term… Strengthening communication between the impact assessment and donor communities should be a priority for CGIAR.”ⁱ

Oversight and advice from System Advisory Functions

The year 2017 saw CGIAR’s System-wide advisory functionsⁱⁱ providing an important source of guidance and objective assurance on the status and performance of CGIAR’s research agenda, the quality of that work, and its overall impact.

Highlights of their work included (see Annex E for a more complete picture).

Independent Science and Partnership Council (ISPC)

Part of ISPC’s work is carrying out ‘big picture’ thinking and foresight which among other things aims to keep CGIAR positioned at the cutting edge to address emerging global challenges.

In 2017, this included:

• Independent Foresight Assessment: 18 background papers were commissioned on global trends affecting agri-food systems, discussed at an international workshop and will be published in 2018. (see Box 1)
• In partnership with Australia’s CSIRO, ISPC worked to generate new insights on agri-food system innovation: including through the analysis of 17 diverse case studies of agricultural research and innovation, and also analyzing key blockages to effective research and innovation.
• ISPC convenes CGIAR’s biennial Science Forum bringing together world experts to discuss specific challenges. 2017 saw consolidation of the learning from the 2016 Forum, on pathways from agricultural research to rural prosperity, and planning for the 2018 Science Forum, on capturing synergies between SDGs through agricultural research.

ISPC also provides appraisal of CGIAR research programs and setting relevant standards.

In 2017, ISPC carried out:

• Independent program review of new CGIAR Research Programs, e.g. the new Grain Legumes and Dryland Cereals proposal.
• ISPC worked to develop a Quality of Research for Development framework (QoR4D) for CGIAR, through a workshop and follow-up consultation. The proposed QoR4D framework is presented as a key element to endorse in the inaugural CGIAR System three-year plan (2019-2021).

Another important ISPC workstream is independent impact assessment of CGIAR-related research, overseen by SPIA, a subgroup of the ISPC. The year 2017 saw the appointment of a new SPIA chair and also the finalization of a large five-year project on Strengthening Impact Assessment in CGIAR. Highlights of SPIA’s work in 2017² included:

• Publishing a database of varietal release and adoption estimates for 11 CGIAR mandate crops for 15 countries in Asia.
• Important advances in methodology for adoption studies on crop varieties, based on plant DNA testing. (see Box 2)
• Studies on adoption of widely promoted Natural Resource Management practices.
• Publishing a database of evidenced policy outcomes of CGIAR research.
• Synthesizing a variety of other studies carried out to assess the impact of widely adopted CGIAR innovations.ⁱⁱⁱ

Some of SPIA’s impact studies challenged conventional wisdom about impacts of agricultural research. Two of the most hard-hitting findings included:

• DNA analysis showed that in many contexts, many farmers cannot reliably identify the crop varieties they are growing, putting into question the published figures from some (but not all) varietal adoption studies, and highlighting the need to develop alternative methods that work at scale.
• Some of the most widely promoted on-farm Natural Resource Management practices have very little uptake among farmers in at-scale studies (e.g. Conservation Agriculture: <10% partial adoption and <5% full adoption in most locations). This has stimulated new work to understand barriers to adoption.
• For more details, see links in text and SPIA webpage.

Independent Evaluation Arrangement (IEA)

Highlights in 2017 included (see Annex G for full list of evaluations conducted):

• Recommendations of an evaluation on gender called for a common CGIAR System-level strategy and action plan for gender equity; greater consistency in gender research; and stronger monitoring and evaluation of outputs and outcomes. To date, this has resulted in agreement^iv to increase human and financial resources for integrating gender issues in research, and to incorporate gender more systematically into results reporting and performance management.
• Recommendations of an evaluation on results-based management called for a shared conceptual understanding of RBM, reflecting CGIAR’s mandate as a research for development organization. The evaluation informed major changes^v in this area, including investment in CGIAR management information systems and a system-level results dashboard.
• Other evaluations and reviews included CGIAR Intellectual Assets Principles, capacity development, partnerships, the Genebank Platform and ISPC. These are referred to in relevant sections of this report.
• A review of IEA evaluation use found 129 citations of IEA evaluations in Program pre-proposals and proposals for the current CGIAR research programs, including 55 that validated the design choices made and 76 that had informed significant changes in program design.
• An IEA Workshop on Development, Use and Assessment of Theories of Change in CGIAR Research^vi brought together decision-makers from across CGIAR. A key finding was that there had been considerable culture change among researchers in using Theories of Change to think through linkages between CGIAR research and desired impacts, and that they needed to be used much more systematically for monitoring and reporting. This has informed CGIAR Program Management Standards being developed in 2018. The workshop also proposed five key characteristics of a good Theory of Change for agricultural research for development.

CGIAR Shared Services Internal Audit Unit

In advance of 2018 agreed changes to the way to achieve overall coordination of internal audit services that are provided across the CGIAR System and the adoption of a ‘combined assurance’ approach to the management of CGIAR System-wide opportunities and risks, 2017 performance outcomes for CGIAR’s Shared Services Internal Audit Unit³ included:

• Capacity building to strengthen internal controls across CGIAR Research Centers, including: publication of four Good Practice Notes on project management; control self-assessment; risk management and research data management; self-assessment tools on IT general controls, and IT security and managing fraud risk; and a review of CGIAR Research Centers’ common financial health indicators, contributing to the overall efforts to strengthen Center financial stability.
• Delivery of a substantive advisory engagement throughout 2017, providing the CGIAR System, including its Funders and Centers, with overall analysis and strategic guidance on the most appropriate System-wide objectives and ‘Risk Families’ to support the CGIAR System’s adoption in November 2017 of the inaugural CGIAR System Risk Management Framework, a whole of System binding CGIAR System Risk Appetite Statement, and the CGIAR System Risk Management Guidelines.
• Providing support to Center/Regional internal audit teams through the CGIAR Shared Services Professional Practice Unit and through a variety of assurance and advisory services.

Intellectual Assets

Strategic management of intellectual assets by CGIAR Research Centers and their partners is essential for realizing CGIAR’s global access and impact. The CGIAR Principles on the Management of Intellectual Assets^vii (IA Principles) provide guidance to Centers on ways intellectual assets can be used to achieve impact for CGIAR target beneficiaries and further the CGIAR strategy. They seek to achieve a delicate balance between maintaining the founding value of global accessibility of CGIAR research results and proactively achieve targeted impacts through the use of intellectual property rights and licensing.

In 2017, a review undertaken by the CGIAR Independent Evaluation Arrangement at the request of the CGIAR System Organization^viii concluded that the IA Principles were “appropriate and have the potential to amplify the impact of the CGIAR System”.

The review made recommendations to strengthen their application and, work is now being undertaken in response, including supporting capacity development within CGIAR Research Centers.

Also in 2017, CGIAR Research Centers reported a total of three provisional patent applications and two non-provisional patent applications. No plant variety protection applications or registrations were reported. Agreements with the private sector were also reported: 23 Limited Exclusivity Agreements and four Restricted Use Agreements. These were all determined to further CGIAR’s vision and to be consistent with the Principles.

Highlights from 2017 are available for review in the CGIAR System Intellectual Assets Management Report 2017 available here.

Footnotes

¹ SPIA is part of CGIAR’s Independent Science and Partnership Council, ISPC

² Impact studies typically take several years from start to publication, and may be reported at different stages in more than one year. Studies mentioned here were presented, published or synthesized in 2017. SPIA also convenes CGIAR Impact Assessment Focal Points and held an important conference in 2017 (see previous section).

³ The November 2017 System Council adoption of a holistic CGIAR System Risk Management Framework resulted in the introduction from January 2018 of a new CGIAR System Internal Audit Function, and the CGIAR System Internal Audit Support Service as a Center-facing capacity building small team. The 2018 changes ensure a more connected approach to opportunity and risk assurance across the CGIAR System, with more effective linkages to Center-own risk management and internal control frameworks.

References

ⁱ ISPC, “Conference on Impacts of International Agricultural Research,” Brief (Rome, Italy: CGIAR Independent Science & Partnership Council (ISPC), 2017), https://ispc.cgiar.org/sites/default/files/events/ispc_brief_57_conference_impacts_ag_research_0.pdf, p.3.

ⁱⁱ In 2017 these included the Independent Science and Partnership Council (ISPC); the Independent Evaluation Arrangement (IEA) and the CGIAR Shared Service Internal Audit Unit.

ⁱⁱⁱ J.R. Stevenson and P. Vlek, “Adoption of CGIAR Priority Natural Resource Management Practices: Synthesis of Nine New Empirical Studies” (February 26, 2018).

^iv CGIAR, “Management Response to the IEA Evaluation of Gender in Research” (CGIAR, July 26, 2017), http://iea.cgiar.org/wp-content/uploads/2017/08/SMB-MngtResp_Eval-Gender-research-For-IEA.pdf.

^v CGIAR, “Management Response to the IEA Evaluation of Gender in Research” (CGIAR, July 26, 2017), http://iea.cgiar.org/wp-content/uploads/2017/08/SMB-MngtResp_Eval-Gender-research-For-IEA.pdf.

^vi CGIAR-IEA, “IEA Workshop on Development, Use, and Assessment of TOC in CGIAR Research: Report: Rome 12-13 January 2017” (Rome, Italy: Independent Evaluation Arrangement (IEA) of CGIAR, 2017), http://iea.cgiar.org/wp-content/uploads/2017/10/IEA_Report_ToCWorkshop2017.pdf.

^vii CGIAR, “CGIAR Principles on the Management of Intellectual Assets (‘CGIAR IA Principles’)” (CGIAR, 2012), http://hdl.handle.net/10947/4486.

^viii CGIAR-IEA, “Review of CGIAR Principles on the Management of Intellectual Assets” (Rome, Italy: Independent Evaluation Arrangement (IEA) of CGIAR, 2017), http://iea.cgiar.org/wp-content/uploads/2017/02/Final-Report.Principles-of-Intellectual-Assets-Review.Oct-2017.pdf.

Photo by P. Lowe/CIMMYT.