What is policy impact? Well, it depends from what we mean by policy change

Photo by Linus Nylund on Unsplash

During the last few weeks I have been working on the  design of an adaptive monitoring and learning framework to assess contribution to policy change on climate adaptation in few countries.  Working on the framework made me re-read some articles and papers about policy change which I had read some years ago. Below the key points I used for the development of the framework.


\’Rather than seeing policy as one single, discrete decision, it is important to broaden one’s view, so that policy is understood as a series of documents and decisions that are best described as a set of processes, activities or actions (Neilson, 2001).[1] If we accept this interpretation, it follows that throughout the policy decision process there are different changes that can be identified as the objective of a policy influence strategy or activity.

Jones and Villar (2008)[2] draw on the study that Keck and Sikkink[3] conducted in 1998 on transnational advocacy and the policy process, to describe different types of policy change or policy impact:

  • Framing the debate and getting issues on to the political agenda: this is about producing and communication evidence that draws the attention of policy makers, public opinion, and the media to new issues/solutions. It can result also in policy makers acquiring new knowledge about a specific policy problem. For example, in 2010 the World Bank published a two years study on governance institutions in Vietnam (Vietnam Development Report 2010: Vietnam development report 2010: Modern Institutions) which received considerable attention from media and prompted responses and comments from government officials on the issue of upward/downward accountability and what worked and what did not work well in the policies of devolution pursued by the government.
  • Attitudinal change is about drawing attention to changes in the way key policy stakeholders think about evidence-informed policymaking. For example, the Prime Minister’s Office expresses interest in setting up a policy analysis unit to coordinate the evidence strategy of the different ministries. A Ministry of Planning draws plans to establish a data analytics unit to test ways to use data analytics/dashboards to monitor forest fires.
  • Policy content change is about actual changes in legislation/regulations. For example, in Indonesia after four years of work involving academics, think tanks, the Ministry of Higher Education, the National procurement Agency, the Office of the President, etc., President Jokowi has signed on 15. March 2018 and new Presidential Decree (Perpres – No. 16/2018) that allows Indonesian non-government organisations to be procured by government organisations to conduct studies and policy research. The policy influence team worked and advised on the actual text of the Perpres.
  • Behavioural changes refer to changes in the way of working of government institutions with regard to the demand and use of evidence or the change in behaviour that a new legislation produces. This is the most comprehensive change, which requires more time to achieve and is more difficult to prove. For example, the Prime Minister’s Office in Finland in 2017 allocated 10ml Euro for 42 research topics requested by ministries. The research topics were defined by working groups involving policy makers, academics, and members of civil society organisations. In the Philippines, the Asia Foundation facilitated the process of establishing a coalition involving lawyers, land titling experts, NGOs, and researchers to amend the land titling legislation. President Arroyo signed the new legislation in March 2010. As a result, 50-60.000 land titles are being issued every year (up from ca. 10.000 per year) which has enabled families in urban disadvantaged areas to back up loans requests to banks and to start businesses.[4]

[1] Nielson, S. (2001) ‘Knowledge Utilization and Public Policy Processes: A Literature Review’, IDRC Evaluation Unit. Available at: https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/31356/117145.pdf?sequence=1

[2] Jones, N. with Villar, E. (2008) ‘Situating children in international development policy: challenges involved in successful evidence-informed policy influencing’ in Evidence and Policy, vol4, no.1: p53-73.

[3] Keck, M. and Sikkink, K. (1998) Activists beyond borders: advocacy networks in international politics. Ithaca, NY: Cornell University Press

[4] Watch the video at https://www.odi.org/opinion/9209-adapting-development-how-local-reformers-revolutionised-land-rights-philippines

1 thought on “What is policy impact? Well, it depends from what we mean by policy change”

  1. Very interesting issue. The study by Nielson was part of a much larger IDRC activity that generated a large number of interesting outputs. One of these was the book by Fred Carden “Knowledge to Policy: Making the Most of Development Research” which is available in print and downloadable from the IDRC website. This reports on a large number of case studies of IDRC policy influencing projects using a common framework.

Leave a Comment

Your email address will not be published. Required fields are marked *

3 − two =