Assessing OGP Action Plans

The Open Government Partnership (OGP) Action Plans are in (!), although not all 55 OGP member countries have submitted theirs, and only some plans were submitted in the electronic format we had hoped for. But with more than 40 plans to analyze, we took the first deep dive into finally making some sense of this whole process.

By way of brief background: member countries of the OGP are asked to deliver Action Plans with commitments towards making their governments more transparent, accountable and participatory (the April 2012 high-level event in Brasilia marked the deadline for non-founding countries to submit their initial Action Plans). The process allows for a great deal of flexibility in defining commitments; however, all commitments are expected to be actionable, time-bound and benchmarked, so we used this as the basis for our analysis. Action Plans are also intended to be “living documents” that are constantly updated and strengthened, year after year, through consultations with domestic civil society; for this reason, our conclusions that follow can be used to inform some thinking around steps.

Our Methodology

By adapting the SMART framework for the purposes of our analysis, we examined each individual country plan to get a better idea of where countries stand in terms of shaping their open government agendas.  Specifically, we evaluated whether a country’s commitments are 1) Specific 2) Measurable 3) Actionable 4) Relevant and 5) Time-bound. (See results attached)

We deemed plans specific when the commitments that comprise them delineate what the country is planning to do.  In other words, plans are specific when countries have articulated their commitments through concrete activities used to reach these goals.

Plans are measurable when the activities that comprise them are benchmarked. Benchmarks – metrics by which action can be measured - are crucial for the next big push and forthcoming review phase of the OGP.

Actionable plans are those that explain how the country will achieve the outputs they set forth. For example, plans need to include the approach, tool, process etc. by which they expect to meet their goals. (It is important to note that for the purposes of an objective analysis, we focus on goals as outputs and not outcomes here – a point that we will revisit later.)

We consider plans relevant when they focus on addressing open government issues as opposed to broader good governance reforms.  For example, take the commitment: “To develop an online Control Panel to monitor the Ministry of Education’s actions” and contrast it with “To improve the quality of education by providing technical and financial resources to schools.” While the latter may be a much-needed step towards improving education, it doesn’t directly address transparency in the way the former does.

Lastly, if the country states projected deadlines for outputs, we classify the plan as time-bound.

Outcomes

Overall, our assessment shows signs of some real reasons to be optimistic - nearly 70% of the submitted Action Plans meet at least four out of the five SMART criteria. Only a handful of the total 42 plans fulfilled two or less of the criteria.

The biggest gap was in benchmarking - a little less than half of the countries outlined metrics for assessing their progress. Slightly better than benchmarking was time-bound commitments – 40% (around 20 countries) have not yet provided a timeline for their activities.

Tracking with the number of overall plans that could be improved, just more than 15% include commitments that are outside of the scope of what we consider to be “open government.”  Around the same number of countries have yet to articulate how they plan to execute their activities.

Lessons Learned

1. Expected and Unexpected Star Performers

Unsurprisingly, countries that have been on the open government radar for some time now including Brazil, Canada, Israel, Croatia and Moldova delivered some of the strongest plans.  However, OGP countries including Jordan and the Dominican Republic faired similarly. For us, this is an exciting discovery because it keeps us optimistic about the prospects for OGP. This means that there are key member countries in all regions of the world that are vocal advocates for open government.

2. “Commitment” is Interpreted in Different Ways

Some countries understand OGP commitments to be grand statements, similar to the OGP framing Grand Challenges: “Increase Public Integrity.”  Others take a slightly more granular approach – for instance, “E-reforms for improving the delivery of public services.”  And still others provide further details by committing to specific actions, for example, “ Releasing public education data – number of students, teachers, facilities etc. for public monitoring.”

For our purposes, the more granular the information a country provides, the better. We decided to base our assessments of the Action Plans on each country’s promised “activities” rather than their Action Plan’s rhetoric – that is, those items that are as measurable, actionable, and specific as possible. Not all countries have fleshed out their activities, and this is something that the Steering Committee will need to encourage more of as we move forward.

3. “Open Government” is Still Not a Term that is Easily or Universally Understood (and for good reason; see this paper and this blog post for reasons why.)

Based on our analysis, a sizeable number of Action Plans included at least one commitment that we classified as being outside of the scope of “open government.”

Take, for example, this commitment: “To develop a gender equality program together with all municipalities.”  We commend this effort and do not downplay the importance of it; however, this is a clear example of conflating democratic rights with activities that specifically target transparency, accountability and/or participation. Importantly, this reminds us that some amount of consensus around a meaningful definition of “open government” is required sooner rather than later for OGP to avoid becoming a dumping ground of random public sector reform efforts.

Next Steps

What You Can Do

We hope that our analysis can be a jumping off point for further work that can lead to a more nuanced understanding of where countries stand.  To encourage this, we want to offer three points of consideration for readers, particularly those involved in their own country assessments (both domestic civil society and government):

1. Consider evaluating to what extent plans contain a mix of targeted goal areas (i.e. commitments not limited to just one issue such as budget transparency or open data).  For example, the UK and Indonesian plans are almost entirely focused on the publication of data. Many of their commitments read something like this: “Promoting transparency, accountability and public participation in the area of government subsidies for education through the publication of budget allocation, disbursement, and expenditure data.” The question is: should these countries be encouraged to expand their focus? On the one hand, opening datasets for all to see and examine is no doubt a step towards a more transparent form of government. On the other hand, as we have expressed before and continue to monitor, seeking “quick wins” by committing to low hanging fruit (such as open data) may be a form of circumventing the tougher, perhaps more impactful open government reforms.
2. Consider assessing whether country plans include a reasonable amount of “stretch” commitments (i.e. they are not simply activities that the country is currently engaged in and/or as a whole show a limited amount of ambition). For example, countries engaging in certain open government reforms may have the capacity do more. Another way to look at this is to read through your country’s action plan and assess if they are selling what they have already done or if they are striving for new achievements?
3. Tied to the idea of “stretching” plans and going beyond low hanging fruit, is the concept of evaluating plans based on outcomes versus outputs for improving plans with a view to generate greater impact. For example, consider if metrics designed for evaluating progress should, for example, put less weight on “How many datasets were released?” and more emphasis on “How was published information used to enforce the law?”

What OGP Will Do

Identifying plans and specific commitments that haven’t quite hit the mark is a step in the right direction. The idea now is to use these insights to work with countries towards improving their Action Plans through greater specificity and ensuring their goals can be met in the short- and medium-term. The forthcoming OGP Independent Review Mechanism (IRM), which will be an independent body tracking the progress of each country’s Action Plan, will be central to this process.

This also means that OGP Networking Mechanism suppliers should get ready! We hope to reach out to those of you who are on the Networking Mechanism roster to brainstorm around specific country engagements in the near future. We anticipate dozens of opportunities to work directly with OGP governments on strengthening and implementing their new Action Plans in the coming months.

-- Nicole Anand

-- Image Credit: Grant Kwok