You Are Here:
Toolkit Contents
>>
UN Convention
>>
Assessment framework

Assessment framework


Edited by Martin Gould, Accessibility Expert

 

“Nothing About Us Without Us” is a longstanding principle of inclusion in international law and is used to communicate the idea that no policy should be decided by any representative without the full and direct participation of members the group(s) affected by that policy. This involves groups that are marginalized from political, social, and economic opportunities. The Convention on the Rights of Persons with Disabilities (CRPD) is grounded in that principle. The CRPD was authored with the involvement of people from disabilities across the globe. Its framework and provisions reflect that fact, along with the continuing need to adhere to the principle of inclusion as witnessed by the following examples regarding the involvement of persons with disabilities.

Under Article 4(3), General Obligations: “In the development and implementation of legislation and policies to implement the present Convention, and in other decision-making processes concerning issues relating to persons with disabilities, States Parties shall closely consult with and actively involve persons with disabilities, including children with disabilities, through their representative organizations.” Under Article 33(3), National Implementation and Monitoring: “Civil society, in particular persons with disabilities and their representative organizations, shall be involved and participate fully in the monitoring process.”

States which ratified the convention have to report to the UN CRPD committee. Persons with disabilities have to be involved and consulted in the official reporting process, and can provide the CRPD committee with their expertise, unique perspective, and alternative views on the advancement of the implementation of the convention.

According to G3ict’s CRPD 2012 ICT Accessibility Progress Report, there is a huge gap between the requirements and expectations of Articles 4(3) and 33(3) and actual implementation by States that have ratified the Convention. G3ict’s 2012 report indicates that while 71 percent of the countries have dispositions to consult persons with disabilities at large, only 16 percent do have a mechanism to involve the DPOs for the designing, implementation and evaluation of laws, policies and programs regarding ICT accessibility.

One of the key resources available to ratifying States Parties to meet the spirit and letter of Article 4(3) and Article 33(3) is through the use of an evaluation tool designed by G3ict that ensures the involvement of persons with disabilities in the evaluation of critical features of the Convention regarding digital accessibility. It should be noted here, though, that this G3ict evaluation tool could be adopted and/or adapted for use by States Parties in their monitoring and evaluation of many, if not most, aspects of the CRPD.

Section Summary
The Assessment Framework was designed as a template to collect data and identify gaps, in order to help countries ascertain their level of ‘compliance’ with the ICT accessibility provisions of the UN CRPD. Such a framework helps the policy makers engage their countries in making better laws, and ensures that member States take ownership of their compliance obligations under the treaty that they have signed. This section describes the steps involved in the self-assessment, and how such an assessment should be conducted and interpreted.

How the G3ict Accessibility Self-Assessment Framework can be useful

What a Self-Assessment Framework is and why a country should conduct one

Who is involved in conducting and participating in the Self-Assessment


How the G3ict Accessibility Self-Assessment Framework was developed


The stages or steps involved in the ICT Accessibility Self-Assessment


How the results of a Self-Assessment can be interpreted


How the results of a Self-Assessment can be used

Opportunities for Improvement – Critical Success Factors

Technical assistance for countries interested in conducting a Self-Assessment

This section of the Toolkit includes the G3ict Self-Assessment Framework based on the United Nation’s Convention on the Rights of Persons with Disabilities (CRPD). This tool, designed as a template to collect data and identify gaps, lists essential generic accessibility dispositions while emphasizing those that deal with ICT matters. The overall purpose of this template is to help countries and their citizens look across all components and facets of their States’ public (and private) operations in an effort to ascertain ‘compliance’ with the ICT accessibility provisions of the CRPD.

Download the G3ict Self-Assessment Framework

How can the G3ict Accessibility Self-Assessment Framework be useful
The template is designed to encourage government policy makers and citizen leaders to engage their countries in striving for laws, policies, programs and practices that when implemented result in more: (a) accessible ICT infrastructure, (b) affordable ICT, and (c) available and effective assistive technology.

What is a Self-Assessment Framework and why should a country conduct one
One fundamental goal of States that have ratified the CRPD is to take ownership of their compliance obligations under the treaty that they have signed. Through the assessment process, ratifying States - as well as States planning to ratify the CRPD - should take the initiative to evaluate their own progress toward domestic conformity with the CRPD’s (ICT) treaty standards. Self-assessment can be the most constructive way to discover problem areas in extant methods of CRPD implementation. The gaps between the reality of the national situation and CRPD requirements should become clear when States scrutinize policy and practice to develop their self-assessment reports. After all, the practical effects of CRPD obligations depend on State actors as all roads lead back to State responsibility.

Self-assessment results and reports can be used to mobilize concerned actors within States to work together to promote the CRPD agenda, especially if various governmental agencies, disabled persons organizations (DPOs) and nongovernmental organizations (NGOs) contribute to the assessment and reporting process. When working toward improved compliance with treaty provisions, concerned organizations should be involved in investigating and monitoring domestic situations. Self-assessment may facilitate advocacy and needed improvement on many levels by encouraging cooperation among concerned actors within States.

Who is involved in conducting and participating in the Self-Assessment
In collecting data for the Self-Assessment, a State may wish to rely on in-country local assessment teams to complete a formal questionnaire review, in most cases with the assistance of a local lawyer or expert with a mastery of the country’s laws, or preferably someone with experience working on issues involving persons with disabilities  -- e.g., representative DPO leaders.

The Self-Assessment team should be expected to justify its answers to specific questions during the completion of the questionnaire. Without a proper justification, the assignment of a score is largely meaningless. In some cases, for example, it will be clear from a simple reading of the excerpted language of the law/legislation/policy that a country’s compliance with a particular article is poor or exemplary. But this may not always be the case, and in any event, it cannot be assumed that, based on the text alone, compliance (or non-compliance) will be apparent to a reader with little or no familiarity with a particular country or to someone from a country with a different legal tradition. 

How was the G3ict Accessibility Self-Assessment Framework developed
There were three general steps used in the methodological approach relied on for the development of the Self-Assessment:
1 - The G3ict Research Committee reviewed the CRPD to identify all provisions that included the terms: communications, technology, information or information services, accommodation, and access, accessible, and accessibility since Article 9 includes ICTs in its definition of accessibility. Once identified, the Committee created an exhaustive listing which included these provisions redrafted as “self-assessment” items (N=50 items out of which it selected 35 items as variable components for the Index) and which also called for an evidentiary justification for the score given for every item. This has been referred to as the Leg #1 self-assessment tool (of a country’s commitments to the CRPD).

2 - Next, the Committee then created a 2nd measurement scoring tool (N=11 items), which it perceived to be directly related to the ICT provisions of the CRPD identified for the Leg #1 assessment, and which the Committee perceived to represent the basic capacity of a country to implement the ICT provisions of the CRPD. This has been referred to as the Leg # 2 self-assessment tool.

3 - Finally, the Committee created a 3rd measurement scoring tool (N=10 items), which it perceived to represent the systemic and/or individual impact(s) of a country’s fulfillment of the ICT provisions of the CRPD. This has been referred to as the Leg #3 self-assessment tool.

The stages or steps involved in the G3ict Accessibility Self-Assessment

The basic activity for the Self-Assessment Framework includes the following tasks:

  • Identify the country commitments: This activity requires identifying the political commitments made with respect to: the national laws, policies, programs and plans of action that are relevant to the ICT provisions under analysis; and, the formal status of the country’s government legal and policy regime in relation to those ICT commitments.  [Leg #1]

  • Identify the capacity/infrastructure for implementation: This involves examining the country’s capacity to implement the ICT provisions under analysis, including the: digital/technology resources available, financial resources available, the human resources available and other factors – such as business,  social, and cultural – that may limit or expand implementation capacity. [Leg #2]

  • Assess the country’s implementation and impact: This requires the development and application of institutional measures to ensure that legal and policy changes are implemented in actual practice. In particular, it looks at the (a) availability, accessibility, and affordability of ICTs and assistive technologies (ATs), (b) availability, accessibility and quality of information and information services, and (c) impact of ‘a’ and ‘b’ on the lives of persons with disabilities. [Leg #3]

  • Draw links between commitment and implementation/impact: This activity involves comparing the country’s commitments to the CRPD with the actual implementation and impact found by the self-assessment. The purpose of linking the implementation and impacts to specific legal and policy obligations is to identify the results which the country should focus on. This also involves linking the country’s capacity to implement the CRPD obligations and identifying the main obstacles the country will have in meeting those obligations. What CRPD commitments have not been achieved by the country? What capacity factors are related to those unfulfilled gaps? 

  • Generate recommendations and the action plan: This activity involves using the results of the above analysis to work with multiple stakeholders on developing proposals for legal, policy and program changes. It involves generating strategies and recommendations for preparing a plan of action to work with legislators, regulators and civil society for improvement of its public laws, policies and programs as well as for necessary private sector changes.

How the results of a Self-Assessment can be interpreted
An in-country Self-assessment team ensures a good deal of objectivity in the results by its reliance on an evidence-base to justify answers to the Self-Assessment Framework’s questions. For each CRPD ICT provision and related assessment question presented, the team may only score an answer of “Yes” if it can also provide a justifying source document as evidence of the State’s fulfillment of a CRPD-mandated commitment. Once all Self-Assessment questions have been duly scored by the in-country assessment team, it is then time for the team to meet with a designated State consensus planning group to discuss and interpret the results.

States that have conducted the Self-Assessment for the first time should consider the scores as baseline results, and use a consensus group with key stakeholders to determine how specifically the findings need to be used.

How the results of a Self-Assessment can be used
The ultimate focus of the Self-Assessment is to identify improvement areas that will be developed into further action plans. It is important to keep in mind that the assessment itself only gives a map of where the State stands right now in relation to the CRPD, driving the actions will be the next step. Integrating the results from Self-Assessment into public (and private) sector planning is a prerequisite to make the efforts produce results.

Members of the in-country assessment team and key State leaders and stakeholders should be gathered to learn of, and then discuss, the results from the Self-Assessment and agree on the strengths and areas for improvements of the State. There are three main objectives for this consensus meeting, namely, to: 1) ensure that everyone views the State’s Self-Assessment results in an objective and holistic perspective, 2) agree on strengths and areas for improvement and 3) prioritize areas for improvement that will be taken into further action planning.

Opportunities for Improvement – Critical Success Factors
In its 2012 report, G3ict identified a number of areas which are ripe for implementation improvement and which directly affect States Parties obligations. One critical success factor involves the need for State Parties to put into place and practice a systematic mechanism to involve the DPOs (persons with disabilities) working in the field of digital access to the drafting, designing, implementation and evaluation of laws and policies. This is in keeping with Article 30-33 of the CRPD. The priority, then is for national and international organizations to focus on building capacity for implementation especially support and expertise for DPOs in the monitoring process.

Technical assistance for countries interested in conducting a Self-Assessment
Should countries be interested to discuss how to best implement this Self-Assessment Framework, contact program@g3ict.org.

Download the G3ict Self-Assessment Framework.