Tracking Progress Towards Post-2015 Targets: Roundtable Consultation December 2, 2014 Hosted by the Center for Universal Education at the ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Tracking Progress Towards Post-2015 Targets: Roundtable Consultation
December 2, 2014
Hosted by the Center for Universal Education at the Brookings Institution
I. Background
The Education For All Steering Committee established a Technical Advisory Group (TAG) to develop post-
2015 education indicators, based on the targets proposed in the Muscat Agreement of May 2014 and
the Open Working Group of July 2014. These indicators, organized in the Towards Indicators for a Post-
2015 Education Framework, focus on enhancing learning outcomes and achieving equity across a range
of goals, including early childhood, primary and secondary education, skills, equity, literacy and
numeracy, and global citizenship. For each goal, the document maps available and potential indicators
using two criteria: alignment with concept and global comparability. For each target and concept, a
matrix with indicators, comments on criteria, and traffic light signs are used to measure availability of
data to date. The focus of the consultation is on how well the indicators align with the concepts to be
measured and how realistic it is to expect countries to collect data on the indicators.
The first two drafts of Towards Indicators for a Post-2015 Education Framework have been submitted to
the Open Working Group in New York. Current consultations, from November 17, 2014 until January 30,
2015, will address four key questions:
1. For each target, does the report identify the best indicators that are most aligned with the
concept and are already being tracked in a large number of countries?
2. What new indicators could be developed to be more closely aligned with the proposed targets
and have the potential to be globally comparable?
3. For each target, please identify or propose the two most important indicators.
4. Are there key issues that the document has not addressed in a satisfactory way or other issues
that also need to be taken into consideration?
The results of the consultation will feed into a revised proposal to be finalized by March 2015. The final
document will then be submitted to the World Education Forum in Seoul, Korea in May 2015.
II. Consultation Overview: “Tracking Progress Towards Post-2015 Targets” Roundtable in Washington,
D.C.
The “Tracking Progress Towards Post-2015 Targets” roundtable consultation took place on December 2,
2014 at the Brookings Institution in Washington, D.C. This consultation brought together more than 20
participants from Washington, DC-based government agencies, teacher’s organizations, NGOs, and think
tanks, along with several international colleagues to provide feedback to the Education For All Steering
Committee on how to assess progress toward targets aimed at enhancing learning outcomes and
achieving equity, with a focus on two specific educational levels: early childhood and the early grades.
The meeting objectives were the following:
1. Convene key partners in Washington, D.C. and provide an opportunity for information sharing
and dialogue on the EFA Technical Advisory Group (TAG)’s proposed post-2015 education
indicators for early childhood development and early grades learning;
2. Provide feedback on the TAG’s proposed indicators; and
13. Learn about progress to-date in equity in learning, early childhood development, and early
grades learning from the Center for Universal Education, DfID, UNESCO Institute for Statistics,
Save the Children, and the World Bank.
The consultation reviewed the overall proposal and then focused on the recommendations in sections
5.1 (Early Childhood) and the “Achievement of relevant learning outcomes” portion of 5.2 (primary and
secondary education) in the Towards Indicators for a Post-2015 Education Framework.
The recommendations will need to be synthesized and effectively communicated to politicians and
policy makers who aren't education specialists to strengthen their ability to negotiate and develop
sound education policies, both for the UN Post-2015 process and at the national levels. It was noted
that in 1990 when the Education for All goals were established, data on basic indicators such as
enrolment were not yet globally comparable. But, in response to the global framework for action, these
indicators improved over time.
Common themes throughout the discussions included:
The extensive indicator framework could be strengthened by selecting the most relevant and
accurate indicators. Instead of being a comprehensive list, a small set of indicators should be
signals for policy interventions by government officials and political leverage points for future
stakeholders.
The bar for global comparability has been set too high. It may not be feasible or appropriate to
measure education, especially learning outcomes, in a globally comparable way in every
country. It is important for countries to be able to find a balance between what is measured
globally and what is measured nationally. The international community should not impose
measures on any particular country.
The proposal does not address out of school children and children not in the system. A
Learning for All indicator which combines access and learning data could address this
challenge.
The target language does not mention competencies at the early grades. Some children do not
stay in school beyond grade 2 or 3, and measurement can be used as an early warning for
later academic achievement.
The framework could be strengthened by acknowledging previous attempts to build
consensus on education and learning outcomes and citing sources where available.
III. Reflections from UK Consultation
First, participants received a report out from a recent consultation hosted by the U.K. Department for
International Development (DFID) in London on November 17, 2014. This meeting, co-hosted by DFID,
the Overseas Development Institute, the U.K. Education Development Forum (UKFIET), and the
University of Cambridge, involved 40 participants. The consultation participants discussed the need to
empathize equity in the goals, targets and indicators which is important in all countries and helps
improve the relevance for high-income countries. Some of the consultation’s common themes included
the following: targets and indicators need to be clearly defined; indicators must be technically robust
and supportive of accountability at all levels; measurements need short, medium, and long term
perspectives; and data collection agencies require improved cooperation. Furthermore, the U.K.
Roundtable in London outlined three key next steps:
2 Promote increased participation in the Technical Advisory Group (TAG) consultation process and
call for a prioritized set of indicators from the TAG prior to the February Education For All
Steering Committee meeting;
Support depth and breadth of technical inputs in learning indicator consultations and the
common metric proposal; and
Encourage partners to host similar consultations focused on areas of comparative advantage
across the target areas.
IV. Discussion on Post-2015 Learning and Equity Indicators
To help frame the discussion, Save the Children presented on their proposed approach to post-2015
learning and equity indicators. Below is an overview of the presentation:
The United Nations General Assembly Open Working Group (UNGA OWG) has focused on the issue of
inequality in its discussions. UNGA OWG’s final recommendations include the need to address
inequalities within and between nations. Moving forward, the post-2015 framework aims to follow
concrete steps to achieve equity by tracking and monitoring which groups are being left behind and
incentivizing inclusive progress.
The main challenges in measuring equity in learning include the following: disaggregation of
administrative data is limited, household survey samples are too small to provide estimates for
vulnerable groups (except for sex, urban/rural, and wealth), and student assessment surveys exclude
children not in school.
Save the Children recently released a paper focusing on stepping stone targets, which are essentially
benchmarks set for interim dates between 2015 and 2030 to ensure that disadvantaged groups are on
track to achieve the 2030 equity goals. The stepping stones can be defined at the national level through
an open, inclusive and participatory process, including organizations representing marginalized groups.
In doing so, this would ensure that marginalized groups are no longer excluded and that dialogue occurs
within societies, thereby enabling discussion on how society tolerates inequalities to raise awareness.
Save the Children’s report, titled Framework for the Future, includes additional measures for promoting
equity across the entire framework. For example, “zero” goals are an important part of the process of
tackling inequality within the post-2015 framework. Under zero goals, it is unacceptable for even one
child to be left behind. Other mechanisms that proposed to be incorporated into the new framework
include:
Indicators to track gaps between rates of progress between the poorest 40% and richest 10% for
all target areas, including income inequality.
A commitment to consider no target met unless it is met for every social and economic group.
All progress monitoring data must be disaggregated by age, disability, ethnicity, gender, region,
urban/rural location, and other social groupings.
A strong focus in the framework on accountability and ensuring that people have the voice and
power they need to monitor progress, influence decisions affecting their lives, and speak out
when development processes are unsuccessful.
Ultimately, Save the Children proposes incorporating a commitment into the new framework that by
2030, no target will be considered met unless met for all groups. However, during the predicted 15-year
time span of the new framework, political changes may prevent support for marginalized groups.
Therefore, a concrete mechanism is required to place immediate and continuous attention on
disadvantaged groups to ensure their needs are met first. This mechanism must ensure that progress is
3maintained for all, all groups meet 2030 targets, and gaps close between more and less advantaged
groups.
A. Early Childhood Development Indicators
The Center for Universal Education (CUE) at the Brookings Institution led a session on early childhood
indicators. Below is an overview of the presentation:
In early childhood, the key concepts to measure as outlined in the Towards Indicators for a Post-2015
Education Framework are: (1) the percentage of children ready for primary school, defined by the
achievement of age-appropriate learning and development across all domains starting at birth; (2)
participation in early childhood development, care, and pre-primary education, including access to both
formal and informal early childhood development programs from birth to the beginning of formal
schooling; (3) quality of care and education received by young children, including quality of early
childhood development, care, and pre-primary education programs and quality of home experiences;
and (4) participation in at least one year of free and compulsory pre-primary education.
CUE outlined its reflections on these key concepts based on the LMTF process and consultations with
teachers, civil society members, and other stakeholders. First, it would be important to specify the
domains of learning and development that could be measured globally, instead of “across all domains”
which leaves this open to wide interpretation. Multiple domains that go beyond the basics of literacy
and numeracy and that encompass social emotional learning as well as health and nutrition should be
included. Through recent work with the Learning Metrics Task Force Learning Champions, there is
indication that countries are paying more attention to ECD and learning, but continue to emphasize
literacy and numeracy in the primary grades.
While early childhood development begins at conception, the LMTF settled upon school entry as a key
educational milestone to be tracked globally despite different ages and measures found at the national
level. School entry data can be useful to both inform improvements in ECD services and inform teaching
and learning in primary school. In addition, it is critical to measure quality of care in education as it is
difficult to achieve large-scale benefits without the provision of early childhood services that are of a
certain standard of quality. A large-scale effort is needed to obtain usable data not just at a global level
but also for countries that are investing in data collection. The data therefore needs to be reflective of
national priorities. The Towards Indicators for a Post-2015 Education Framework document captures
these issues well.
Currently the proposal includes an indicator on achieving universal pre-primary education. There is a
need to discuss whether the emphasis should be on promoting universal pre-primary education versus
targeting the most vulnerable children. Additionally, the role of private ECD providers needs to be
included in the proposal including how they can be involved in ensuring free education or subsidies for
countries with low or nonexistent public funding.
There are several existing and emerging cross-national approaches to child development and learning
metrics for ages 0-8, including the WHO 0-3 Measurement framework, the East Asia Pacifica Early Child
Development Scales (EAP-ECDS) and Save the Children’s IDELA Tool. In low- and middle-income (LAMI)
countries, the SABER-Early Childhood Development (SABER-ECD) analyzes data to assess each
participating country’s progress towards three key policy goals: (1) establishment of an enabling
environment, (2) wide implementation, and (3) quality monitoring and assurance. The typical standards
set for these structural variables include teacher-child ratios, square meters per child, and operating
hours. In some Latin American and Caribbean countries, teacher training and qualifications, family and
4community engagement, and health, safety, and nutrition are additionally monitored. However,
curriculum, instruction, and interactions are infrequently monitored.
Between now and mid-2015, the United Nations Educational, Scientific and Cultural Organization
(UNESCO), the United Nations International Children’s Emergency Fund (UNICEF), and the World Bank,
and CUE in collaboration with national and international technical partners, will develop a set of open-
source, freely available population-based measures of (1) child development and learning and (2) quality
of early learning environments. The consortium will assist governments in taking the measures to scale,
using the data effectively, and developing a comprehensive set of tools.
i. Consultation Discussion
Following the presentation, participants were divided into discussion groups focused on the following
topics:
1. Key Concept 1: Readiness for Primary School
2. Key Concepts 2 and 4: Participation1
3. Key Concept 3: Quality2
Each group was tasked with responding to the following guiding questions:
Does this group of indicators accurately measure the construct?
Are they feasible in all countries, given sufficient resources?
Are they actionable?
Are there any unintended consequences to measuring these indicators?
1. Comments from Participants on Readiness for Primary School Indicators:
The bar for global comparability has been set too high. It may not be feasible or appropriate at
this level. It is important for countries to be able to find a balance between what is measured
globally and what is measured nationally. The international community should not impose
measures on any particular country.
The Early Childhood Development Index indicator is a measurement tool included in MICS
rather than an indicator. Instead of listing the ECDI this could be changed to “readiness for
school as measured by a diagnostic measurement tool of the country’s choosing.”
Need to take into consideration the potential consequences associated with using the words
ready/readiness as it is often associated with entry exams and we don’t want people to
misuse the data for this purpose.
Need to clarify what “readiness for school” means and how this data can and should be used.
There should be more deliberate links between early childhood and health indicators. Could
there be a weighted index? More deliberate links between early childhood and health
indicators could have a positive impact on outcomes and the sustainability of interventions in
both public health and public education. The relationship between early healthcare and early
childhood education can be made more explicit in the description of/rationale provided for
indicators.
2. Comments from Participants on Participation Indicators:
There is too much focus on provision. If countries are forced into developing new pre-primary
programming that does not already exist, it could potentially be problematic especially in
1
In early childhood development, this refers to participation in pre-primary education and one year of free and compulsory
pre-primary education.
2
In early childhood development, this refers to the quality of early childhood development care and pre-primary education.
5developing countries and/ places where early childhood opportunities are provided by the
private sector. The indicator as it currently exists places too much emphasis on outcome and
not on the process to achieve the target.
If the focus is just on outputs and attending pre-primary education does not yield positive
results, countries will not be able to determine the reason, therefore making it important to
measure access and participation as well as aspects of an enabling environment (i.e. Is the
parental environment being supported? Is some kind of early childhood education provision
guaranteed?).
There is concern over the fact that equity of participation is not addressed in the proposal,
including within the early childhood indicators. While there are different ways to measure
equity as related to the outcome, it is not enough to look at the average outcome. It is
important to measure and reduce the gap between most and least advantaged groups.
At the same time it is important to consider the potential for stagnating the progress of most
advantaged groups while trying to reduce the gap between the most and least advantaged
groups. The idea of a third measure to track the progress of the least advantage group may
mitigate this situation.
Measures that apply to out-of-school children for pre-primary ages as well as other potential
barriers, such as limited access due to the presence of a disability are not mentioned in the
proposal. There should be a measure that looks at the variance of achievement for
participation in early childhood education.
Overall, there is not enough emphasis on early childhood in the post 2015 agenda.
Might be good to include some aggregate measures and potentially build in a longitudinal
study that looks at the same children over time.
There are too many indicators. Need to choose the most relevant and accurate.
3. Comments from Participants on Quality Indicators:
The proposed indicators (pupil-teacher ratio and the completion of pre-primary) are not
sufficient to measure pre-primary quality. Pupil per trained teacher could be a possible
alternative. However, this is complicated because of the need to then define “trained” across
countries and different contexts.
Teacher-pupil ratios are hard to disaggregate and may be inflated representations of how
many children actually spend time with a teacher each day. It might be more accurate to use a
classroom-pupil ratio instead, but participants noted that neither are a sufficient indicator for
quality.
Participants discussed that one way to assess quality in the classroom is through observations
of the environment and interactions. In the absence of that, basic indicators of facilities can be
proxies (i.e. pupil-toilet ratios).
While text book-pupil ratio is not relevant in pre-primary, some indicator of learning materials
could be used to try to understand what is available to children, e.g., literacy materials per
child ratios, which could include books, posters, labels, etc.
Another idea for measuring quality includes structural indicators on a national level standards
and regulations around school safety and health.
Discussion
Overall the group felt that it is important to identify what is developmentally appropriate to measure for
children and to draw lessons from other sectors including health. Several participants felt that it is
6necessary to review the use of the phrase “readiness for primary school” and the potential negative
impact. The group felt it would be useful to identify a word or phrase that reflects the needs of a child
that would not unintentionally classify some children as “ready for school” vs. “not ready for school.”
Participants felt that a lot of the discussion and emphasis in the proposal included measures and
enrollment around center based care when in fact the majority of early childhood care in developing
countries is home-based with parents, grandparents and/or older siblings providing care. As such, it is
important to measure the outcomes of those children as well. The home environment, materials,
interactions, and parental perceptions and questions about services receiving could be added. Citizen-
led assessments (ASER, Uwezo) could be a way to get data without direct observations.
It is important to take into account the role private providers play in early childhood development and
the added pressure requiring all countries to pay for a certain amount of provision would bring. Private
providers for ECD may or may not be subsidized, and parents often have to pay fees. It is important to
identify a minimum amount of ECD that should be mandated as compulsory and calculate the burden
that would place on countries.
Instead of focusing on universal free pre-primary education, participants suggested that the focus for
countries should be on ensuring that the most vulnerable children have access to high-quality, free
provision. Governments should work to ensure all children have access to quality early care and learning
opportunities, and private providers should not be dissuaded to provide early childhood services as well.
B. Early Grade Learning Indicators
The World Bank led a session on early grade learning indicators. Below is an overview of the
presentation:
The key concepts to be measured for early grade learning as outlined in the Towards Indicators for a
Post-2015 Education Framework are: (1) achievement of relevant and effective learning outcomes for
primary, lower and upper secondary education; (2) access to and participation in primary, lower and
upper secondary education; (3) completion of primary, lower and upper secondary education of at least
nine years’ duration; (4) quality of primary and secondary education; and (4) guarantee of free and
compulsory education of at least nine years’ duration.
The World Bank reflected on the alignment of the indicators with the concepts of the targets and
whether there is a need for global tracking of reading and math indicators at the lower grade levels,
particularly in grade 2. The indicators cover minimum proficiency standards that are resonate with
content and performance standards but not necessarily what is required to succeed in school and
beyond. The report also does not reflect the diversity and degree of misalignment that is in all of the
existing data sets and the implications of creating a baseline. The indicators are going to require better
collection and use of data as the priority focus shifts to national systems and national efforts to collect
data.
The World Bank’s education sector strategy up to 2020 is to invest early, smartly and learning for all
with funding focused at the national level.
i. Consultation Discussion
7Participants decided to discuss early grade learning indicators together with end-of-primary learning
outcomes and primary participation. Following the presentation, participants were divided into
discussion groups focused on the following topics:
1. Achievement of Relevant Learning Outcomes
2. Completion of Primary and Secondary Education
Each group was tasked with responding to the following guiding questions:
Does this group of indicators accurately measure the construct?
Are they feasible in all countries, given sufficient resources?
Are they actionable?
Are there any unintended consequences to measuring these indicators?
1. Comments from Participants on Achievement of Relevant Learning Outcomes Indicators:
Measuring learning in secondary education at this point may not be feasible since this level of education
is not compulsory in most countries. The population of students who are completing secondary school is
often not representative of the population that begins school due to multiple factors including drop-out;
therefore, secondary school completers in developing countries often represent groups that have been
advantaged in some way.
There were questions on minimum proficiency standards, particularly why the word
“competency” was used.
There is no mention in target language of competencies at the early grades. This needs to be
measured as some children don’t make it beyond grade 2 or 3 and can also be used as an
early warning for later academic achievement. After 10 years or so of schooling it may be too
late to address basic quality issues in an educational system for the individuals affected and
those who are not learning.
The distinction between learning to read and reading to learn as a critical threshold point
needs to be more explicit in the document. Reading is made up of a continuum of skills that
can be reflected as parallel to a developmental process. Reading to learn is at the high end of
that continuum.
It would be helpful to create a diagram of how targets and indicators are interconnected,
rather than providing a laundry list. This diagram could serve as a conceptual map or
framework and be included up front.
Out of school children/children not in the system are still missing from the proposal.
Encourage increased government-run household based assessments to include children
outside of the system.
Indicators should be signals for policy interventions by government officials and political
leverage points for future stakeholders.
What is going to be feasible and comparable -globally comparable measures vs. national
assessments?
Need a broader definition of learning as currently only looking at reading and numeracy.
Proposed expansion of indicators from literacy and numeracy to other factors.
Assessments, national benchmarks, and stepping stones must drive development and school
improvement.
8 The use of national benchmarks and stepping stones should be developed by countries and
be specific to the structures, needs and resources in each country while aiming for optimal
growth.
The language in the report seems to imply that data will be collected at a global level not a
national level
Need to be aware of high stake assessments for global monitoring and the unintended
consequences, like with high school entry exams. Need to allow countries to use their own
system however if they use high stake exams it might change the measure.
2. Comments from Participants on Completion of Primary and Secondary Education:
Reduce the redundancy between the participation indicators that focus on children in school
and net enrollment in school as it is the same information presented differently. It is not
clear whether to keep rates or number as a number is often a more powerful statistic at a
global level while rates are more powerful at the national level.
There is a need to consider and include metrics for out-of-school children and identify what
groups are being excluded and why.
Recommended countries adjust their equity focus – for example, if a country is near 100%
equity in schools, it should focus on equity for children out of schools.
Need to reduce the redundancy under completion indicators from two per level down to one
per level.
Equity discussions may not go beyond focusing on sex, wealth, and location – need to focus
on visibility, ethnicity, linguistics, etc. of other disadvantaged groups as well.
Need improved data disaggregation of outcome indicators.
Need to emphasize girls’ education in EFA and Open Working Group proposals. There is a
need to emphasis multiple underserved or traditionally excluded populations, including girls,
people with disabilities, minorities, etc.
Discussion
There was some confusion over whether the group was to react only to the indicators or whether there
was some room to tweak the targets. It was agreed and subsequently confirmed by UNESCO that there
is indeed still room to tweak the targets. There was broad and deep criticism of target two and the fact
that all quality seemed to be reduced into one broad target that also included access. Relatedly,
secondary learning indicators are limiting since they are not compulsory and many students have
dropped out by then.
Overall the group felt that there are too many indicators under the primary and secondary education
section and there is a need to shift the proposal from a highly technical lens to a policy focused
document that it is more accessible. A user friendly version would be helpful for policy makers and those
people who will help negotiate these issues at the UNGA 2015. It is important to look at the indicators
and determine who governments will be accountable to. The indicators and process should be
understood and used by citizens as the citizen use of data will be the driver of progress.
Participants expressed concern that the Learning Metrics Task Force consultations and
recommendations have not been adequately reflected in the proposal including its findings on the lack
of technically robust and globally comparable measures. Also, some participants had attended a Reading
Indicators meeting at UIS in March 2014 and felt that the work accomplished at that meeting was not
incorporated into the Towards Indicators for a Post-2015 Education Framework. The framework could
9be strengthened by acknowledging previous attempts to build consensus and citing sources where
available.
10Annex A. Agenda & Participant List
Roundtable Discussion: Tracking Progress Towards Post 2015 Targets
Meeting Details, Agenda, and Participants
Details Tuesday, December 2, from 12:30pm-5:00pm ET
Somers Room, The Brookings Institution
1775 Massachusetts Avenue, NW
Washington, D.C. 20036
Domestic Toll Free Number: 800-747-5150
International Number: +1 303-248-1290
UK Toll Free Number: 800-496-0577
Participant Passcode: 7976180#
Agenda
12:30 –1:00pm Lunch, Opening remarks and Introductions—Rebecca Winthrop, Center for
Universal Education
1:00 – 1:15pm Presentation on Towards Indicators for a Post-2015 Education Framework —
Alison Kennedy, UNESCO Institute for Statistics
1:15 – 1:25pm Update on London Meeting: Consultation Outcomes —Louise Banham, UK
Department for International Development
1:25 – 1:35pm Post-2015 Learning and Equity Indicators – Heather Simpson and Meredy Talbot-
Zorn, Save the Children
1:35 – 3:10pm Reflections on Early Childhood Development Indicators — Kate Anderson,
Center for Universal Education
Discussion on Early Childhood Development Indicators – facilitated by Rebecca
Winthrop, Center for Universal Education
3:10pm – 3:20pm Break
3:20pm – 4:50pm Reflections on Early Grade Learning Indicators – Marguerite Clarke, World Bank
Discussion on Early Grade Learning Indicators—facilitated by Rebecca Winthrop,
Center for Universal Education
4:50pm – 5:00pm Next steps and conclusion— Rebecca Winthrop, Center for Universal Education
Participants
11Kate Anderson, Center for Universal Education
Tony Baker, RESULTS Educational Fund
Louise Banham, U.K. Department for International Development
Sarah Beardmore, Global Partnership for Education
Penelope Bender, U.S. Agency for International Development
Lisa Blonder, U.S. Department of State
Manuel Cardoso, United Nations Children’s Fund
Marguerite Clarke, The World Bank
Patrick Collins, U.S. Agency for International Development
Amber Gove, RTI International
Christine Janes, U.S Agency for International Development
Dana Kelly, National Center for Education Statistics
Alison Kennedy, UNESCO Institute for Statistics
Dan McGrath, National Center for Education Statistics
Maureen McLaughlin, U.S. Department of Education
Becky Miller, U.S. Department of Education
Noel Schroeder, Women Thrive Worldwide
Heather Simpson, Save the Children
Meredy Talbot-Zorn, Save the Children
Rob Weil, American Federation of Teachers
Rebecca Winthrop, Center for Universal Education
12You can also read
