Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework

 
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Comparative Analysis of the
     European Qualifications
Framework and the New Zealand
    Qualifications Framework:
     Joint Technical Repor t
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
LEGAL NOTICE
This document was prepared by the New Zealand Qualifications Authority (NZQA) and the European Commission.
Neither the European Commission or NZQA nor any person acting on behalf of the Commission or NZQA may be
held responsible for the use that may be made of the information contained in this publication.

More information on the European Union is available on the Internet (http://www.europa.eu).

Luxembourg: Publications Office of the European Union, 2016
ISBN: 978-92-79-63854-1
doi: 10.2767/211812
Catalogue number: KE-07-16-055-EN-N

New Zealand:
ISBN (Print): 978-1-877444-87-6
ISBN (Online): 978-1-877444-88-3

© European Union/New Zealand Qualifications Authority

Reuse for non-commercial use is authorised provided the source is acknowledged. No part of this publication
is to be distributed or have copies made for commercial use without the prior written consent of NZQA
or the European Commission.

IMAGE COPYRIGHT
Page 15: © European Union, 2016 / Source: EC – Audiovisual Service / Photo: Andre Roque
Page 19: © European Union, 2016 / Source: EC – Audiovisual Service / Photo: Mauro Bottaro
Page 58-59 © European Patent Organisation, 2016 / Source: EC – Audiovisual Service / Photo: Heinz Troll
Page 69 © European Union, 2016 / Source: EC – Audiovisual Service / Photo: Mauro Bottaro
Images on pages (7, 54-55) courtesy of Education New Zealand

                                         Europe Direct is a service to help you find answers
                                            to your questions about the European Union.
                                       Freephone number (*): 00 800 6 7 8 9 10 11
                                          (*)The information given is free, as are most calls
                                  (though some operators, phone boxes or hotels may charge you).

EUROPEAN COMMISSION                                                   NEW ZEALAND QUALIFICATIONS AUTHORITY
Directorate-General for Employment, Social Affairs and Inclusion      Level 13, 125 The Terrace
Directorate E – Skills                                                Wellington 6011
Unit E.2 – Skills & Qualifications
                                                                      Contact: Internationalunit@nzqa.govt.nz
Contact: EMPL-E2-UNIT@ec.europa.eu@ec.europa.eu
European Commission
B-1049 Brussels
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Table of contents

Executive summary............................................................................................................................. 2
Introduction: context, scope and purpose..................................................................................... 7
Introduction on methodology........................................................................................................ 15
Principles for comparison............................................................................................................... 19
    Principle 1: The roles and responsibilities of the New Zealand Qualifications Authority and the
    corresponding authorities for the European Qualifications Framework are clear and transparent..............20
    Principle 2: Comparability of the New Zealand Qualifications Framework and the European
    Qualifications Framework and their levels......................................................................................................................................24
    Principle 3: The New Zealand Qualifications Framework and the European Qualifications
    Framework are based on learning outcomes................................................................................................................................31
    Principle 4: There are transparent processes for the inclusion of qualifications in both
    qualifications frameworks..........................................................................................................................................................................33
    Principle 5: Both qualifications frameworks are underpinned by quality assurance
    and are consistent with international quality assurance principles.....................................................................................39
    Principle 6: National or regional policies for the validation of all learning and credit systems,
    where these exist, are a key feature of the qualifications frameworks............................................................................51

Lessons learned................................................................................................................................. 54
Glossary.............................................................................................................................................. 58
Annexes............................................................................................................................................... 69
    Annex I: Technical comparison of the NZQF and the EQF.................................................................................................70
    Annex II: Paper by NZQA to inform discussion on the matching of the New Zealand
    Qualifications Framework level 8 to the European Qualifications Framework.........................................................80

                                                                                                                                                                                                     1
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Executive Summary

2   Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Executive Summary

Introduction                                            in place qualifications frameworks is to facilitate
                                                        recognition of qualifications to support mobility
This report sets out an analysis of the technical       of learners and workers – both within and
and conceptual characteristics of the European          between countries.
Qualifications Framework (EQF) and the
New Zealand Qualifications Framework (NZQF)             New Zealand shares a common educational
in their operational contexts, identifying key          heritage with many European countries, and
elements of comparability and similarities and          there is significant mobility of students and skilled
differences in a mutually beneficial way.               workers between New Zealand and Europe.
                                                        New Zealand and Europe have a long history of
The analysis is the product of a joint project          education engagement through bilateral relations
completed by a Technical Working Group                  with individual nations and with the European
formed between the New Zealand Qualifications           Union. This project comparing the NZQF and EQF
Authority and the EQF Advisory Group in 2014-15.        can help to further strengthen existing relationships
The work on this report was completed in 2016.          between New Zealand and the European Union.

                                                        While this report does not reference the NZQF
                                                        with European national qualifications frameworks,
The frameworks                                          it is intended to be useful for people in the European
The EQF was established in 2008 as a regional           Union seeking to understand New Zealand
common reference framework with the purpose             qualifications, their place in the New Zealand
of improving the transparency, comparability            education and qualification system and the
and portability of qualifications in Europe. European   quality assurance system that underpins the
national qualifications frameworks are currently        NZQF; and vice versa for people in New Zealand
being ‘referenced’ to the EQF. Referencing in the       to acquire better understanding of the EQF
EQF is a process that results in the establishment      objectives and implementation and of qualifications
of a formal relationship between the levels of a        referenced to the EQF through European national
European national qualifications framework and          qualifications frameworks.
the levels of the EQF. As a meta-framework,
the EQF does not contain qualifications itself.         The primary audience for this report are policy
                                                        makers within New Zealand, Europe and beyond
The NZQF was established in July 2010. It replaced      who wish to gain further understanding of the
the both the National Qualifications Framework          commonality and differences between the
(established in 1992) and the New Zealand Register      NZQF and the EQF. Equally, it is intended that
of Quality Assured Qualifications (established in       the report be used as a resource to inform policy
2001). The NZQF is a single, unified framework          decision-making for future education and training
for all New Zealand’s quality assured qualifications,   cooperation and engagement, particularly in
from senior secondary school to doctoral degrees.       key policy areas of strategic importance to
                                                        New Zealand and the European Union.

                                                        This report can serve as a source of information
Project purpose and scope                               to support transparent and consistent recognition
                                                        decisions informed by a strong understanding
International comparability of qualifications is
                                                        and appreciation of the learning outcomes
important in New Zealand and Europe as set out
                                                        delivered by the frameworks. It will not, however,
in the objectives and policies of both qualifications
                                                        result in automatic or guaranteed recognition of
frameworks. One of the objectives for putting

                                                                                                              3
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Executive Summary
Continued

qualifications. Recognition of qualifications is carried              • Principle 6: National or regional policies for
out by relevant competent authorities in Europe                         the validation of all learning and credit systems,
and New Zealand.                                                        where these exist, are a key feature of the
                                                                        qualifications frameworks.

Project approach                                                      The comparative analysis process included
                                                                      structural and technical analysis and comparison
A joint Technical Working Group was established                       of key concepts, detailed analysis of level
between the NZQA and the EQF Advisory Group                           descriptors of both frameworks, and contextual
in 2014 to carry out exploratory technical work                       and social matching. The structural and technical
comparing the NZQF to the EQF. This technical                         comparison provided an initial view, but for some
work included a series of detailed information                        framework levels, further research was required
sharing meetings via video conference, and a study                    to make a more comprehensive comparison.
visit to New Zealand by EQF Technical Working                         A contextual and social effects matching process
Group members in February 2015. NZQA also                             which examined the place of qualifications of
gave two presentations to EQF Advisory Group                          certain levels within their national contexts was
in Brussels: the first in June 2014 on the NZQF,                      used to deepen the comparison.
and the second in December 2015 on the
comparative analysis report.                                          To generate trust, analysis of the NZQF and the
                                                                      EQF required an equivalent level of transparency
The Technical Working Group agreed to                                 between the members of the Technical Working
structure the content of this report according                        Group, particularly in relation to key features
to the following principles adapted from the                          such as the use of learning outcomes and
Criteria and procedures for referencing national                      quality assurance.
qualifications levels to the EQF. These principles
provided the basis through which the foundations                      As part of the project process, NZQA and
and key aspects of each framework could be                            the EQF Advisory Group consulted with their
compared with the other.                                              respective education sectors and relevant bodies.

• Principle 1: The roles and responsibilities of
  NZQA and the corresponding authorities                              Project outcomes
  for the EQF are clear and transparent.
• Principle 2: Comparability of the NZQF                              The Technical Working Group examined the
  and the EQF and their levels.                                       characteristics of the EQF as a meta-framework
                                                                      for European countries and the NZQF as a
• Principle 3: The NZQF and the EQF are                               national framework. While there are conceptual
  based on learning outcomes.                                         and functional differences between the NZQF
• Principle 4: There are transparent processes                        and the EQF, due to their different purposes and
  for the inclusion of qualifications on both                         functions, the analysis shows that both frameworks
  frameworks.                                                         can be considered comparable.
• Principle 5: Both qualifications frameworks
                                                                      The deepened knowledge and understanding
  are underpinned by quality assurance and
                                                                      of the complex quality assurance mechanisms
  are consistent with international quality
                                                                      underpinning both frameworks gained through
  assurance principles.
                                                                      this comparative analysis will provide a sound basis

4       Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
for strengthening mutual trust and understanding                      Key findings
of qualifications frameworks and qualifications,
and more importantly, their applications in the                       The comparative analysis confirmed that the
real world. This also extends to empowering                           NZQF and EQF are comparable and established a
decision-makers to recognise qualifications in                        relationship between levels of the two frameworks.
the context of their frameworks and can further                       The following diagram provides a summary of the
provide the grounds for improved student and                          correspondence between the NZQF and EQF
labour mobility between European Union                                levels and examples of national qualifications
Member States and New Zealand.                                        types (linked to the EQF via NQFs referenced
                                                                      to the EQF).

                                   NZQF                                                     EQF

                                                                                     Third cycle degrees (Doctorate)
                Doctoral Degrees                            10         8             Higher professional qualifications
                                                                                      EE: occ. qual. ‘chartered engineer’

                                                                                     Second cycle degrees (Master’s)
                Master’s Degrees                             9         7            Higher professional qualifications
                                                                                  CZ: ‘Chemical engineer product ‘manager’

                Bachelor Honours Degrees*
                Postgraduate Diplomas                        8
                and Certificates

                                                                                      First cycle degrees (Bachelor’s)
                                                                                       IE: Ordinary Bachelor’s degree,
                Bachelor’s Degrees                           7         6                  Honours Bachelor Degree
                                                                                     Higher professional qualifications
                                                                                      DE: ‘Master Craftsman (certified)’

                Diploma Qualifications                       6
                                                                                          SCHE qualifications
                                                                       5            Higher professional qualifications
                Diploma Qualifications                       5

                Certificate level 4
                Trade qualifications                         4                           Upper secondary general
                                                                                         school-leaving certificates
                                                                       4
                Certificate level 3                                                    Upper secondary VET school
                NCEA (secondary school
                – leaving certificate)
                                                             3                             leaving certificates

                                                                                           VET qualifications
                Certificate level 2                          2         3             Secondary education certificates

                Certificate level 1                                                   Lower-secondary education
                NCEA                                         1         2                Basic VET qualifications

                                                                                          Primary education
                                                                       1                Basic VET qualifications

           *reflects the Technical Working Group’s agreement that NZQF level 8 will remain unlevelled to the EQF.

                                                                                                                             5
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Acronyms

CEDEFOP               European Centre for the Development of Vocational Training
CUAP                  Committee on University Academic Programmes
ECTS                  European Credit Transfer and Accumulation System
ECVET                 European Credit System for Vocational Education and Training
EER                   External Evaluation and Review
EHEA                  European Higher Education Area
ENQA                  European Association for Quality Assurance in Higher Education
EQAR                  European Quality Assurance Register
EQAVET                European Quality Assurance for Vocational Education and Training
EQF                   European Qualifications Framework
ESG                   European Standards and Guidelines for Higher Education
ETF                   European Training Foundation
EU                    European Union
HE                    Higher Education
ITO                   Industry Training Organisation
ITP                   Institute of Technology and Polytechnic
IVET                  Initial Vocational Education and Training
LLL                   Lifelong Learning
MOE                   Ministry of Education
NCEA                  National Certificate of Educational Achievement
NCP                   National Coordination Point (for EQF implementation at national level,
                      also known as EQF-NCP)
NQF                   National Qualifications Framework
NZQA                  New Zealand Qualifications Authority
NZQF                  New Zealand Qualifications Framework
PISA                  Programme for International Student Assessment
PTE                   Private Training Establishment
QF-EHEA               Qualifications Framework in the European Higher Education Area
SCHE                  Short-cycle Higher Education
TEO                   Tertiary Education Organisation
TVET                  Technical and Vocational Education Training
TWG                   Technical Working Group
UNESCO                United Nations Educational, Scientific and Cultural Organisation
VET                   Vocational Education and Training
VNFIL                 Validation of Non-formal and Informal Learning

6   Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Introduction: context,
   scope and purpose

                     7
Joint Technical Report - Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework
Introduction: context, scope and purpose

This report is a result of a Technical Working                            objectives for putting in place qualifications
Group (TWG) between the New Zealand                                       frameworks is to facilitate recognition of
Qualifications Authority (NZQA) and the                                   qualifications to support mobility of learners
European Qualifications Framework (EQF)                                   and workers – both within and between
Advisory Group, following a comparative                                   countries. Qualifications frameworks are rapidly
examination of the New Zealand Qualifications                             emerging around the world, with UNESCO,
Framework (NZQF) and the European                                         the European Centre for the Development of
Qualifications Framework (EQF).                                           Vocational Training (Cedefop), and the European
                                                                          Training Foundation (ETF) showing that in 2014,
In June 2014, there was agreement by both                                 there were more than 150 countries and territories
the NZQA and the EQF Advisory Group to                                    involved in the development and implementation
examine the characteristics of the EQF as a                               of qualifications frameworks.1 International
meta-framework for European countries and                                 cooperation in using NQFs via transnational
the NZQF as a national framework.                                         frameworks and for recognition purposes is a
                                                                          growing trend. The same is true for meta or
The purpose of this joint report is to analyse                            regional qualifications frameworks to which
and document the technical and conceptual                                 NQFs of that region are referenced.2
characteristics of the respective frameworks in
their operational contexts, and systematically                            A comparison of the NZQF and the EQF
identify key elements of their comparability,                             can provide the grounds for improved mobility
similarities and differences, in a mutually                               between European Union (EU) Member States
beneficial way for both framework communities.                            and New Zealand. Qualifications frameworks
In effect, the comparison of the NZQF and EQF                             form part of a country or region’s overall quality
will create a ‘zone of mutual trust’, leading to a                        assurance framework, and can improve stakeholder
better functional understanding and appreciation                          confidence and trust in education and qualifications
of NZQF qualifications and respective learning                            systems. The deepened knowledge and
outcomes in Europe, and a better understanding                            understanding of the complex quality assurance
of the EQF in New Zealand, and the respective                             mechanisms underpinning both frameworks gained
European national qualifications frameworks                               through this comparative analysis will provide
(NQFs) that are referenced to it. By broadening                           a sound basis for strengthening mutual trust
and deepening the functional knowledge                                    and understanding of qualifications frameworks
and understanding of respective frameworks,                               and qualifications, and more importantly, their
transparency in the frameworks can be further                             applications in the real world. This also extends
elevated, which in turn, can enhance opportunities                        to empowering decision-makers to recognise
for future cooperation between New Zealand                                qualifications in the context of their frameworks
and Europe.                                                               to better support learner and worker mobility.

International comparability of qualifications is                          Within this context, it is important to stress there
important in New Zealand and Europe, and is                               are different parameters within each framework
articulated through the objectives and policies                           that must be considered and understood as the
of both qualifications frameworks. One of the                             differences explain the limitations of this activity.

1   http://www.cedefop.europa.eu/en/publications-and-resources/publications/2213
2   EQF, ASEAN qualifications reference framework, the Caribbean Qualifications Framework, the Gulf Qualifications Framework, the Pacific
    Qualifications Framework, the Southern African Development Community Qualifications Frameworks and the Transnational Qualifications
    Framework for the Virtual University of Small States of the Commonwealth

8        Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
The EQF was established in 2008 as a regional                             education and training systems and through the
common reference framework with the purpose                               EU treaties, Member States have assigned certain
of improving the transparency, comparability and                          powers to the EU. The current recommendation
portability of qualifications in Europe. As a meta-                       of the EU Council and the European Parliament in
framework it does not contain qualifications                              relation to the EQF does not contain the mandate
itself. European NQFs, which do encompass                                 to engage in international formal arrangements.
qualifications, are referenced to the EQF. However,
referencing a qualification to NQFs and thereby                           The relative simplicity of the EQF can mask the
to the EQF does not give individuals any rights                           extensive diversity of the European qualifications
for automatic recognition of their qualification.                         landscape. The EQF is based on independent
                                                                          national qualifications systems and foresees no
In 1992, New Zealand developed one of the                                 harmonisation or ‘merger’ of diverse European
first qualifications frameworks in the world –                            qualifications systems. National diversity is seen
the National Qualifications Framework. In July                            as a strength and the comparison of the NZQF
2010, the NZQF replaced both the National                                 and the EQF must not overlook this essential
Qualifications Framework and the New Zealand                              aspect. The role of the EQF as a central ‘hub’
Register of Quality Assured Qualifications.3                              or reference point will facilitate direct dialogue
Specifically, the NZQF:                                                   between New Zealand and European NQFs and
                                                                          will also make it easier for education and training
• conveys the skills, knowledge and attributes                            institutions, employers and recognition bodies in
  a graduate has gained through completing                                EQF countries to develop understanding of the
  a qualification                                                         frameworks and build links with their New Zealand
                                                                          counterparts. Analysis of the comparability,
• requires the development of integrated and
                                                                          similarities and differences between the NZQF
  coherent qualifications that meet the needs of
                                                                          to the EQF also strengthens the overall credibility
  individuals, groups, industry and the community
                                                                          of the EQF as a reference framework.
• enables and supports the provision of
  high-quality education pathways                                         There has been growth in qualifications
• enhances confidence in the quality and                                  frameworks and their international linkages on
  international comparability of New Zealand                              a global scale. This joint activity will assist in those
  qualifications                                                          linkages and support the development of bilateral
                                                                          cooperation and trust between New Zealand
• contributes to Ma-ori success in education by                           and the EU. The relative success of the EQF
  recognising and advancing ma-tauranga Ma-ori                            so far is based on the gradual development
• represents value for money, is sustainable                              of trust between the countries involved in
  and robust.                                                             the development and implementation of the
                                                                          framework in the region. To generate trust,
This joint activity is concerned with a comparison                        analysis of the NZQF and the EQF requires
of the EQF as a meta-framework with the NZQF                              an equivalent level of transparency and
which is an NQF. This project does not reference                          documentation, particularly in relation to key
the NZQF with European NQFs. Within the EU,                               features such as the use of learning outcomes
Member States are fully responsible for their                             and quality assurance.

3   For further information about the NZQF timeline see: http://www.nzqa.govt.nz/studying-in-new-zealand/understand-nz-quals/nzqf/timeline/

                                                                                                                                              9
Introduction: context, scope and purpose
Continued

The NZQF and the Asia-Pacific                                                The EQF and the European
Region                                                                       Region
New Zealand maintains a close political and                                  Supporting cross-border mobility of learners
economic relationship with Australia. At a                                   and workers and facilitating recognition of
government-to-government level, New Zealand’s                                qualifications and lifelong learning across Europe
relationship with Australia is the closest and most                          is a major aim of the EU. This means that
comprehensive of all its bilateral relationships.                            qualifications need to be understandable across
Both governments have stated the importance                                  different countries and systems in Europe.
of the relationship and their intention to strengthen                        The EQF is one of the core European instruments
links further, especially through deeper economic                            for supporting mobility and lifelong learning and
integration. New Zealand recently referenced                                 has been the main catalyst in the development
the NZQF with the Australian Qualifications                                  of NQFs for lifelong learning in Europe. The EQF
Framework (AQF) as one way of strengthening                                  has also been a pioneer for the development
the links between New Zealand and Australia                                  of regional reference frameworks in the world.
and supporting the mobility of students and                                  Qualifications frameworks, as powerful descriptions
skilled workers between the countries.4                                      of qualifications systems, are also outward looking
                                                                             and are attractive to people in other countries
New Zealand and Malaysia have also completed                                 as a quick reference to qualifications in countries
qualification recognition projects that aim to                               with NQFs. They act as bridges for understanding
facilitate mobility between the two countries.                               qualifications between countries.
These projects, which contributed to the
New Zealand-Malaysia Free Trade Agreement,                                   The EQF has been designed to act as a reference
have demonstrated the comparability of both                                  for different qualifications systems and frameworks
countries’ Bachelor’s, Master’s and Doctoral                                 in Europe. It takes into account the diversity of
Degrees.5                                                                    national systems and facilitates the translation and
                                                                             comparison of qualifications between countries.
New Zealand also engages in a range of bilateral                             It does not concern itself with the ways in which
and multilateral fora in the Asia-Pacific to support                         countries structure and prioritise their education
student and labour mobility in the region.                                   and training policies, structures and institutions
New Zealand supports the development of the                                  nor does it directly include qualifications. It is a
Association of South East Asian Nations (ASEAN)                              meta-framework that is a reference point for
Qualifications Reference Framework (AQRF). The                               these national systems and national and sectoral
AQRF is a meta or regional qualifications framework                          qualifications frameworks based on learning
that will enable mobility between ASEAN Member                               outcomes in which qualifications are classified
States through the development and strengthening                             to relate to each other. Qualifications are not
of NQFs and systems in the region.                                           directly allocated to EQF levels, as they are only
                                                                             linked to EQF levels via the referencing of national
                                                                             qualifications levels to the EQF levels. By acting
                                                                             as a translation device, the EQF aids in the

4    See: http://www.nzqa.govt.nz/assets/About-us/Our-role/Enhancing-mobility.pdf
5    See: http://www.nzqa.govt.nz/about-us/our-role/our-role-in-international-education/country-specific-recognition-arrangements/malaysia/

10        Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
understanding of qualifications allocated to national                         meetings between EQF national coordination points
levels across the different countries and education                           (NCPs) and ‘Bologna’ framework coordinators.
and qualification systems in Europe.

By June 2016, 29 countries had referenced their                               New Zealand-European Union
national qualifications levels to the EQF. These
countries were Austria, Belgium (FL, FR), Bulgaria,                           Bilateral Relations
Croatia, the Czech Republic, Denmark, Estonia,
                                                                              New Zealand and the EU share historic and
France, former Yugoslav Republic of Macedonia,
                                                                              cultural links as well as a commitment to build
Germany, Greece, Hungary, Iceland, Ireland, Italy,
                                                                              a liberal, democratic, rules-based international
Latvia, Liechtenstein, Lithuania, Luxembourg, Malta,
                                                                              system. To support such a world, New Zealand
Montenegro, the Netherlands, Norway, Poland,
                                                                              and the EU cooperate closely on a broad and
Portugal, Slovenia, Sweden, Switzerland and the
                                                                              expanding set of issues, including: security,
United Kingdom. The remaining countries are
                                                                              international development, climate change,
expected to present their referencing reports
                                                                              research and innovation among others. In 2014
in 2016-17.
                                                                              the partners placed the spectrum of their bilateral
                                                                              relations in a treaty-level framework for the first
The development of NQFs in Europe reflects the
                                                                              time with the conclusion of negotiations for
Bologna process and the agreement to implement
                                                                              the Partnership Agreement on Relations and
qualifications frameworks in the European higher
                                                                              Cooperation (PARC). In March 2014 Prime
education area (QF-EHEA).6 All countries involved
                                                                              Minister Key and then-Presidents of the European
in EQF implementation are participating in the
                                                                              Council and European Commission, Herman Van
Bologna process. Twenty five countries had ‘self-
                                                                              Rompuy and Jose-Manuel Barroso, set in motion
certified’ their higher education (HE) qualifications
                                                                              a reflection process to determine whether the
to the QF-EHEA by February 2016. Countries
                                                                              two sides might deepen their already close trade
are increasingly combining referencing to the EQF
                                                                              and investment relations.
and self-certification to the QF-EHEA;7 Austria,
Bulgaria, Croatia, Estonia, former Yugoslav Republic
                                                                              New Zealand shares a common educational
of Macedonia, Hungary, Iceland, Latvia, Lithuania,
                                                                              heritage with many European countries, and
Luxembourg, Malta, Montenegro, Norway,
                                                                              there is significant mobility of students and skilled
Poland, Portugal and Slovenia have all produced
                                                                              workers between New Zealand and Europe.
joint reports on both processes, reflecting the
                                                                              New Zealand and Europe have a long history
priority given to the development and adoption
                                                                              of education engagement through bilateral
of comprehensive NQFs covering all levels and
                                                                              relations with individual nations and with the EU.
types of qualifications. It is expected that this
                                                                              According to UNESCO, in 2013 approximately
approach will be chosen by most countries
                                                                              393 New Zealand students were studying in
preparing to reference to the EQF in 2016-
                                                                              Europe at tertiary level and 1900 European
17. This development reflects the increasingly
                                                                              students were studying in New Zealand at tertiary
close cooperation between the two European
                                                                              level.8 A shared understanding of New Zealand’s
framework initiatives, also illustrated by regular

6   http://www.ehea.info/article-details.aspx?ArticleId=65
7   Self-certification reports verify the compatibility of the national framework for higher education with the QF-EHEA. Self-certification is done
    on basis of the ‘Dublin Descriptors’, which are fully compatible with the descriptors of EQF levels 5-8. – cf. http://www.ehea.info/Uploads/
    Documents/QF-EHEA-May2005.pdf
8   Global Flow of Tertiary-Level Students http://www.uis.unesco.org/Education/Pages/international-student-flow-viz.aspx

                                                                                                                                                      11
Introduction: context, scope and purpose
Continued

and Europe’s qualifications and systems will support                       fostering mutual trust, and building capacity for
increased mobility of students and skilled workers                         qualifications recognition. This relies on information
and the mutual recognition of qualifications.                              and transparency tools, including national and
                                                                           regional qualifications frameworks. A subsidiary text
New Zealand has been an active member of the                               adopted in 2012 included a recommendation that
Asia-Europe Meeting (ASEM) since it officially                             qualifications frameworks should be used to make
joined the summit in October 2010. The ASEM                                it easier for competent recognition authorities
brings together the European Union, 21 Asian                               to assess foreign qualifications and encouraged
countries, the ASEAN secretariat and Australia and                         states to explore how framework levels, learning
New Zealand. At the ASEM meeting of minsters of                            outcomes, quality and workload can be used to
education in Riga in 2015 the ministers “Reaffirmed                        facilitate recognition.10
the relevance of achieving more transparency and
improving understanding of the different education                         A comparative analysis of the NZQF and
systems within and between both regions and                                EQF could serve as a source of information
making education systems more comparable,                                  which informs recognition decisions made by
facilitating mobility and enhancing collaboration.9                        competent recognition authorities in Europe and
Recently, New Zealand was invited to participate                           New Zealand, but will not result in automatic or
as an ASEAN policy dialogue partner at the                                 guaranteed recognition. A foreign qualification’s
inaugural SHARE Policy Dialogue. SHARE is an EU                            context in a national or regional qualifications
support project to higher education in the ASEAN                           framework may be taken into consideration,
region to strengthen regional cooperation, enhance                         but this is a matter for the competent recognition
quality, competitiveness and internationalisation of                       authorities in the country to determine.
ASEAN higher education institutions.
                                                                           New Zealand has been a signatory to the European
                                                                           Convention on the Equivalence of Diplomas Leading
Lisbon Recognition Convention
                                                                           to Admission to Universities of 1953 since 1978.
The Council of Europe – UNESCO Convention on                               Article 1.1 of the convention recognises the
the Recognition of Qualifications concerning Higher                        equivalence of university entrance qualifications
Education in the European Region of 1997 (Lisbon                           between signatory countries, where that entrance
Recognition Convention) is a legal instrument                              is subject to state control.
developed by the Council of Europe and UNESCO
which binds over 50 countries, including European                          New Zealand is also a member of the European
countries, New Zealand and Australia, to adopt                             Network of National Information Centres (ENIC)
fair practices in the recognition of higher education                      and the National Academic Recognition Information
qualifications. New Zealand became a party to                              Centres Network (NARIC). This report will broaden
the Convention in 2008.                                                    the range of agencies and the number of people
                                                                           who understand and recognise New Zealand
The Lisbon Convention enhances internationalisation                        and European qualifications.
and mobility by introducing and improving
qualifications recognition policies and processes,

9  5th ASEM Education Ministers’ Meeting (ASEMME5) Riga, 27-28 April 2015, ASEM Education Collaboration for Results: Conclusions by the
   Chair. http://asem-education-secretariat.kemdikbud.go.id/asemme5/
10 Recommendation on the use of qualifications frameworks in the recognition of foreign qualifications http://www.coe.int/t/dg4/highereducation/
   Recognition/DGIIEDUHE(2012)14%20Rev09%20FINAL%20-%20LRC%20Supplementary%20Text%20on%20the%20Use%20of%20QFs%20
   ENGLISH.asp#TopOfPage

12        Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
Arrangements between New Zealand                                          • NCEA Level 2 is comparable to the United
and individual EU Member States                                             Kingdom GCE (grades A–C)
                                                                          • NCEA Level 3 is comparable to the United
Some European Member States have already                                    Kingdom GCE Advanced Level (A-Level)
established bilateral understandings or agreements
with New Zealand in the matters of qualifications.                        • University Entrance with NCEA Level 3 (with
The present study does not invalidate or                                    Merits/Excellences in subjects to be studied at
undermine these agreements. It does not take                                higher education institutions) is comparable to
these as a constraint on its own independent                                those with the overall GCEA Level.
consideration of the NZQF. Neither does the
study imply any extension to these agreements                             Scope
or require other countries to engage in similar
processes. They are documented here in                                    Individuals seek qualifications recognition for
the interests of demonstrating some of the                                a variety of purposes including admission to
understanding and trust that has already been                             further study, occupational registration/licensing,
established between European states and                                   employment and migration. Recognition is carried
New Zealand.                                                              out by the competent authorities in Europe and
                                                                          New Zealand.
Ireland
                                                                          The report can support transparent and
From 2008-2010, New Zealand and Ireland                                   consistent recognition decisions informed by
collaborated on a project to improve the basis                            a strong understanding and appreciation of the
on which respective qualifications are recognised.                        learning outcomes delivered by the frameworks.
The joint New Zealand-Irish report relates to                             Its outcomes do not entitle any holder of a
levels 7, 8, 9 and 10 of both the Irish National                          European or New Zealand qualification to claim
Framework of Qualifications and the New Zealand                           (automatic) recognition, but it will supplement
Register of Quality Assured Qualifications.11                             the existing body of knowledge acquired over
The project agreed that New Zealand’s level 7                             the many years of student and labour mobility
Bachelor’s Degrees, level 8 Bachelor Honours                              between Europe and New Zealand.
Degrees, level 9 Master’s Degrees and level 10
Doctoral Degrees are broadly compatible with
levels 7-10 of the Irish National Framework                               Intended audience
of Qualifications.
                                                                          The report is intended to be useful for people
                                                                          in the EU seeking to understand New Zealand
The United Kingdom                                                        qualifications, their place in the New Zealand
The United Kingdom recognises levels of NCEA                              education and qualification system and the quality
as equivalent to different levels of the General                          assurance system that underpins the NZQF; and
Certificate of Education (GCE).12                                         vice versa for people in New Zealand to acquire
                                                                          better understanding of the EQF objectives and
                                                                          implementation and of qualifications referenced
UK NARIC (the United Kingdom National
                                                                          to the EQF through European NQFs. The primary
Academic Recognition Information Centre)
states that:

11 http://www.nzqa.govt.nz/assets/About-us/Our-role/Qualifications-Compatibility-Project.pdf
12 http://www.nzqa.govt.nz/studying-in-new-zealand/secondary-school-and-ncea/international-recognition-of-ncea/specific-country-requirements/

                                                                                                                                            13
Introduction: context, scope and purpose
Continued

audience for this report are policy makers within
New Zealand, Europe and beyond who wish to
gain further understanding of the commonality
and differences between the NZQF and the EQF.
Equally, it is intended that the report be used as
a resource to inform policy decision-making for
future education and training cooperation and
engagement, particularly in key policy areas of
strategic importance to New Zealand and the EU.

14     Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
Introduction on
   methodology

              15
Introduction on methodology

Technical Working Group                                                   • open and in-depth analysis and discussion on
                                                                            opportunities, challenges, benefits and risks
A joint Technical Working Group (TWG) was                                 • development of a joint set of criteria as the
established in 2014 between NZQA and the                                    technical basis for comparability
EQF Advisory Group. For the NZQF, the TWG
comprised the New Zealand Qualifications                                  • reporting back to the EQF Advisory Group
Authority (NZQA). For the EQF the TWG                                       and the countries’ national bodies at key stages
comprised the EQF Advisory Group members                                  • recommending the final report for
from the Czech Republic, Germany and Ireland,                               consideration.
and Cedefop and the European Commission.
                                                                          The TWG agreed to the approach below to
The working language of the TWG is English. It                            technical work between the NZQA and the
is worth noting that while much technical work                            EQF Advisory Group at the June 2014 meeting:
around the EQF is conducted in English, the
framework itself is presented in the 22 official                          • analysis of risks and benefits of cooperation
languages of the EU Member States.13 The individual
NQFs in Europe that have been referenced to                               • technical work on identifying the relationship
the EQF are in their national languages. All have                           between the third country qualifications
been translated into English. European tools are                            framework and the EQF.
available to facilitate translation.14 Comparisons that
rely heavily on detailed lexical analysis, such as this                   The TWG met in November 2014 by video
one, are further complicated by the multilingual                          conference to discuss the benefits and risks papers
background of the participants and the linguistic                         drafted by New Zealand and Europe and to agree
contexts in which the referenced frameworks                               a proposed outline of the joint technical report.
and qualifications were originally written.                               It further held a number of video-conferences to
                                                                          progress the report and share information on
The TWG was formed to carry out exploratory                               the frameworks.
technical work comparing the NZQF and the EQF.
It exchanged information on key aspects which                             A study visit of the EQF TWG members took
relate to qualification frameworks by ensuring:                           place to New Zealand (24-27 February 2015).
                                                                          Through an interesting programme the visit
• a much clearer understanding of the different                           provided excellent insights into the functioning
  systems supporting qualifications                                       of the NZQF in all its composing parts and in
                                                                          practice. The EQF delegation met with persons
• an understanding of the key drivers relating                            in charge of the administration of the NZQF,
  to qualifications frameworks and how these                              with bodies/persons in charge of quality assurance
  materialise in New Zealand and Europe                                   and funding, as well as with industry stakeholders.
• analysis of the comparability of the two                                The EQF delegation met with the New Zealand
  frameworks and their levels                                             Minister for Tertiary Education, Skills and
                                                                          Employment, Hon Steven Joyce, showing
                                                                          a high level of political interest in the project.

13 http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32008H0506(01)
14 http://www.cedefop.europa.eu/en/publications-and-resources/publications/4064
   https://ec.europa.eu/esco/home; http://www.cedefop.europa.eu/sl/publications-and-resources/publications/6119

16        Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
The EQF was presented to stakeholders in                 Consultation process
New Zealand, through a combined presentation
of the EQF (by the Commission services and
Cedefop) and of European NQFs referenced                 New Zealand
to it (presentation of the Irish Qualifications
                                                         New Zealand engaged with the wider sector
Framework).
                                                         through the New Zealand Advisory Group
                                                         members. Members covered the education sector,
The TWG agreed to the structure and content              social and economic partners and government
of the report according to the following principles      agencies as set out below:
adapted from the Criteria and procedures for
referencing national qualifications levels to the EQF:
                                                         • Universities New Zealand – Te Po-kai Tara
• Principle 1: The roles and responsibilities of         • New Zealand Institutes of Technology
  NZQA and the corresponding authorities                   and Polytechnics
  for the EQF are clear and transparent.                 • The Metro Group
• Principle 2: Comparability of the NZQF                 • Te Tauihu o Nga- Wa-nanga
  and the EQF and their levels.
                                                         • Independent Tertiary Education New Zealand
• Principle 3: The NZQF and the EQF are
  based on learning outcomes.                            • New Zealand Secondary Principals’ Association

• Principle 4: There are transparent processes           • Business New Zealand
  for the inclusion of qualifications on both            • New Zealand Council of Trade Unions
  frameworks.
                                                         • Industry Training Federation
• Principle 5: Both qualifications frameworks
  are underpinned by quality assurance and               • New Zealand Union of Students’ Association
  are consistent with international quality              • The Ministry of Foreign Affairs and Trade
  assurance principles.
                                                         • Education New Zealand
• Principle 6: National or regional policies for
                                                         • Ministry of Education
  the validation of all learning and credit systems,
  where these exist, are a key feature of the            • Ministry of Business, Innovation
  qualifications frameworks.                               and Employment.

                                                                                                        17
Introduction on methodology
Continued

NZQA met with the New Zealand Advisory                               Representatives from these institutions included
Group in May, August and November 2014,                              Deputy Vice Chancellors Academic, Academic
and February, May, August, and November 2015.                        Managers, Academic Directors, Deans of Graduate
NZQA also formed a focus sub-group specifically                      Studies, Academic Policy and Regulations staff
to consider the lower levels of the NZQF and                         members, and Heads of Departments.
the EQF and this group met on 15 August 2014.
Attendees included representatives from:                             NZQA also consulted closely with the Universities
                                                                     New Zealand Committee on University Academic
• schools                                                            Programmes (CUAP) to reach agreement on the
                                                                     best outcomes for NZQF qualifications that are
• the vocational sector
                                                                     delivered in universities.
• industry
• higher education.                                                  Initial engagement concentrated on comparing the
                                                                     levels of the two frameworks. Engagement with the
Representatives from these groups encouraged                         sector was very positive. Stakeholders fully support
NZQA to attend peak body forums to discuss the                       establishing comparability between the NZQF and
project in more detail and to meet with interested                   the EQF and their levels.
institutions. Initial engagement occurred throughout
September and October 2014 with:                                     EQF
• Private Training Establishments (PTE) peak                         On the European side the main stakeholder
  body forum                                                         forum is the EQF Advisory Group, composed by
                                                                     government representatives as well as European
• Institutes of Technology and Polytechnics (ITP)
                                                                     level education and labour market stakeholders.
  peak body forum
• Industry Training Organisations (ITO) peak
  body forum
• all eight New Zealand Universities – Auckland,
  Auckland University of Technology, Waikato,
  Massey, Victoria, Canterbury, Lincoln and Otago
• WelTec.

18     Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
Principles for
 comparison

             19
Principles for comparison

Principle 1: The roles and responsibilities of the New Zealand
Qualifications Authority and the corresponding authorities for
the European Qualifications Framework are clear and transparent.
Summary                                                              Universities New Zealand has delegated authority
                                                                     for university programme approval, accreditation,
This principle looks at the key bodies or                            listing of university qualifications on the NZQF,
entities that are responsible for the NZQF                           training scheme approval, and ancillary powers
and the EQF governance.                                              under section 253A of the Education Act 1989.

Responsibility for the NZQF and the EQF is                           NZQA and the New Zealand Qualifications
clearly defined, transparent and accessible. NZQA                    Framework
is the body responsible for the development
                                                                     NZQA is the body responsible for the development
and maintenance of the NZQF and the related
                                                                     and maintenance of the NZQF and the related
Directory of Assessment Standards. The EQF
                                                                     Directory of Assessment Standards. The NZQF
Advisory Group, chaired by the European
                                                                     was established in July 2010 as a single unified
Commission, oversees EQF implementation.
                                                                     framework for all New Zealand qualifications. It
It comprises representatives of all participating
                                                                     replaced the National Qualifications Framework
countries, the Council of Europe, EU social
                                                                     and the New Zealand Register of Quality
partners, Cedefop, ETF and other important EU
                                                                     Assured Qualifications.
stakeholders such as public employment services,
student union, and lifelong learning stakeholders.
With regards to the EQF, NCPs have been                              The Education Amendment Act 2011 established
established in all participating countries and are                   the NZQF and the Directory of Assessment
charged with the coordination and promotion of                       Standards in law, replacing general references
their own NQFs which are referenced to the EQF.                      to a ‘qualifications framework’.

                                                                     The effectiveness and quality of the NZQF and
NZQF                                                                 the related Directory of Assessment Standards are
                                                                     supported by a multi-layered and integrated quality
A key function of NZQA is to set the overarching                     assurance system. There are quality checks at each
statutory rules for the quality assurance of                         level and for each component of the system as
qualifications and the tertiary education                            well as aspects of the health of the system overall.
organisations that provide them (section 253                         NZQA applies rules and quality criteria to ensure
of the Education Act 1989). To implement these                       a high and consistent standard.
rules, New Zealand has two quality assurance
agencies with responsibilities for separate parts                    NZQA’s other responsibilities
of the tertiary education sector (section 159AD
of the Education Act 1989):                                          NZQA is designated as New Zealand’s National
                                                                     Education Information Centre under the Lisbon
• NZQA maintains and quality assures                                 Recognition Convention.
  New Zealand’s qualifications system for
  the non-university tertiary education sector                       NZQA is also responsible for maintaining effective
• Universities New Zealand fulfils this function                     relationships with overseas certifying and validating
  for the university sector.                                         bodies. This work allows NZQA to recognise
                                                                     overseas educational and vocational qualifications

20     Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
in New Zealand and have New Zealand educational                         have full and equal participation in society as
and vocational qualifications recognised by other                       Ma- ori, including education.
countries (see section 246A(h) of the Education
Act 1989).                                                              NZQA has an Office of the Deputy Chief
                                                                        Executive Ma-ori, which provides cultural advice
NZQA has the responsibility for the regulation                          and services to NZQA, and has a strategy for
of international education. It administers the Code                     raising the achievement of Ma-ori learners as
of Practice for the Pastoral Care of International                      Ma- ori. This strategy is called Te Rautaki Ma-ori
Students.                                                               2012-2017.

NZQA has responsibility for assessment in senior
                                                                        EQF
secondary schools, and the Education Review
Office evaluates and reports on the education                           The EQF Recommendation
and care of students in early childhood services,
and primary and secondary schools.                                      The development and implementation of the EQF
                                                                        is based on the Recommendation of the European
NZQA’s governance structure                                             Parliament and the Council of 23 April 2008 on
                                                                        the EQF for lifelong learning.15 The objective of this
NZQA has an independent Board of Directors.                             Recommendation is to create a common reference
The directors are appointed by the relevant                             framework which should serve as a translation
Minister of the Crown. The directors are drawn                          device between different qualifications systems
from a wide range of backgrounds. They all bring                        and their levels, whether for general and HE
diverse and valuable experience to the role.                            or for vocational education and training (VET).
                                                                        The EQF recommendation calls on Member
The NZQA Board ensures that NZQA carries                                States particularly to:
out its legislative functions effectively and efficiently,
fulfilling NZQA’s mandate to create and run a                           • link their national qualification systems/
robust and world-class qualifications system in                           frameworks to the EQF (‘EQF referencing’)
New Zealand.
                                                                        • indicate the EQF level on all newly issued
                                                                          certificates, diplomas or Europass documents
NZQA honours the Treaty of Waitangi
                                                                        • designate NCPs to support and guide the
As a Crown entity, NZQA actively upholds the                              relationship between national qualifications
principles and spirit of the Treaty of Waitangi.                          systems and the EQF.

The Treaty of Waitangi is the founding document
of New Zealand which establishes the relationship
between the Crown and Ma- ori and recognises
Ma- ori as tangata whenua (indigenous peoples)
of New Zealand. The Treaty protects Ma-ori
knowledge and skills (ma-tauranga Ma-ori) as a
national taonga (treasure) and ensures that Ma-ori

15 Recommendation of the European Parliament and the Council on the European Qualifications Framework for lifelong learning (EQF), 2008/C
   111/01

                                                                                                                                       21
Principles for comparison
Principle 1 continued

Governance of the EQF at European level                                       The second important network at European level
                                                                              is the NCPs. The 2008 EQF Recommendation
At the EU level, the EQF Advisory Group, chaired                              invites countries to set up NCPs to be able to
by the European Commission, oversees the EQF                                  ‘speak with one voice’ on behalf of complex
implementation. It comprises representatives of                               national qualifications systems. This was considered
all participating countries, Council of Europe                                necessary to succeed in consistent referencing
(for the coordination with the Bologna process),                              to the EQF. NCPs have been established in
EU social partners, Cedefop, ETF and other                                    all participating countries. They support the
important EU stakeholders (e.g. public employment                             referencing to the EQF and in some countries
services, student union, lifelong learning                                    they are also in charge of overall NQF
stakeholders, youth and voluntary sector).                                    coordination and promotion. The institutional
The work is organised in the form of regular                                  basis of the NCPs varies largely between countries
meetings (around five times a year). This is                                  and includes NCPs operating under the remit of
supported by peer learning activities which address                           ministries of education or labour, NCPs situated
particular technical, methodological and policy                               within the same organisation such as the ENIC/
issues which inform practice; and build Member                                NARIC Centre,19 independent organisations,
State capacity and foster mutual trust.16 Cedefop                             and NCPs operating as joint initiatives of several
supports the work by providing analytical and                                 government bodies.
progress reports for discussion.
                                                                              The QF-EHEA is under the governance of the
The EQF Advisory Group has adopted 10                                         Bologna Process. The decision making body is
criteria and procedures to ensure that NQFs                                   the triennial Ministerial Meeting.20 Between these
are referenced to the EQF in a coherent and                                   meetings the Bologna Follow-up Group (BFUG)
transparent way.17 Transparent procedures                                     organises policy development and monitors the
for including qualifications into an NQF,                                     implementation of the framework among the other
underpinning quality assurance arrangements                                   action lines of the Bologna Process. It is supported
and the requirement to demonstrate a clear                                    in this work by the Bologna Secretariat and various
and demonstrable link between the national                                    working groups. The Council of Europe serves
qualifications levels and the EQF are among                                   as the liaison between the BFUG and the EQF
the most important ones. The criteria also help                               Advisory Group.
structure the referencing reports that countries
present to the EQF Advisory Group.18 The EQF                                  EQF implementation at national level
Advisory Group discusses these reports and                                    in European countries
provides feedback to the presenting countries.
The presentation and discussion of the reports are                            The EQF has been the main catalyst for the rapid
intended to improve understanding of qualification                            developments and implementation of learning
systems among countries.

16 Topics recently addressed include international qualifications, levelling qualifications, writing learning outcomes, EQF level 5 qualifications and
   master craftsperson qualifications.
17 See Criteria and procedures for referencing national qualifications levels to the EQF, https://ec.europa.eu/ploteus/sites/eac-eqf/files/criteria_
   en.pdf
18 Already presented referencing reports are available on https://ec.europa.eu/ploteus/documentation
19 Networks of academic recognition centres (the European network of information centres (ENIC) and the National academic recognition
   information centres (NARIC), http://www.enic-naric.net
20 For example the Yerevan meeting in 2015 formally included the short cycle in the QF-EHEA, using the Dublin Descriptor previously referred to
   in the EQF.

22        Comparative Analysis of the European Qualifications Framework and the New Zealand Qualifications Framework: Joint Technical Report
You can also read
NEXT SLIDES ... Cancel