Managing National Assessment Report 2013 Sir Edmund Hillary Collegiate Senior School

Managing National
    Assessment
      Report

         2013

  Sir Edmund Hillary
Collegiate Senior School

Managing National Assessment Report

Sir Edmund Hillary Collegiate Senior School

6 September 2013

Introduction
The purpose of the Managing National Assessment (MNA) external review is:
  •    to help schools achieve valid, fair, accurate and consistent internal assessment
       according to the requirements of the Assessment and Examination Rules for
       Schools with Consent to Assess 2013 (Assessment Rules); and
  •    in combination with the most recent Education Review Office report, to ensure
       that schools are meeting the requirements of the Consent to Assess Against
       Standards on the Directory of Assessment Standards Rules 2011 and its
       Guidelines (CAAS Guidelines) in order to maintain their consent to assess.
This process is a partnership between schools and the New Zealand Qualifications
Authority (NZQA) to ensure that assessment systems remain effective and internal
assessment decisions are valid.

The MNA review has two components:
  1. The annual external moderation of assessment materials for an NZQA-selected
     sample of internally assessed standards across the curriculum and teacher
     grade judgements on the student work identified by the school’s random-
     selection process.
  2. An external review of each school’s assessment systems at least once every
     four years.

This report summarises NZQA’s evaluation of how effectively:
  •    senior management has addressed issues identified during the previous MNA
       review on 13 May 2010
  •    the school manages assessment for national qualifications
  •    the school manages internal and external moderation
  •    the school manages and makes use of assessment-related data
  •    the school maintains the currency of assessment policy and procedures, and
       communicates them to staff, students and families.

In preparing this report, the School Relationship Manager (SRM) met with the
Principal’s Nominee and the Heads of Department (HODs) for Geography, History,
Information and Communication Technology, and Science. The school also provided
the SRM with pre-review information and the staff assessment procedures guides.

There was a report-back session with the Principal, and the Principal’s Nominee at
the end of the visit to highlight strengths and weaknesses, with suggested strategies
for good practice and to agree on recommended actions.

                                                         2
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

2010 Managing National Assessment Report (CAAS Guidelines 3v)
Evidence was sought that action items in the previous MNA report have been
effectively addressed.

The majority of the action items in the 2010 MNA report have been addressed.

To complete its response to the identified actions, the school should:
     •    reintroduce a designated student assessment handbook
     •    ensure a randomly selected sample of student work is filed for every
          standard assessed and therefore available for eventual external moderation.

Assessment Practice (CAAS Guidelines 2.5v-vii, 2.6, 3v)
How does the school’s assessment practice allow for the effective management of
assessment for national qualifications?

Meeting students’ assessment needs The school is close to Manukau Institute of
Technology (MIT). With careful planning, the school provides many students with
access to specialised or higher level courses that reflect their interest, ability or future
aspirations. In doing so, the school makes effective use of the flexibility that the New
Zealand Qualifications Framework (NZQF) offers.

In one course, some students have the scope to fulfil part of the requirement for
NZQA qualifications, or other NZQF qualifications, up to Level 5 of the NZQF. They
are fully supported by the teacher, who helps them enrol in MIT courses, makes a
school computer suite available to them outside of class time, monitors their progress
towards qualifications, and advises them on their future course options.

Students in the course work at their own pace and they are assessed when they are
ready; this having been determined by the competency they display in completing
assignments and course work. Students who display less aptitude or interest access
a core suite of standards that enables them to also work and be assessed at their
own pace.

The schools links with MIT for courses in Engineering, Information and
Communication Technology, and Travel and Tourism. The school facilitates this
arrangement through its timetable so that students can work off campus for an
extended period of time without it affecting their attendance at their school-based
courses.

As a next step, the school should review existing courses offered in partnership with
MIT against the newly introduced Vocational Pathways and identify any areas where
making a slight change or realignment could enable students to also meet the
requirements for one of the five Vocational Pathways. In conjunction with existing
arrangements with MIT, this will smooth the transition pathway for more students
from secondary education to tertiary education, training or employment.

Literacy and numeracy The majority of Year 11 students are working below the
expected curriculum level for literacy, numeracy and writing. The school manages
this for numeracy by making extensive use of the Work and Study Skills numeracy
standards. For three of the five Year 11 Mathematics courses, these standards form
the core of students’ assessment programmes.
                                                         3
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

The school has not yet, however, made any use of the Work and Study Skills literacy
standards and it is recommended that the school review this decision for 2014 and
use them with students who are at risk of not meeting the NCEA literacy requirement
from achievement standards.

Student entries The school encourages teachers to arrange their course
assessment programme so students can access at least 16 NZQF credits from each
of their five subjects. From experience, the school has identified this to be a realistic
and manageable workload for most students. Students who are able to manage more
than 16 credits can opt to enter an additional externally assessed standard, and one
teacher reported that almost half of her students choose to do this.

For Mathematics, however, the majority of Year 11 students do not possess sufficient
numeric skills to access the Mathematics achievement standards and their course
focuses on attaining the 10 credits available from the Work and Study Skills
numeracy standards. As a result, in 2012, 50 per cent of Year 11 students were
entered in fewer than 80 credits and this made them ineligible to attain NCEA Level 1
in the current year. The school has carefully considered the trade-off between
making appropriate student entries and making sufficient entries to provide students
with access to qualifications in the current year and has concluded that its current
practice is the most appropriate course of action. This review acknowledges the
careful consideration that the school has given to this matter.

Evidence of achievement Teachers who participated in the review described some
of the ways they assess students and the assessment conditions that students work
within to craft their evidence. For practical reasons, including effectively managing
authenticity, students are typically provided with formal conditions in which to
complete assessments.

It is recommended that the school work with HODs and teachers to consider how
they can gather evidence of achievement from a broader range of settings. Teachers
should continue to be encouraged to reflect on how draft evidence, or that from
practice or other opportunities, can be recorded and used as evidence to report a
grade if it represents a student’s best effort.

In some cases, teachers may be able to develop ways to record and use a greater
range of non-written evidence where standards allow for this, but where students
typically submit written work because of convenience or by convention. The scope for
students to record verbal evidence while they undertake their investigation in the
community, rather than complete it in written form upon their return to class, was
enthusiastically discussed with one teacher.

Resubmission Teachers are provided with clear information about the parameters
to follow if they offer students an opportunity to identify and correct a minor error that
they have made in their originally submitted work. For one subject, students work
outside these parameters because they are provided with an extended period of time
to improve their work. This practice should be changed so that it aligns with the
school’s guidelines for students to be given an opportunity to ‘rapidly resolve’
problems with their work.

Derived grades Teachers reported that they annually purchase their practice
examination materials from commercial sources, critique them to ensure they are fit
for use with their students and remind students of the importance of these
examinations in the event that they need to apply for a derived grade at the end of
                                                         4
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

the year.

In 2012, there were no students who were unable to sit external examinations or had
their performance impaired because of exceptional circumstances beyond their
control.

Special assessment conditions In 2012 and 2013, the school did not make an
application for special assessment conditions (SAC) for any students. The school is
currently investigating alternatives for funding and testing students who may be
eligible for SAC.

Identified actions
NZQA and senior management agree on the following actions to improve the
management of assessment for national qualifications. Senior management
undertakes to:
    •    consider reviewing existing courses offered in partnership with MIT against
         the newly introduced Vocational Pathways
    •    review its intended use of the Work and Study Skills literacy standards for
         2014
    •    work with HODs and teachers to consider how they can gather evidence of
         achievement from a broader range of settings or use of non-written evidence
    •    ensure resubmission opportunities offered to students in one subject require
         them to rapidly resolve issues with their work.

Moderation (CAAS Guidelines 2.6, 3v)
How effectively does school internal and external moderation assure assessment
quality?

Internal moderation processes The MNA review undertaken in 2010 reported that
the teachers who participated in the review took pride in their internal moderation
processes, used the Internal Moderation Cover Sheet to record compliance, had
established networks to verify grade judgments and to improve and maintain their
understanding of the national standard and approved of the school’s closer
monitoring of the process.

The HODs who participated in this review tabled comprehensive evidence to confirm
that the effective practices that were evident in 2010 continue to be the norm. They
ensure that a ‘green’ assessment folder is developed and maintained each year for
every internally assessed standard offered in their department. They have effective
processes for ensuring that a sample of marked student work is verified by a
colleague with suitable experience with the standards in question.

For subjects with an exemplary external moderation history, verification of a sample
of grades is undertaken within the department. For others, HODs arrange for an
external colleague to verify a sample of marked student work.

Materials for the current year are well stored and available for review by the PN when
they are called for at the end of the year or for external moderation the following year
if selected by NZQA.

                                                         5
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

The PN’s oversight of internal moderation extends from reminding staff at the start of
the year of the school’s procedures and expectations through to her receiving a copy
of each cover sheet and sighting the related assessment folder before the end of the
year.

Assessment results are entered on the student management system and
electronically reported to NZQA once the PN is satisfied that the teachers have
effectively complied with the school’s written procedures and taken all reasonable
steps to ensure that the grades being reported are at the national standard.

External moderation The school’s most recent submission of materials for external
moderation was in July 2013 and involved the school submitting assessment
materials for 58 standards selected by NZQA from across a range of subjects and
courses. The required materials were submitted for all but one of the selected
standards.

For approximately one-third of the standards, moderators identified one or more
minor changes that must be made to the assessment materials before they are used
again.

For eight standards, moderators agreed with half or less than half of teacher grade
judgments. These standards are across a range of subjects and, in most cases, are
for recently registered standards for which teachers are still developing an
understanding of the national standard.

The school uses its external moderation results as a gauge of the effectiveness of
departmental internal moderation processes. For one subject, external moderation
results for 2012 prompted the school to:
     •    immediately change the arrangements they had made to verify a sample of
          grade judgments
     •    request that NZQA externally moderate a significant number of standards for
          one subject in 2013.
The results for external moderation for 2013 identified another subject that required
the same action. The school is commended for these responses. They demonstrate
its increasing capacity to make an effective and timely response to an identified risk.

Response to external moderation The PN reviews all of the external moderation
reports as they are received so as to be aware of all of the issues that external
moderators bring to the school’s attention. HODs follow up those for their department
and the action plan developed with the relevant teacher is recorded on the
Moderation analysis sheet.

One HOD also uses a Log of Action for any standard with significant external
moderation concerns as a way to record the changes made to assessment materials
as well as conversations with external colleagues and other actions over successive
years. This helps ensure assessment materials and assessor judgements are at the
national standard. This enables the HOD to evaluate the effectiveness of the
department’s response to an identified risk and apply the most effective strategies to
future situations.

The PN uses the results of external moderation to inform her decisions about staff
members who must be given priority access to professional development, including
attendance at Moderation Best Practice Workshops or support from Team Solutions.
                                                         6
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

The PN closely monitors the assessor/moderator agreement rate for the sample of
student work submitted for external moderation. As a next step, it is suggested that
she use the information from each Internal Moderation Cover Sheet to monitor the
assessor/verifier agreement rate for the sample of student work verified for internal
moderation. This will further inform her about the effectiveness of the school’s
internal moderation procedures and convey to teachers the parity of importance
between internal and external moderation.

Identified action
NZQA and senior management agree on the following action to improve the school’s
internal and external moderation to assure assessment quality. Senior management
undertakes to:
    •    consider using information from each Internal Moderation Cover Sheet to
         monitor the assessor/verifier agreement rate for the sample of student work
         verified for internal moderation.

Data (CAAS Guidelines 2.6, 2.7)
How effectively does the school manage and make use of assessment-related data?

Accuracy of assessment-related data Students are provided with a monthly
printout of their entry and results information so they can check their accuracy.

Tutor teachers encourage students to use their NZQA Learner login to check that
their results are accurate and complete. In 2012, two-thirds of students registered for
Learner login and 75 per cent of these used the facility at least once throughout the
year to access their results.

The school regularly submits data files to NZQA during the year and the PN follows
up on alerts or error messages that are signalled back to the school as each file is
processed.

Entries in externally assessed standards are well managed by the school before the
1 September deadline. In 2012, there were 41 requests made after this date for
additional examination materials for students it entered late in externally assessed
standards. The majority of the late entries were single entries in individual standards
and some of these can be attributed to students opting late in the year to attempt an
additional examination standard.

The PN effectively oversees the end of year process of entering provisional and final
grades for non-examination externally assessed standards in Cook Islands Māori,
Technology and Visual Arts.

Use of assessment-related data Students’ monthly printout of their entries and
results are used by tutor teachers and Deans to predictively identify students who are
at risk of falling short of the number of credits required to attain a national
qualification.
At the start of each year, teachers undertake an analysis of student achievement
data for the classes they taught the previous year. The school has developed a
template for school-wide use. For each standard, it requires teachers to analyse and
interpret the data, identify what they plan to do differently and set student
achievement targets for the coming year and beyond. Those sighted during the
                                                         7
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

review indicate that the process is thoughtfully completed and that assessment-
related data is effectively used to inform teaching and assessment practice for the
following year.

Data about the attainment of NCEA qualifications in 2012 shows that 50 per cent of
Year 11 students were entered in fewer than 80 credits and ineligible to attain NCEA
Level 1 until the following year. The school has identified the reasons for this and, as
discussed in the Assessment Practice section of this report, has evaluated its 2012
entries as appropriately reflecting “individual student’s prior knowledge, strengths,
interests and needs” as the school’s Curriculum Assessment and Reporting Policy
requires.

Privacy and password security The school has effectively engaged with the
Education Sector Authentication and Authorisation (ESAA) process so that teachers
have on-going access to the NZQA generated data and statistical reports that are
available through the school’s NZQA website login. The PN is responsible for
maintaining the list of teachers with ESAA access to school data.

Identified action
No action was identified as needed to improve the management and use of
assessment-related data.

Communication (CAAS Guidelines 2.4i, 2.6, 3v)
How effectively does the school maintain the currency of assessment policy and
procedures, and communicate them to staff, students and families?

Assessment policy and procedures The following documents were provided for
this visit:
     • Curriculum, Assessment and Reporting (Board of Trustees Policy)
     • Assessment and Quality Assurance, 2013 (staff handbook)
     • a sample of course outlines from departments that participated in the review.

Communication of policies and procedures to staff The staff handbook is
comprehensive and well organised. It is made available to all staff and its layout and
use of plain language makes it a useful document for them to refer to. The PN
receives EmaiLink updates from NZQA throughout the year and forwards pertinent
information to the relevant staff. The PN ensures that teachers comply with key dates
such as for making entries in externally assessed standards, submitting materials for
external moderation and entering grades for non-examination externally assessed
standards.

Communication of policies and procedures to students All of the course
outlines sighted contain a general information section which informs students of the
school procedures for managing issues such as appealing an assessment-related
decision, authenticity, further assessment and missed and late and assessments. It is
recommended that the school change this for 2014 and reintroduce a designated
student handbook so as to provide students with a single copy of this information. It
should also include information about the NCEAs, certificate and course
endorsement, payment of fees, Learner login and University Entrance.

                                                         8
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

Current course outlines also provide students with information about the standards
that are on offer. The effectiveness of the template can be improved by teachers
flagging standards that contribute towards the NCEA and/or University Entrance
literacy or numeracy requirement or a vocational pathway. For each standard, there
should also be an indication of whether there is an opportunity for resubmission or
further assessment.

Identified actions
NZQA and senior management agree on the following actions to maintain the
currency of assessment policy and procedures, and communicate them to staff,
students and families. Senior management undertakes to:
       • develop a designated student handbook for 2014 and remove general
         assessment-related information from course outlines
       • update the template in existing course outlines about the standards that are
         on offer.

                                                         9
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013

Summary
Sir Edmund Hillary Collegiate Senior School is commended for its on-going
commitment to ensuring the credibility of assessment for national qualifications.

The PN is responsible for assuring the quality of assessment undertaken in the
school. Heads of Department oversee this for courses and standards in their
department. Those who participated in this review provided evidence that confirmed
their understanding and application of the school’s assessment procedures, as well
as their capacity to make an appropriate response to a potential or evident risk.

In partnership with Manukau Institute of Technology, the school makes effective use
of the flexibility that the New Zealand Qualifications Framework offers. Further
investigation of Vocational Pathways will smooth the transition of more students from
secondary education to tertiary education, training or employment.

In 2012, 50 per cent of Year 11 students were entered in 2012 in fewer than 80
credits and this made them ineligible to attain NCEA Level 1 in the current year. The
school has carefully considered the trade-off between making appropriate student
entries and making sufficient entries so as to provide students with access to
qualifications in the current year.

Internal moderation is effectively undertaken by teachers and monitored by Heads of
Department and the Principal’s Nominee. The school uses its external moderation
results as a gauge of the effectiveness of departmental internal moderation
processes. In response to identified issues, the school makes an effective and timely
response.

The school regularly submits data files to NZQA during the year and the PN follows
up on alerts or warning messages that are signalled back to the school once each file
is processed. Teachers effectively analyse data to inform teaching and learning and
to set student achievement targets for the coming year and beyond.

The school has undertaken to reintroduce a designated student handbook for 2014
and this will enable teachers to remove the general student-related assessment
procedures from course outlines.

The Principal’s Nominee ably leads the staff and effectively supports Heads of
Department. She is fully supported by the Principal.

The 2013 Managing National Assessment review identified no significant issues that
prevent the school from meeting the requirements of the Consent to Assess Against
Standards on the Directory of Assessment Standards Rules 2011 and its Guidelines.

Based on the outcome of this assessment systems check, it is anticipated that the
next Managing National Assessment review will be conducted within four years.

                                                     10
Sir Edmund Hillary Collegiate Senior School MNA Report
September 2013