EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME

 
EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME
Research article
Geosci. Commun., 4, 147–168, 2021
https://doi.org/10.5194/gc-4-147-2021
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Evaluating participants’ experience of extended interaction with
cutting-edge physics research through the PRiSE “research in
schools” programme
Martin O. Archer1,a , Jennifer DeWitt2,3 , Charlotte Thorley4 , and Olivia Keenan5
1 School of Physics and Astronomy, Queen Mary University of London, London, UK
2 Instituteof Education, University College London, London, UK
3 Independent Research and Evaluation Consultant, UK
4 Public Engagement and Involvement Consultant, UK
5 South East Physics Network, London, UK
a now at: Space and Atmospheric Physics, Department of Physics, Imperial College London, London, UK

Correspondence: Martin O. Archer (martin@martinarcher.co.uk)

Received: 21 July 2020 – Discussion started: 31 July 2020
Revised: 28 January 2021 – Accepted: 24 February 2021 – Published: 8 April 2021

Abstract. Physics in schools is distinctly different from, and   1   Introduction
struggles to capture the excitement of, university research-
level work. Initiatives where students engage in independent     Research, policy, and practice all agree that participation in
research linked to cutting-edge physics within their school      science, technology, engineering and mathematics (STEM)
over several months might help mitigate this, potentially fa-    needs to be increased and widened (e.g. Campaign for Sci-
cilitating the uptake of science in higher education. How-       ence and Engineering, 2014), with these issues being par-
ever, how such initiatives are best supported remains unclear    ticularly acute for the subject of physics (e.g. Murphy and
and understudied. This paper evaluates a provision frame-        Whitelegg, 2006; IOP, 2014). Physics as a field has become
work, Physics Research in School Environments (PRiSE),           within society strongly aligned with intelligence/cleverness,
using survey data from participating 14–18-year-old students     masculinity and whiteness, all of which can dissuade school
and their teachers to understand their experience of the pro-    students (even those highly enthusiastic about the subject)
gramme. The results show that PRiSE appears to provide           from pursuing it further and thereby showing inequitable ef-
much more positive experiences than typical university out-      fects on those from under-represented backgrounds (Archer
reach initiatives due to the nature of the opportunities af-     et al., 2020). Some of these issues arise from practices
forded over several months, which schools would not be           in school-level physics education. Debarring and gatekeep-
able to provide without external input. The intensive sup-       ing of physics based on attainment (disproportionately so
port offered is deemed necessary, with all elements appearing    compared to other subjects) simply feeds the alignment of
equally important. Based on additional feedback from inde-       physics with cleverness and can make even high-attaining
pendent researchers and engagement professionals, we also        students’ confidence in the subject precarious. Teachers and
suggest the framework could be adopted at other institutions     the school environment often (even unconsciously) reinforce
and applied to their own areas of scientific research, some-     stereotypes about physics and physicists that are patterned
thing which has already started to occur.                        by biases. Curriculum practices in physics often teach over-
                                                                 simplifications at younger ages which are later completely
                                                                 reconceptualised without being presented as refinements to
                                                                 a model, making students perceive the simpler versions as
                                                                 “lies”. Furthermore the general deferment of “interesting”
                                                                 physics in the curriculum produces a disconnect between

Published by Copernicus Publications on behalf of the European Geosciences Union.
EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME
148                                                                M. O. Archer et al.: Physics research in school environments

“school physics” and “real physics”, i.e. the cutting-edge re-       ternal support, and funding/costs. It was noted that such pro-
search undertaken by professional physicists, making contin-         grammes place demands on time and money beyond stan-
ued participation in physics education something of a “test          dard provisions for all stakeholders, on the skills required
of endurance”. These concerns are further reflected in re-           by teachers and other adults involved, and on the supporting
sults from national surveys. While 20 % of 16–18-year-old            infrastructure. For successful projects Dunlop et al. (2019)
physics school students in the UK aspire towards a physics           recommend that students should be given the freedom to de-
degree and 80 % aspire towards STEM more broadly (Well-              vise a research question, have ownership over their own data
come Trust, 2017), only 9.7 % and 59.3 % actually go on              analysis and decision-making, and be given access to experts
to study either physics or STEM respectively (McWhinnie,             in their project work. Broadly there are two distinct formats
2012). These constitute odds ratios for aspirations vs. desti-       of independent research projects.
nations of 2.3 and 2.7, both of which are considerable. All
of these issues raised cultivate and contribute to reproducing          – Those associated with dedicated out-of-school events
inequitable, and low overall, patterns of participation within            of only a few weeks’ duration such as intern-
physics (Archer et al., 2020).                                            ships/apprenticeships (e.g. Nuffield Research Place-
   Davenport et al. (2020) suggest that for STEM subjects                 ments in the UK, Cilauro and Paull, 2019; Raising Inter-
in general an intervention approach that sustains and sup-                est in Science and Engineering in the USA, Stanford Of-
ports science identity is most appropriate for students in late           fice of STEM Outreach, 2020), summer schools/camps
secondary/high-school education in the context of their ed-               (e.g. the International Astronomical Youth Camp run
ucational journey. However, in the case of physics specifi-               across Europe and parts of Africa, Dalgleish and Veitch-
cally, Archer et al. (2020) comment that existing interven-               Michaelis, 2019), or science competitions/fairs (particu-
tions based on simply enthusing, inspiring and informing                  larly prevalent in the USA, e.g. Yasar and Baker, 2003).
students about physics will not significantly change uptake             – Those undertaken within school itself over the course
or diversity in post-compulsory physics. While they advo-                 of several months to a year, either in class or sup-
cate for widespread changes in science education policy and               plemented with time in after-school clubs (e.g. an
practice, both at school and university levels, they note that            after-school mechatronics project in Taiwan, Hong
if interventions are also used they need to fundamentally ad-             et al., 2013, class-based biology project in Singapore,
dress the problematic processes and practices present within              Chin and Chia, 2010, or various CarboSchools climate
both physics teaching and physics as a field generally.                   change projects between research institutes and schools
   The stark differences between school, university, and pro-             across seven European countries, Dijkstra and Goed-
fessional science practices have long been noted – while re-              hart, 2011).
search is one of the main activities of professional scientists,
it is quite removed from how science is taught in schools,           We do not discuss the former here as they are necessarily
with some arguing that science education is not “authen-             limited in reach, catering only to heavily bought-in individu-
tic” in this respect (e.g. Hodson, 1998; Braund and Reiss,           als (i.e. typically one to three students; Paull and Xu, 2017)
2006). Indeed, Yeoman et al. (2017) report that school stu-          from any given school. Most independent research projects
dents are largely unaware of what research actually is, find-        based within schools are not linked to current cutting-edge
ing a disconnection between “research as information gath-           and novel scientific research topics or questions. However,
ering” and the “research question”, and in general have little       relatively recently so-called “research in schools” projects
opportunity to set their own research questions within their         have emerged, which do provide students with experiences
school environment. Independent research projects, which             of genuine contemporary STEM research within their own
provide extended opportunities for students to lead and tackle       school environment over several months. While several citi-
open-ended scientific investigations (not simply literature re-      zen science projects have also run within schools and aim to
views or essays, e.g. Conner, 2009; Corlu, 2014), may be             help participants learn about current science and to experi-
one way of exposing students to “real science”. These align          ence the scientific research process, these are typically sec-
with established international pedagogical initiatives such as       ondary aims since they primarily concern a single (or small
“inquiry-based science” (e.g. Minner et al., 2010), “problem-        number) of well-defined science questions which will be as-
based learning” (e.g. Gallagher et al., 1995), and “authen-          sisted through developed citizen science protocols (Bonney
tic science” (e.g. Braund and Reiss, 2006). A survey of              et al., 2009, 2016; Shah and Martinez, 2016). This contrasts
such projects across 12 countries (Australia, Ireland, Israel,       with independent research projects and thus also “research
Netherlands, New Zealand, Qatar, Singapore, Spain, Taiwan,           in schools”, where positively affecting the participants is the
Turkey, UK, and USA), however, found them to be rare glob-           primary concern and the projects are necessarily open-ended.
ally and only sometimes supported by mentors from univer-            Nonetheless, the different approaches can have some over-
sity/industry (Bennett et al., 2016, 2018). This review found        lap, and indeed some projects denoted as citizen science,
considerable variability in the nature of independent research       such as the “curriculum-based” projects based in the USA
projects, such as their focus, delivery/provision models, ex-        described by Bonney et al. (2016), might perhaps be better

Geosci. Commun., 4, 147–168, 2021                                                         https://doi.org/10.5194/gc-4-147-2021
EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME
M. O. Archer et al.: Physics research in school environments                                                                 149

framed as “research in schools”. “Research in schools” pro-        personal communication, 2020), with IRIS itself being the
grammes appear at present to be most prevalent in the UK,          main point of contact for schools. With a recent change
and we are aware of three featuring projects in the physical       of staff at IRIS in late 2019 has come a reformulation of
sciences (outside of that at Queen Mary University of Lon-         how they classify their projects. “Seed” projects are for new
don, QMUL, which forms the subject of this paper).                 schools, are the most straightforward, and receive the most
    HiSPARC (High School Project on Astrophysics Research          support from IRIS, though it is not clear what form that takes.
with Cosmics) is a scintillator–photomultiplier cosmic ray         “Sprout” projects are more advanced, seeing students carry-
detector project originating in the Netherlands at Radboud         ing out more complex activity to assist scientists with their
University in 2004, which has subsequently been adopted            research questions, though how this collaboration operates is
by other Dutch (Amsterdam, Eindhoven, Groningen, Leiden,           not specified. “Grow” projects are where students have pro-
Nikhef, Twente, Utrecht) as well as UK (Bath, Bristol, Birm-       posed their own research questions, either independently or
ingham, Sussex) and Danish (Aarhus) universities (Colle            using IRIS resources, with IRIS merely providing advice in
et al., 2007; van Dam et al., 2020; HiSPARC, 2018). Many           producing posters, talks, or papers as well as opportunities to
of these universities operate a tiered membership scheme for       present.
schools: “Gold” enables schools to buy their own detector             ORBYTS (Original Research By Young Twinkle Scien-
(GBP 5,500 in the UK); “Silver” is a detector rental scheme        tists), based at University College London, was piloted from
(GBP 300 p.a. plus an installation fee) with the contract spec-    January 2016 and is nominally based around the Twinkle
ifying if they do not participate the detector will be collected   mission but has expanded into other research areas since
with an additional fee; and “Bronze” membership (GBP 200           (Sousa-Silva et al., 2018; ORBYTS, 2019). A select group
p.a.) gives schools access to HiSPARC data but not their own       of students (with an imposed limit of four to six) from
detector. Schools signing up for “Silver” or “Bronze” mem-         each school undertakes fortnightly meetings with early ca-
bership are contractually obliged to generate funding to up-       reer researchers (either PhD students or post-docs) through-
grade to “Gold”. While the “Gold” membership fee covers            out their project aiming to achieve, where possible, publish-
the costs of the detector and installation, the other member-      able scientific results (McKemmish et al., 2017; Chubb et al.,
ships are to ensure that schools make a commitment to work-        2018; Holdship et al., 2019). Teachers, while present, are not
ing with the university (Jaap Velthuis and Maria Pavlidou,         typically actively involved in these sessions, and students
personal communication, 2016; National HE STEM SW,                 tend to do little independent work outside of the sessions
2012). It is not possible to compare how these schools go          (William Dunn, personal communication, 2018). The con-
about project work and how much support they are given by          tent of the projects changes each year to align with the re-
participating universities, which may vary by institution, as      searchers’ current focus, with them typically working with
at the time of writing HiSPARC has not published any re-           only one school per year each. PhD researchers are paid for
views of their processes or evaluation.                            their (preparation, travel, and session) time with funds from
    IRIS (Institute for Research in Schools) is a UK char-         independent schools, who pay not only for their school, but
ity formally launched in March 2016 (IRIS, 2020), build-           also for enabling an additional school from a lower socio-
ing on the previous CERN@School project conceived in               economic background to take part.
2007 (Whyntie et al., 2016; Parker et al., 2019). While IRIS’         It is clear that there is currently a lack of published de-
projects cover all the sciences, current physics projects in-      tails and evaluation on provision within the emerging area
clude the aforementioned CERN@School, Higgs Hunters                of “research in schools”. This paper therefore explores these
(Barr et al., 2018), LUCID (Furnell et al., 2019; Hatfield         aspects applied specifically to the “research in schools” pro-
et al., 2019), and Webb Cosmic Mining (in preparation for          gramme of QMUL’s School of Physics and Astronomy. This
the James Webb Space Telescope). They have rapidly ex-             was piloted between 2014 and 2016, as detailed in Archer
panded across the UK since formation, having worked in             (2017), and is now known as Physics Research in School
some capacity with over 230 schools as of 2020. Publica-           Environments (PRiSE, 2020). Section 2 introduces PRiSE’s
tions have provided technical details of their projects and        framework, which is then evaluated in Sect. 4 in terms of par-
case studies of some students’ successes within them, in-          ticipating students’ and teachers’ experience. We also briefly
cluding a few examples of resulting peer-reviewed scien-           discuss how the PRiSE approach has been received by the
tific work; however, the exact provision/delivery model im-        university sector in Sect. 5.
plemented and precisely how project work is supported are
not fully explored in the available literature. IRIS aims to
develop “teacher scientists”, teachers that identify as both       2   PRiSE framework
science teachers and research-active scientists (Rushton and
Reiss, 2019), which suggests a teacher-driven model. While         Physics Research in School Environments (PRiSE) is a
some researchers/academics have designed or consulted on           collection of physics-based “research in schools” projects
some IRIS projects, they appear in general to have little in-      (see Table 1) brought together under a coherent provi-
volvement in supporting students or teachers (Oliver Moore,        sion framework. The programme aims to equip 14–18-year-

https://doi.org/10.5194/gc-4-147-2021                                                    Geosci. Commun., 4, 147–168, 2021
EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME
150                                                                M. O. Archer et al.: Physics research in school environments

old school students (particularly those from disadvantaged           university; therefore, students are not pressured into being in-
backgrounds) with the ability, confidence, and skills to in-         volved. Students and schools can drop out at any point within
crease/sustain their aspirations towards physics or more             the programme with no penalty, which does occur in a minor-
broadly STEM, ultimately enabling them to realise these              ity of schools.
at higher education and thus contributing to increased up-
take and diversity of physics and to some extent STEM                2.1.2   Role of teachers
(cf. Archer et al., 2020). Through working with teachers,
PRiSE also aims to develop their professional practice and           The involvement of teachers at all stages is of paramount im-
build long-term university–school relationships that raise the       portance in terms of delivery and safeguarding. They have
profile of science and mitigate biases/stereotypes associated        helped shape the design of the programme, inform how we
with physics within these schools, generally making them en-         update it each year, and serve as our liaison to schools and the
vironments which nurture and enhance all students’ science           students involved. It is the teachers that decide who projects
capital (cf. IOP, 2014). Our rationale for these particular out-     are offered to within their school, with us simply advising
comes is left to the Supplement. This section summarises the         that the projects should be suitable for all A-Level (16–18-
PRiSE framework, discussing the ethos of the programme               year-old) students as well as high-ability GCSE (14–16-year-
and the roles played by the schools and university as well           old) students (further contextual information on the UK ed-
as outlining the various activity stages, interventions, and re-     ucation system is given in Appendix A). These recommen-
sources which it consists of. In-depth practical details aimed       dations were made based on the basic background knowl-
at practitioners looking to replicate the framework are given        edge required to meaningfully engage with the research. In-
in the Supplement.                                                   variably teachers choose to involve older age groups, with
                                                                     79±1 % of PRiSE students being aged 16–18 (and so far only
2.1     Approach
                                                                     one student below our recommended ages has been involved,
                                                                     being 13–14). Unfortunately, we do not have any specific in-
PRiSE takes the “research in schools” approach to schools            formation on exactly how teachers go about selecting stu-
engagement, whereby students are given the opportunity to            dents. However, the average number of students per school
lead and tackle open-ended scientific investigations in ar-          each year is around 12, which compared to the national av-
eas of current research. Therefore, the PRiSE projects were          erage class size in A-Level physics of 16 (RAE et al., 2015)
developed to transform current scientific research methods,          indicates teachers involve a significant majority (or in many
making them accessible and pertinent to school students so           cases the entirety) of their cohorts in PRiSE. We allow teach-
that they could experience, explore, and undertake scientific        ers to determine how best to integrate the projects within
research themselves.                                                 their school though provide advice on this. We also aim,
                                                                     through our resources and communications, to equip teach-
2.1.1    Ethical considerations
                                                                     ers to manage the day-to-day aspects of the projects without
                                                                     overly burdening them – their role is chiefly one of encourag-
Since the programme intends to influence school students             ing their students to persist, providing what advice they can,
and teachers, a number of ethical considerations have been           and then communicating with the university.
taken into account, following the BERA (2018) guidance for              Teachers’ involvement at all stages also presents opportu-
educational research, with regard to safeguarding and to en-         nities to them for continuing professional development, help-
sure that no harm results. Firstly, to ensure equality of ac-        ing them nurture and cultivate STEM aspirations among stu-
cess to the programme, we do not charge schools to be in-            dents throughout their school (i.e. not just PRiSE students).
volved (cf. Harrison and Shallcross, 2010; Jardine-Wright,           This is implemented informally and integrated within the
2012) and try to provide them with all the physical resources        programme in the form of both bespoke resources and on-
they need for their project, thereby removing potential bar-         going dialogues between teachers and researchers. These are
riers to entry for less resourced schools. Our targeting takes       aimed at enhancing teachers’ knowledge about the under-
into account several school-level metrics (type of school, stu-      lying science and how they link to curriculum-based topics
dents on free school meals, indices of multiple deprivation,         where appropriate, their skills and confidence surrounding
gender balance, etc.) to ensure diversity. We aim for the pro-       current research topics and methods, and their pedagogy in
gramme to be equitable with all schools being offered the            mentoring independent project work.
same interventions/opportunities, taking into account and be-
ing flexible to their specific needs where necessary. We work        2.1.3   Role of the university
with as many schools as we have capacity to do so each year
and do not withhold interventions from any students for the          It was recognised that teachers in general likely will not
purpose of having control groups. The projects are optional          have the skills or experience in research to manage projects
and presented as an opportunity that students can take ad-           without expert assistance (Shah and Martinez, 2016; Bennett
vantage of which will be supported by their teacher and the          et al., 2016, 2018). Therefore, PRiSE was designed to be sup-

Geosci. Commun., 4, 147–168, 2021                                                         https://doi.org/10.5194/gc-4-147-2021
EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME
M. O. Archer et al.: Physics research in school environments                                                                   151

Table 1. A summary of the existing PRiSE projects at QMUL.

      Project                          Abbreviation      Years     Field                      Description
      Scintillator Cosmic Ray          SCREAM          2014–2020   Cosmic rays                Scintillator–photomultiplier
      Experiments into Atmospheric                                                            tube detector usage
      Muons
      Magnetospheric Undulations       MUSICS          2015–2020   Magnetospheric physics     Listening to ultra-low
      Sonified Incorporating Citizen                                                          frequency waves and analysing
      Scientists                                                                              in audio software
      Planet Hunting with Python       PHwP            2016–2020   Exoplanetary transits      Learning computer
                                                                                              programming, applying this to
                                                                                              NASA Kepler and TESS data
      ATLAS Open Data                  ATLAS           2017–2020   Particle physics           Interacting through an online
                                                                                              tool with LHC statistical data
                                                                                              on particle collisions

ported by active researchers equipped with the necessary ex-       be used as criteria for academic promotions (cf. Hillier et al.,
pertise to draw upon in offering bespoke, tailored guidance to     2019). Physics researchers though are largely unmotivated
the students and teachers. Well-defined roles within the uni-      in delivering curriculum content as part of their engagement
versity have been established for each of the PRiSE projects       work, valuing instead aspects relating to their research and
to provide this support.                                           role as a researcher (Thorley, 2016). PRiSE thus also aligns
  – Outreach Officer. Manages the entire programme, in-            with this direction. Ultimately, researcher buy-in is vital to
    cluding university–school relationships, communica-            the delivery of protracted research-based engagement pro-
    tions, intervention/event co-ordination, programme fi-         grammes such as PRiSE.
    nances, and evaluation.                                           It is clear from Table 1 that the topics and activities of
                                                                   current PRiSE projects vary considerably. This suggests that
  – Project Lead. Visible figurehead for the project to            a wide range of fields and project ideas might be able to
    schools, typically an academic member of staff.                adopt the PRiSE framework. How projects have been devel-
  – Researcher. Providing advice and guidance to students          oped has also varied (further explored in the Supplement),
    and teachers throughout the programme. Can be dele-            though we have adopted a pragmatic approach in taking ad-
    gated to or shared with early career researchers.              vantage of opportunities (grant funding, internships, etc.) and
                                                                   adapting existing materials where possible, since creating a
Since the primary focus of PRiSE is (unlike typical citi-          project from scratch is a significant undertaking far beyond
zen science) on the participants rather than the research, re-     what most academics (unfortunately) have the capacity to do
searchers should not consider students contributing to novel       (cf. Thorley, 2016).
research to be their rationale for being involved. Our position
is that it is rather unreasonable to expect investigations that
                                                                   2.2     Structure
are motivated by school students themselves (an established
element of good practice in independent research projects,         PRiSE runs from the start of the UK academic year to
e.g. Dunlop et al., 2019) to be able to make meaningful con-       just before the spring/Easter break, which teachers had in-
tributions to the physics research as a matter of course. We       formed us during the pilot stage is manageable and largely
note that in some exceptional cases PRiSE students’ work           fits around exams/other activities for most (but not neces-
has arrived at promising preliminary results, though these         sarily all) schools (Archer, 2017). The structure has evolved
have required significant follow-up work by professional re-       naturally from the pilot to that shown in Fig. 1.
searchers to transform the results into publishable research
(e.g. Archer et al., 2018) and thus should not be considered
                                                                   2.2.1    Activity stages
the archetype. Instead, researchers are enticed by the possi-
bility of societal impact underpinned by their research. This      Students work in research groups of typically five people
is something which is increasingly called upon from funders        and they are advised to try and work on the project on av-
(e.g. National Coordinating Centre for Public Engagement,          erage for 1–2 h a week. The bulk of this is done outside of
2020) and is notoriously difficult for areas of “blue skies”       regular physics lessons, though some schools integrate the
research such as physics. Furthermore, significant contribu-       projects within their timetabled “science clubs” or required
tion to a coordinated departmental outreach programme can          extra-curricular blocks, whereas other teachers arrange a reg-

https://doi.org/10.5194/gc-4-147-2021                                                      Geosci. Commun., 4, 147–168, 2021
152                                                                 M. O. Archer et al.: Physics research in school environments

Figure 1. Graphic summarising the PRiSE framework including a timeline of the different project activity stages (rectangles), interventions
and stakeholders’ roles within them (rounded rectangles), and resources provided (document shapes). Arrows indicate over what dates
interventions (identified by colour) typically occur.

ular slot for students to work on the projects or leave it up           2.2.2   Interventions
to the students to arrange (though this latter approach often
                                                                        Several different interventions form the structure and support
proves unsuccessful). PRiSE projects are split up into three
                                                                        behind the activity stages.
activity stages (see Fig. 1).
                                                                           – Assignment. The opportunity is advertised to schools via
  – Prescribed work. Given that independent research in                      existing teacher networks and teachers apply to partic-
    STEM is probably unfamiliar to the students, rather than                 ipate in the following academic year. Schools are as-
    expecting them to be able to come up with their own                      signed a project before the summer break.
    avenues of investigation in an unfamiliar research topic               – Kick-off. Typically on-campus event featuring an intro-
    straight away, we instead give them an initial prescribed                ductory science talk, outline of how the project will
    stage of research.                                                       work, and a hands-on workshop.
                                                                           – Visit. Researchers visit schools to mentor students (and
                                                                             their teachers) on their project work in a student-driven
  – Independent project. Groups are encouraged to set their
                                                                             intervention.
    own research questions and undertake different projects
    in the topic area when ready. This enables every group                 – Webinars. Drop-in online sessions similar to school vis-
    to explore something different so that students gain a                   its but allowing students and teachers to gain further
    sense of independence and ownership of their work.                       support.
                                                                           – Ad hoc. Further asynchronous support via email as re-
                                                                             quired.
  – Writing up. Near the end of the project students produce
    either a scientific poster or talk to be presented at an               – Comments. Students are offered the opportunity to re-
    annual conference.                                                       ceive comments on their draft slides or posters.

Geosci. Commun., 4, 147–168, 2021                                                              https://doi.org/10.5194/gc-4-147-2021
M. O. Archer et al.: Physics research in school environments                                                                153

  – Conference. Students present the results of their projects     However, PRiSE’s efficiency in researcher time is reflected
    as oral or poster presentations at a special conference        in its reach as shown in Table 2, demonstrating the model
    at the university, attended by researchers as well as the      now serves around 30 schools per year having reached a to-
    students’ teachers, peers, and family.                         tal of 67 schools and over 1300 students with the direct in-
                                                                   volvement of 88 teachers as of 2020. We note that a minor-
Stakeholders’ roles within these interventions are given in        ity of schools do not complete the full programme and oth-
Fig. 1, with photos depicting some of them displayed in            ers do not return for subsequent years; however, we do not
Fig. 2. All stages of the programme and the processes in-          provide a full analysis of the types of schools involved and
volved are communicated to teachers via email to pass on to        their retention within the programme in this paper. Compar-
their students.                                                    ing the number of schools to other physical science “research
                                                                   in schools” projects/programmes, University of Oxford re-
2.2.3     Resources                                                searchers have interacted directly with only 14 students from
                                                                   five schools through their Higgs Hunters IRIS project which
To enable the students to take part in PRiSE, the students and
                                                                   commenced in 2016 (Oliver Moore, personal communica-
teachers are also provided with numerous resources. While
                                                                   tion, 2020, though we note information regarding other IRIS
some projects require specific equipment, data and software,
                                                                   projects is not available), ORBYTS (2019) reports collab-
here we discuss more common types of resources across the
                                                                   orating with 17 schools since 2016, and HiSPARC (2018)
different projects as shown in Fig. 1.
                                                                   discloses 140 schools since 2004 across their network of 13
  – Project poster/flyer. Given to teachers to help advertise      Dutch, Danish, and UK universities (van Dam et al., 2020),
    projects in their school.                                      i.e. around 11 per institution. Therefore, the reach through
                                                                   the PRiSE framework by a single university department is
  – Project guide. Each project has a student guide cov-           considerable compared to the rest of the sector.
    ering an introduction to the research field, back-
    ground physics/theory, an explanation of the equip-
                                                                   3     Methods
    ment/data, discussion of analysis techniques, details of
    the initial prescribed activity, suggested research ques-      To determine the perceived value of PRiSE’s approach with
    tions/methods for independent research, and links to           its key stakeholders, namely participating students and teach-
    other sources of information. Teachers are provided            ers as well as those across the wider university sector, we
    with the same guide but with extra guidance.                   have maintained regular collection of evaluative data (cf.
  – Project webpage. These showcase anonymised exam-               Rogers, 2014, and references therein) via various surveys
    ples of good-quality talks/posters that previous students      which we detail here. These data underpin our understand-
    have produced and provide any links or videos relevant         ing of PRiSE and have been collected securely to protect all
    to the project.                                                participants, in compliance with GDPR and in line with the
                                                                   BERA (2018) guidelines for educational research.
  – “How to” guide. General articles applicable to most re-
    search projects, such as producing scientific talks and        3.1    Instruments and participants
    posters.
                                                                   We gathered feedback from participating students and teach-
                                                                   ers via paper questionnaires handed out at our student con-
2.3     Scalability
                                                                   ferences each year. The only exception to this was in 2020,
PRiSE’s framework attempts to find a balance between the           where online forms were used due to the COVID-19 pan-
(necessarily competing) reach and significance of the inter-       demic causing that year’s conference to be postponed. The
actions. For example, an academic staff member acting as           questionnaire method was chosen so as to gather data from
both project lead and researcher within PRiSE can support          as wide a range of students and teachers as possible, respect-
four schools’ participation (∼ 50 students), taking around 8 h     ing the limited time/resources of all involved (both on the
over the course of 6 months (cf. Fig. 1). Using PhD students       school and university sides). For ethics considerations all
or post-docs in the researcher role(s) makes even more effi-       feedback was anonymous, with students and teachers only
cient use of time. In contrast, under the ORBYTS model each        indicating their school (pseudonyms are used here to pro-
early career researcher can support only one school (four to       tect anonymity) and which project they were involved with.
six students) with 10 h of their time. As noted in the introduc-   Students were not asked to provide details of any protected
tion, mentorship from active researchers throughout does not       characteristics (such as gender or race) or sensitive infor-
appear prevalent in other “research in schools” programmes         mation (such as socio-economic background). Both students
at present.                                                        and teachers were informed via an ethics statement on the
   Programmes of repeat interventions with schools will nec-       form that the information was being collected for the pur-
essarily have a smaller reach than various one-off events.         pose of evaluating and improving the programme and that

https://doi.org/10.5194/gc-4-147-2021                                                    Geosci. Commun., 4, 147–168, 2021
154                                                                    M. O. Archer et al.: Physics research in school environments

Figure 2. Photos of various stages of the PRiSE framework: students participating in an on-campus kick-off workshop (a), students inter-
acting during the poster session at a conference (b), a group of students display their prizes won at a conference along with their teacher (c),
and a group presents a talk at a conference (d).

Table 2. Number of schools, students, and teachers involved in PRiSE by academic year as the programme has grown.

                    Academic year                 2014/2015      2015/2016     2016/2017      2017/2018     2018/2019      2019/2020
                                      Schools               1             6            18             29            27             33
         Number per year              Students             20           115           163            310           311            407
                                      Teachers              1             7            25             29            31             38
                                      Schools               1             6            20             39            50            67
         Unique cumulative total      Students             20           135           298            608           919          1326
                                      Teachers              1             7            28             44            63            88

they could leave any question they felt uncomfortable an-                 ered throughout. There is no indication that the respondents
swering blank (this functionality was also implemented on                 differed in any substantive way from the wider cohorts par-
the online form for consistency).                                         ticipating in the programme. While ideally one would also
   The open and closed questions concerning participants’                 gather feedback from schools which dropped out during the
experience of the programme, which varied slightly year to                year, a similar formal feedback process has not been viable
year, are given in Appendix B. While we attempted to col-                 bar in a few cases where only the teachers responded.
lect responses from all participants in attendance, invariably               Feedback from the university sector came from a session
only a fraction did so, yielding results from 153 students and            at the 2019 Interact symposium that presented the challenges
45 teachers across 37 schools. A breakdown of the number                  to STEM outreach practice highlighted by recent educational
of respondents and their schools per year is given in Table 3,            research and the need for deeper programmes of engage-
where the numbers of participants and schools in attendance               ment with young people and then summarised the PRiSE
at our conferences are also indicated. We do not have reliable            approach as one possible example (Archer, 2019). Through-
information on how many students, teachers, and schools                   out this workshop an anonymous interactive online survey
would have successfully completed the programme in 2020                   was used for interactivity and to collect data presented in this
due to the COVID-19 disruption. Students and teachers did                 paper. The survey included both closed and open questions
not always answer all of the questions asked, and hence we                as listed in Appendix C. Attendees were fairly evenly split
indicate the number of responses for each question consid-                between UK university researchers and engagement profes-

Geosci. Commun., 4, 147–168, 2021                                                                 https://doi.org/10.5194/gc-4-147-2021
M. O. Archer et al.: Physics research in school environments                                                                       155

Table 3. Response rates to questionnaires at PRiSE student conferences.

                                      2015            2016                2017             2018           2019    2020
                    Students                  13/26 (50 %)     21/70 (30 %)         46/92 (50 %)   38/97 (39 %)   35/?
                    Teachers    1/1 (100 %)    6/6 (100 %)      6/11 (55 %)          9/16 (56 %)    6/16 (38 %)   17/?
                    Schools     1/1 (100 %)    6/6 (100 %)    11/11 (100 %)         13/15 (87 %)   11/15 (73 %)   19/?

sionals (gauged in-person by attendees raising their hands                    3. Thematic review. Codes are used to generate themes and
when asked), with 19 people participating in the survey and                      identify associated data.
only 7 not doing so. Participants were allocated a unique
number by the online survey itself, which did not distinguish                 4. Application. Codes are reviewed through application to
between researchers and engagement professionals.                                the full data set.

                                                                              5. Analysis. Thematic overview of the data is confirmed,
3.2   Analysis                                                                   with examples chosen from the data to highlight the
                                                                                 themes.
Both qualitative and quantitative approaches were utilised
in data analysis, as the open- and closed-ended questions
present in the questionnaires produced different types of data.           4     Feedback from participants
   For all quantitative data, standard (i.e. 68 %) con-
fidence intervals are presented throughout. For propor-                   In this section we use the feedback from participating stu-
tions/probabilities these are determined through the Clopper              dents and teachers to evaluate the provision offered within
and Pearson (1934) method, a conservative estimate based on               the PRiSE framework, specifically assessing their experience
the exact expression for the binomial distribution, and there-            and the level of support offered.
fore represent the expected variance due to counting statis-
tics only. Several statistical hypothesis tests are used with
                                                                          4.1    Experience
effect sizes and two-tailed p values being quoted, with the
required significance level being α = 0.05. In general we opt             Firstly from 2016 onwards we asked both students (n = 150)
to use nonparametric tests as these are more conservative and             and teachers (n = 42) “Have you been happy with the re-
suffer from fewer assumptions (e.g. normality, interval scal-             search project overall?”, giving options on a five-point Lik-
ing) than their parametric equivalents such as t tests (Hollan-           ert scale, which we coded to the values 1–5. This scale and
der and Wolfe, 1999; Gibbons and Chakraborti, 2011). The                  the results are displayed in Fig. 3, revealing that 91 ± 3 %
Wilcoxon signed-rank test is used to compare single sam-                  of students and 95 ± 5 % of teachers rated their experience
ples to a hypothetical value, testing whether differences in              as positive (scores of 4–5), with only three students giving a
the data are symmetric about zero in rank. When comparing                 negative reaction (scores of 2). Teachers tended to rank this
unpaired samples a Wilcoxon rank-sum test is used, which                  question somewhat higher (their mean score was 4.50±0.09,
tests whether one sample is stochastically greater than the               where uncertainties refer to the standard error in the mean)
other (often interpreted as a difference in medians). Finally,            than students (mean of 4.17 ± 0.05), with p = 0.002 in a
for proportions we use a binomial test, an exact test based on            Wilcoxon rank-sum test. The PHwP project scored slightly
the binomial distribution of whether a sample proportion is               higher (average of 4.59 ± 0.11, p = 8 × 10−4 ) than the over-
different from a hypothesised value (Howell, 2007). For ease              all results with students, whereas ATLAS scored slightly
of reference, further details about the quantitative analyses             lower with both students (3.92 ± 0.09, p = 0.012) and teach-
are incorporated into the relevant sections of the findings.              ers (3.80 ± 0.20, p = 0.017) than their respective means. No
   Qualitative data were analysed using thematic analysis                 obvious trends were present by school.
(Braun and Clarke, 2006). Instead of using a priori codes, the               While suggestive of extremely positive experiences with
themes were allowed to emerge naturally from the data us-                 PRiSE, one also needs to compare these distributions against
ing a grounded theory approach (Robson, 2011; Silverman,                  the typical responses of students and teachers for schools’
2010) as follows.                                                         STEM engagement programmes. We use the results of Ven-
                                                                          nix et al. (2017) as such a benchmark, which surveyed
  1. Familiarisation. Responses are read and initial thoughts             729 high-school students and 35 teachers about 12 different
     noted.                                                               STEM outreach activities in the USA and Netherlands. This
                                                                          comparison reveals that PRiSE seems to be perceived consid-
  2. Induction. Initial codes are generated based on review               erably more positively than usual by both students (bench-
     of the data.                                                         mark average 3.66 ± 0.01, p = 1 × 10−15 in a one-sample

https://doi.org/10.5194/gc-4-147-2021                                                              Geosci. Commun., 4, 147–168, 2021
156                                                                    M. O. Archer et al.: Physics research in school environments

                                                                              I learned so much! I would recommend it to all
                                                                              the younger kids at my school! (Student 111, St
                                                                              Trinians, SCREAM 2019)
                                                                              I definitely learnt many new and interesting things
                                                                              and it helped me to develop my understanding of
                                                                              particle physics while aiding my A-Level knowl-
                                                                              edge. (Student 140, Jedi Academy, ATLAS 2020)
                                                                           They found the projects’ content and methods interesting
                                                                         (47 responses).
                                                                              It brought my interests in programming, physics,
                                                                              maths and space together. (Student 100, Octavian
                                                                              Country Day School, PHwP 2019)
Figure 3. Distribution of students’ (blue) and teachers’ (green)
                                                                              It has been very interesting to work with actual data
overall happiness with their PRiSE projects. Error bars denote stan-
                                                                              and plan our own research project. (Student 116, St
dard (1σ ) Clopper and Pearson (1934) intervals.
                                                                              Trinians, SCREAM 2019)
                                                                              It was very interesting to learn more in regards to
                                                                              astrophysics and the MUSICS project was a very
Wilcoxon signed-rank test) and teachers (benchmark average                    safe space to do so. We got lots of support and it
3.84 ± 0.08, p = 1 × 10−7 ).                                                  was fun. (Student 119, Pokémon Technical Insti-
   Secondly, students (n = 135) were asked for adjectives de-                 tute, MUSICS 2020)
scribing their experience of the projects overall. They were                  Coming into the Planet Hunting With Python
free to use any words they wanted and were not given a pre-                   project, my interests were mainly focused on the
selected list. Teachers (n = 38) were similarly asked to in-                  physics side of understanding brightness-curves
dicate observations of their students’ experience also. Since                 and finding equations to solve for planetary pa-
2016 this has resulted in 88 unique adjectives, with both stu-                rameters. However, in this project, my eyes were
dents and teachers typically writing two to three words each.                 opened to the many uses of coding to analyse data,
We present the results as the word cloud in Fig. 4, where                     and it was a wonderful experience to learn about
students and teachers have been given equal prominence by                     such an interesting area through a combination of
normalising their counts by their respective totals. We have                  theory and practical coding. (Student 120, Octa-
indicated by colour from which group(s) the words origi-                      vian Country Day School, PHwP 2020)
nated, generally showing a lot of agreement between stu-                   They enjoyed the style of working in and with research
dents’ thoughts and teachers’ observations. The most cited               which differs from their regular school experience (33 re-
adjectives were (in descending order) interesting, challeng-             sponses).
ing, exciting, inspiring, and fun, similarly to those from the
pilot (Archer, 2017), with the top two adjectives being sig-                  It was quite interesting to broaden our views and
nificantly greater than the subsequent ones. While in the pi-                 experience high level education. (Student 90, Hill
lot stage only positive adjectives were expressed, since then                 Valley High School, ATLAS 2019)
a few negative experiences have been conveyed, such as                        I enjoyed the opportunity to do science instead of
time-consuming, frustrating, and stressful. These constitute a                just learning it. (Student 101, Octavian Country
small minority of experiences though (6 ± 1 %), and in most                   Day School, PHwP 2019)
cases the same students also listed positive adjectives, apart                It was very nice to work with friends and work to-
from only four individuals.                                                   gether to produce something. (Student 106, Boston
   Following on from these quantitative analyses, we quali-                   Bay College, PHwP 2019)
tatively explore the potential reasons behind the results. The
most common themes that emerged from students’ (n = 110)                      It was very fun to do our own research and I appre-
responses to open questions about their experience were that                  ciated that help was always available even though it
they feel they learnt a lot (62 responses).                                   is very independent. It also shows how challenging
                                                                              research can actually be but also how rewarding it
                                                                              is once you start making progress. (Student 143,
                                                                              Sunnydale High School, MUSICS 2020)
      We have learnt so many new things relating to the
      magnetosphere and waves and we have developed                        Neutral or negative experiences tended to be due to stu-
      new skills. (Student 3, Xavier’s Institute for Higher              dents finding the projects’ content or open-ended way of
      Learning, MUSICS 2016)                                             working difficult or confusing (seven responses).

Geosci. Commun., 4, 147–168, 2021                                                             https://doi.org/10.5194/gc-4-147-2021
M. O. Archer et al.: Physics research in school environments                                                                              157

Figure 4. Word cloud of students’ experiences. Colours indicate words identified by students (blue), teachers (green), or both (cyan). Students
and teachers have been given equal total weight.

     I did not understand most of the project or what I                         Excellent project that is open-ended allowing stu-
     was supposed to do. (Student 147, Jedi Academy,                            dents to take it where they want. (Teacher 27, Hog-
     ATLAS 2020)                                                                warts, SCREAM 2019)
   The vast majority of students seemed to ultimately enjoy                     This year I had an extremely motivated, enthusi-
this challenge though.                                                          astic and well-organised group of 7 students who
                                                                                fully immersed themselves into the project and
     The project gave us a lot of freedom and chal-                             quickly took it in a direction outside my own un-
     lenged us to think in different ways. (Student 122,                        derstanding of this area of science. This is exactly
     Jedi Academy, ATLAS 2020)                                                  the experience I wanted them to have, and they
     [It] made me more willing to take a go at chal-                            were able to discover some genuinely novel pro-
     lenges and what I deem hard. (Student 130, Bend-                           cesses that had not been observed before – the hall-
     ing State College, PHwP 2020)                                              mark of great scientific research! (Teacher 44, Sun-
                                                                                nydale High School, MUSICS 2020)
  Teachers agreed that the learning curve involved with the
projects was advantageous for students.                                      Therefore, both quantitative and qualitative data suggest
                                                                          students and teachers had much more positive and reward-
     When the students got the hang of it they really en-
                                                                          ing experiences participating in PRiSE projects than is typi-
     joyed it. (Teacher 16, Tree Hill High School, MU-
                                                                          cal for schools engagement programmes from universities in
     SICS 2018)
                                                                          general.
     The students found it hard to identify what to do a
     project on and would have liked guidance on that,                    4.2   Support and resources
     but I felt this was a good experience. They have
     developed grappling with open ended and difficult                    We originally asked students whether they felt they had re-
     material. (Teacher 23, Smeltings, ATLAS 2019)                        ceived adequate support, finding overall positive results on
                                                                          a five-point Likert scale (Archer, 2017). However, students’
  Teachers’ feedback on their experience (n = 34) tended to
                                                                          qualitative responses explaining their answers often revealed
praise how the projects allow their students to access and ex-
                                                                          a conflation of the support provided by Queen Mary with
plore beyond the curriculum (29 responses).
                                                                          that offered by their teacher. Therefore, from 2019 onwards
     A great framework to explore physics beyond the                      we explicitly separated these two aspects. Students (n = 68)
     syllabus but still accessible. (Teacher 26, Octavian                 were asked “Do you feel that support from your teacher
     Country Day School, PHwP 2019)                                       was provided/available during the project?”, which yielded

https://doi.org/10.5194/gc-4-147-2021                                                             Geosci. Commun., 4, 147–168, 2021
158                                                              M. O. Archer et al.: Physics research in school environments

the following results: Strongly Agree (30), Agree (34), Nei-            My own experience with research was handy but I
ther Agree or Disagree (3), and Disagree (1). The aver-                 felt that without this the students would still have
age response is 4.37 ± 0.08, which is considerably greater              been supported (Teacher 2, Hogwarts, SCREAM
than the benchmark on teacher support reported by Vennix                2016)
et al. (2017) of 3.60 ± 0.03 (p = 4 × 10−10 ). Students’ com-           only through Martin’s support (Teacher 4, Sweet
ments explaining their ratings (n = 56) revealed that teachers          Valley High School, MUSICS 2016)
provided them with advice, encouragement, and enthusiasm
(49 responses).                                                         the teacher version of the handout was useful,
                                                                        but otherwise I could only give generic advice
      My teacher has been very supportive and has                       (Teacher 16, Tree Hill High School, MUSICS
      helped us when we didn’t understand something as                  2018)
      well as encouraging us to taking a more innovative
      approach. (Student 124, Quirm College for Young                 Several teachers raised pressure on their time, with one
      Ladies, MUSICS 2020)                                         teacher using this to justify their negative response.
      If we had a question, teachers were probably not                  sheer time pressure – big limiting factor
      useful. But if we did not know what to do or we                   (Teacher 13, St Trinians, SCREAM 2017)
      were stuck, here teachers were really useful and
      that was what we needed. (Student 145, Sunnydale                Another said this somewhat limited their ability to support
      High School, MUSICS 2020)                                    students.
   They arranged regular sessions for students to meet and              not as much as I would have liked (lack of avail-
visits or calls from the university when required (seven re-            able time) (Teacher 3, Xavier’s Institute for Higher
sponses).                                                               Learning, MUSICS 2016)
      Our teacher arranged a Skype call with a profes-                However, most said it was manageable.
      sor from QMUL when we needed to ask questions
      about how certain parts of the data were calculated.              The autonomous group work, with very little input
      (Student 126, Harbor School, ATLAS 2020)                          from me, was great to see (Teacher 1, Hogwarts,
                                                                        SCREAM 2015)
      [Our] teacher would often ask us about it and hold
      meetings to catch up with us on our progress. (Stu-               Students were quite self-sufficient so if I made
      dent 143, Sunnydale High School, MUSICS 2020)                     suggestions they were able to do the leg work
                                                                        (Teacher 15, Spence Academy for Young Ladies,
      [Our] teacher answered some of our questions and
                                                                        MUSICS 2018)
      organised a weekly meet-up where students could
      ask each other questions and work together. (Stu-              Teachers’ ability and confidence in supporting the projects
      dent 148, Jedi Academy, ATLAS 2020)                          were other themes that emerged. Even with the teacher-
   Neutral or negative responses tended to be explained by         specific resources provided, some felt they did not have the
their teacher lacking specific knowledge about the research        specific knowledge or skills to support the projects.
in response to students’ queries (two responses).                       [Unable to support due to a] lack of knowledge of
      Although they were always ready to help, some-                    Python (Teacher 19, Boston Bay College, PHwP
      times they didn’t know the answers to our ques-                   2018)
      tions. (Student 122, Jedi Academy, ATLAS 2020)
                                                                      Other teachers reflected that, similarly to their students,
      [They] didn’t understand the content of the project.         they too had experienced a learning curve through their in-
      (Student 139, Jedi Academy, ATLAS 2020)                      volvement.
   This is something we do not expect of teachers (cf. Shah             [I found it] difficult at first (Teacher 8, Coal Hill
and Martinez, 2016; Bennett et al., 2016, 2018), hence why              School, MUSICS 2017)
support from the university is also offered.
   Teachers’ (n = 18) responses on a yes/no scale (chosen            They ultimately became more determined and confident
due to expected small number statistics) on whether they           with time and in subsequent years.
felt able to support their students were also highly posi-
tive, with only two negative responses, a significant major-            First time we’ve done this – I will do better next
ity (p = 0.001 in a two-tailed binomial test). Bear in mind,            time (Teacher 17, Sunnydale High School, MU-
however, that these responses were in light of the support              SICS 2018)
provided from the university, something which a few teach-              Second year that I ran it I feel more confident
ers referenced in explaining their answers.                             (Teacher 21, Hogwarts, SCREAM 2018)

Geosci. Commun., 4, 147–168, 2021                                                       https://doi.org/10.5194/gc-4-147-2021
M. O. Archer et al.: Physics research in school environments                                                                         159

   The final theme raised was that for successful participation
teachers believed the students needed the external motivation
to come from the university rather than having project deliv-
ery being solely teacher-driven.

     Dr Archer was a great external lead to have. If I
     had been pushing them myself they would have
     taken it less seriously (Teacher 17, Sunnydale High
     School, MUSICS 2018)

   Therefore, the comments from both students and teach-
ers indicate that teachers alone would likely not have been
able to successfully support these research projects in their
schools without both the resources and external motiva-
tion/mentoring provided as part of the PRiSE framework.
   We now consider the specific elements of support shown
in Fig. 1. From 2019 onwards we investigated participants’
thoughts on each of these various aspects offered. Students
(n = 68) and teachers (n = 23) were asked to rate the use-         Figure 5. Usefulness of support provided to teachers (T, n = 23)
fulness of these as either “unimportant”, “helpful”, “essen-       and students (S, n = 68). Results are divided (black lines and asso-
tial”, or “unsure”. This was chosen over a five-point Likert       ciated error bars) into negative (red) and positive responses, with the
scale due to an expected low number of responses, particu-         latter subdivided (grey lines and error bars) into “essential” (blue)
larly from teachers. Any unsure or blank responses are ne-         and “helpful” (yellow) elements. Error bars denote standard (1σ )
glected, yielding 326 (out of a potential 408) student and 156     Clopper and Pearson (1934) intervals.
(out of 161) teacher responses. We divide these responses
into negatives (“unimportant”) and positives (“helpful” or
                                                                   essary. This has been further elaborated on in teacher feed-
“essential”), though we acknowledge some may consider the
                                                                   back.
“helpful” response to be neutral, and thus our analysis takes
both interpretations into account. The results are displayed            It is very well set up and open ended and the sup-
in Fig. 5 for the individual elements as well as overall re-            port received is magnificent. (Teacher 33, Boston
sults obtained from totalling all responses. Both students and          Bay College, PHwP 2020)
teachers overall rated the elements positively – coding the
responses to values of 1 (negative) to 3 (essential), the over-         Martin’s guide to help the students was a very good
all means were 2.62 ± 0.04 for teachers and 2.23 ± 0.03 for             balance of useful guidance and allowing them to
students. The majority of teachers tended to give “essen-               find their own way through. (Teacher 38, Royal
tial” ratings to most aspects, and while these majorities are           Dominion College, MUSICS 2020)
not statistically significant in a two-tailed binomial test, the        Truly excellent support from the Queen Mary
average value for each element was greater than 2 to high               team! They have visited us multiple times and have
confidence (p < 0.002 in one-sample Wilcoxon signed-rank                been so generous with their time. Students have
tests). Students, on the other hand, mostly rated each ele-             learnt a great deal from them! (Teacher 44, Small-
ment as “helpful” as well as stating slightly more negative             ville High School, PHwP 2020)
responses than teachers, though again all elements’ mean
scores (apart from the kick-off workshop at 2.15 ± 0.08)
were significantly greater than 2 (p < 0.023). While there         5   Feedback from the university sector
are some variations in scoring amongst the different support
elements, such as students and teachers respectively rating        Based on the highly positive results from participants, we
researcher visits and communications (the latter of which in-      think there is potential for the PRiSE framework to spread
cludes webinars and ad hoc emails) as the most essential,          beyond QMUL and be applied to other institutions’ own ar-
these differences to each group’s overall results are slight and   eas of physics (and perhaps even STEM more generally) re-
not statistically significant. One interpretation of this might    search. We therefore wanted to assess how it is perceived by
be that most respondents answered unreflectively, ticking the      those from the university sector with interests and/or exper-
same boxes for each item. However, no students and only            tise in schools engagement. Feedback from our partner or-
three teachers gave the same answer in every category. This        ganisations seemed promising, for example with the South
therefore suggests that all of the elements of support pro-        East Physics Network including PRiSE in their public en-
vided as part of PRiSE are almost equally important and nec-       gagement strategy (SEPnet, 2017), several of their member

https://doi.org/10.5194/gc-4-147-2021                                                       Geosci. Commun., 4, 147–168, 2021
You can also read