EVALUATING PARTICIPANTS' EXPERIENCE OF EXTENDED INTERACTION WITH CUTTING-EDGE PHYSICS RESEARCH THROUGH THE PRISE "RESEARCH IN SCHOOLS" PROGRAMME
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Research article
Geosci. Commun., 4, 147–168, 2021
https://doi.org/10.5194/gc-4-147-2021
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.
Evaluating participants’ experience of extended interaction with
cutting-edge physics research through the PRiSE “research in
schools” programme
Martin O. Archer1,a , Jennifer DeWitt2,3 , Charlotte Thorley4 , and Olivia Keenan5
1 School of Physics and Astronomy, Queen Mary University of London, London, UK
2 Instituteof Education, University College London, London, UK
3 Independent Research and Evaluation Consultant, UK
4 Public Engagement and Involvement Consultant, UK
5 South East Physics Network, London, UK
a now at: Space and Atmospheric Physics, Department of Physics, Imperial College London, London, UK
Correspondence: Martin O. Archer (martin@martinarcher.co.uk)
Received: 21 July 2020 – Discussion started: 31 July 2020
Revised: 28 January 2021 – Accepted: 24 February 2021 – Published: 8 April 2021
Abstract. Physics in schools is distinctly different from, and 1 Introduction
struggles to capture the excitement of, university research-
level work. Initiatives where students engage in independent Research, policy, and practice all agree that participation in
research linked to cutting-edge physics within their school science, technology, engineering and mathematics (STEM)
over several months might help mitigate this, potentially fa- needs to be increased and widened (e.g. Campaign for Sci-
cilitating the uptake of science in higher education. How- ence and Engineering, 2014), with these issues being par-
ever, how such initiatives are best supported remains unclear ticularly acute for the subject of physics (e.g. Murphy and
and understudied. This paper evaluates a provision frame- Whitelegg, 2006; IOP, 2014). Physics as a field has become
work, Physics Research in School Environments (PRiSE), within society strongly aligned with intelligence/cleverness,
using survey data from participating 14–18-year-old students masculinity and whiteness, all of which can dissuade school
and their teachers to understand their experience of the pro- students (even those highly enthusiastic about the subject)
gramme. The results show that PRiSE appears to provide from pursuing it further and thereby showing inequitable ef-
much more positive experiences than typical university out- fects on those from under-represented backgrounds (Archer
reach initiatives due to the nature of the opportunities af- et al., 2020). Some of these issues arise from practices
forded over several months, which schools would not be in school-level physics education. Debarring and gatekeep-
able to provide without external input. The intensive sup- ing of physics based on attainment (disproportionately so
port offered is deemed necessary, with all elements appearing compared to other subjects) simply feeds the alignment of
equally important. Based on additional feedback from inde- physics with cleverness and can make even high-attaining
pendent researchers and engagement professionals, we also students’ confidence in the subject precarious. Teachers and
suggest the framework could be adopted at other institutions the school environment often (even unconsciously) reinforce
and applied to their own areas of scientific research, some- stereotypes about physics and physicists that are patterned
thing which has already started to occur. by biases. Curriculum practices in physics often teach over-
simplifications at younger ages which are later completely
reconceptualised without being presented as refinements to
a model, making students perceive the simpler versions as
“lies”. Furthermore the general deferment of “interesting”
physics in the curriculum produces a disconnect between
Published by Copernicus Publications on behalf of the European Geosciences Union.
148 M. O. Archer et al.: Physics research in school environments “school physics” and “real physics”, i.e. the cutting-edge re- ternal support, and funding/costs. It was noted that such pro- search undertaken by professional physicists, making contin- grammes place demands on time and money beyond stan- ued participation in physics education something of a “test dard provisions for all stakeholders, on the skills required of endurance”. These concerns are further reflected in re- by teachers and other adults involved, and on the supporting sults from national surveys. While 20 % of 16–18-year-old infrastructure. For successful projects Dunlop et al. (2019) physics school students in the UK aspire towards a physics recommend that students should be given the freedom to de- degree and 80 % aspire towards STEM more broadly (Well- vise a research question, have ownership over their own data come Trust, 2017), only 9.7 % and 59.3 % actually go on analysis and decision-making, and be given access to experts to study either physics or STEM respectively (McWhinnie, in their project work. Broadly there are two distinct formats 2012). These constitute odds ratios for aspirations vs. desti- of independent research projects. nations of 2.3 and 2.7, both of which are considerable. All of these issues raised cultivate and contribute to reproducing – Those associated with dedicated out-of-school events inequitable, and low overall, patterns of participation within of only a few weeks’ duration such as intern- physics (Archer et al., 2020). ships/apprenticeships (e.g. Nuffield Research Place- Davenport et al. (2020) suggest that for STEM subjects ments in the UK, Cilauro and Paull, 2019; Raising Inter- in general an intervention approach that sustains and sup- est in Science and Engineering in the USA, Stanford Of- ports science identity is most appropriate for students in late fice of STEM Outreach, 2020), summer schools/camps secondary/high-school education in the context of their ed- (e.g. the International Astronomical Youth Camp run ucational journey. However, in the case of physics specifi- across Europe and parts of Africa, Dalgleish and Veitch- cally, Archer et al. (2020) comment that existing interven- Michaelis, 2019), or science competitions/fairs (particu- tions based on simply enthusing, inspiring and informing larly prevalent in the USA, e.g. Yasar and Baker, 2003). students about physics will not significantly change uptake – Those undertaken within school itself over the course or diversity in post-compulsory physics. While they advo- of several months to a year, either in class or sup- cate for widespread changes in science education policy and plemented with time in after-school clubs (e.g. an practice, both at school and university levels, they note that after-school mechatronics project in Taiwan, Hong if interventions are also used they need to fundamentally ad- et al., 2013, class-based biology project in Singapore, dress the problematic processes and practices present within Chin and Chia, 2010, or various CarboSchools climate both physics teaching and physics as a field generally. change projects between research institutes and schools The stark differences between school, university, and pro- across seven European countries, Dijkstra and Goed- fessional science practices have long been noted – while re- hart, 2011). search is one of the main activities of professional scientists, it is quite removed from how science is taught in schools, We do not discuss the former here as they are necessarily with some arguing that science education is not “authen- limited in reach, catering only to heavily bought-in individu- tic” in this respect (e.g. Hodson, 1998; Braund and Reiss, als (i.e. typically one to three students; Paull and Xu, 2017) 2006). Indeed, Yeoman et al. (2017) report that school stu- from any given school. Most independent research projects dents are largely unaware of what research actually is, find- based within schools are not linked to current cutting-edge ing a disconnection between “research as information gath- and novel scientific research topics or questions. However, ering” and the “research question”, and in general have little relatively recently so-called “research in schools” projects opportunity to set their own research questions within their have emerged, which do provide students with experiences school environment. Independent research projects, which of genuine contemporary STEM research within their own provide extended opportunities for students to lead and tackle school environment over several months. While several citi- open-ended scientific investigations (not simply literature re- zen science projects have also run within schools and aim to views or essays, e.g. Conner, 2009; Corlu, 2014), may be help participants learn about current science and to experi- one way of exposing students to “real science”. These align ence the scientific research process, these are typically sec- with established international pedagogical initiatives such as ondary aims since they primarily concern a single (or small “inquiry-based science” (e.g. Minner et al., 2010), “problem- number) of well-defined science questions which will be as- based learning” (e.g. Gallagher et al., 1995), and “authen- sisted through developed citizen science protocols (Bonney tic science” (e.g. Braund and Reiss, 2006). A survey of et al., 2009, 2016; Shah and Martinez, 2016). This contrasts such projects across 12 countries (Australia, Ireland, Israel, with independent research projects and thus also “research Netherlands, New Zealand, Qatar, Singapore, Spain, Taiwan, in schools”, where positively affecting the participants is the Turkey, UK, and USA), however, found them to be rare glob- primary concern and the projects are necessarily open-ended. ally and only sometimes supported by mentors from univer- Nonetheless, the different approaches can have some over- sity/industry (Bennett et al., 2016, 2018). This review found lap, and indeed some projects denoted as citizen science, considerable variability in the nature of independent research such as the “curriculum-based” projects based in the USA projects, such as their focus, delivery/provision models, ex- described by Bonney et al. (2016), might perhaps be better Geosci. Commun., 4, 147–168, 2021 https://doi.org/10.5194/gc-4-147-2021
M. O. Archer et al.: Physics research in school environments 149
framed as “research in schools”. “Research in schools” pro- personal communication, 2020), with IRIS itself being the
grammes appear at present to be most prevalent in the UK, main point of contact for schools. With a recent change
and we are aware of three featuring projects in the physical of staff at IRIS in late 2019 has come a reformulation of
sciences (outside of that at Queen Mary University of Lon- how they classify their projects. “Seed” projects are for new
don, QMUL, which forms the subject of this paper). schools, are the most straightforward, and receive the most
HiSPARC (High School Project on Astrophysics Research support from IRIS, though it is not clear what form that takes.
with Cosmics) is a scintillator–photomultiplier cosmic ray “Sprout” projects are more advanced, seeing students carry-
detector project originating in the Netherlands at Radboud ing out more complex activity to assist scientists with their
University in 2004, which has subsequently been adopted research questions, though how this collaboration operates is
by other Dutch (Amsterdam, Eindhoven, Groningen, Leiden, not specified. “Grow” projects are where students have pro-
Nikhef, Twente, Utrecht) as well as UK (Bath, Bristol, Birm- posed their own research questions, either independently or
ingham, Sussex) and Danish (Aarhus) universities (Colle using IRIS resources, with IRIS merely providing advice in
et al., 2007; van Dam et al., 2020; HiSPARC, 2018). Many producing posters, talks, or papers as well as opportunities to
of these universities operate a tiered membership scheme for present.
schools: “Gold” enables schools to buy their own detector ORBYTS (Original Research By Young Twinkle Scien-
(GBP 5,500 in the UK); “Silver” is a detector rental scheme tists), based at University College London, was piloted from
(GBP 300 p.a. plus an installation fee) with the contract spec- January 2016 and is nominally based around the Twinkle
ifying if they do not participate the detector will be collected mission but has expanded into other research areas since
with an additional fee; and “Bronze” membership (GBP 200 (Sousa-Silva et al., 2018; ORBYTS, 2019). A select group
p.a.) gives schools access to HiSPARC data but not their own of students (with an imposed limit of four to six) from
detector. Schools signing up for “Silver” or “Bronze” mem- each school undertakes fortnightly meetings with early ca-
bership are contractually obliged to generate funding to up- reer researchers (either PhD students or post-docs) through-
grade to “Gold”. While the “Gold” membership fee covers out their project aiming to achieve, where possible, publish-
the costs of the detector and installation, the other member- able scientific results (McKemmish et al., 2017; Chubb et al.,
ships are to ensure that schools make a commitment to work- 2018; Holdship et al., 2019). Teachers, while present, are not
ing with the university (Jaap Velthuis and Maria Pavlidou, typically actively involved in these sessions, and students
personal communication, 2016; National HE STEM SW, tend to do little independent work outside of the sessions
2012). It is not possible to compare how these schools go (William Dunn, personal communication, 2018). The con-
about project work and how much support they are given by tent of the projects changes each year to align with the re-
participating universities, which may vary by institution, as searchers’ current focus, with them typically working with
at the time of writing HiSPARC has not published any re- only one school per year each. PhD researchers are paid for
views of their processes or evaluation. their (preparation, travel, and session) time with funds from
IRIS (Institute for Research in Schools) is a UK char- independent schools, who pay not only for their school, but
ity formally launched in March 2016 (IRIS, 2020), build- also for enabling an additional school from a lower socio-
ing on the previous CERN@School project conceived in economic background to take part.
2007 (Whyntie et al., 2016; Parker et al., 2019). While IRIS’ It is clear that there is currently a lack of published de-
projects cover all the sciences, current physics projects in- tails and evaluation on provision within the emerging area
clude the aforementioned CERN@School, Higgs Hunters of “research in schools”. This paper therefore explores these
(Barr et al., 2018), LUCID (Furnell et al., 2019; Hatfield aspects applied specifically to the “research in schools” pro-
et al., 2019), and Webb Cosmic Mining (in preparation for gramme of QMUL’s School of Physics and Astronomy. This
the James Webb Space Telescope). They have rapidly ex- was piloted between 2014 and 2016, as detailed in Archer
panded across the UK since formation, having worked in (2017), and is now known as Physics Research in School
some capacity with over 230 schools as of 2020. Publica- Environments (PRiSE, 2020). Section 2 introduces PRiSE’s
tions have provided technical details of their projects and framework, which is then evaluated in Sect. 4 in terms of par-
case studies of some students’ successes within them, in- ticipating students’ and teachers’ experience. We also briefly
cluding a few examples of resulting peer-reviewed scien- discuss how the PRiSE approach has been received by the
tific work; however, the exact provision/delivery model im- university sector in Sect. 5.
plemented and precisely how project work is supported are
not fully explored in the available literature. IRIS aims to
develop “teacher scientists”, teachers that identify as both 2 PRiSE framework
science teachers and research-active scientists (Rushton and
Reiss, 2019), which suggests a teacher-driven model. While Physics Research in School Environments (PRiSE) is a
some researchers/academics have designed or consulted on collection of physics-based “research in schools” projects
some IRIS projects, they appear in general to have little in- (see Table 1) brought together under a coherent provi-
volvement in supporting students or teachers (Oliver Moore, sion framework. The programme aims to equip 14–18-year-
https://doi.org/10.5194/gc-4-147-2021 Geosci. Commun., 4, 147–168, 2021
150 M. O. Archer et al.: Physics research in school environments
old school students (particularly those from disadvantaged university; therefore, students are not pressured into being in-
backgrounds) with the ability, confidence, and skills to in- volved. Students and schools can drop out at any point within
crease/sustain their aspirations towards physics or more the programme with no penalty, which does occur in a minor-
broadly STEM, ultimately enabling them to realise these ity of schools.
at higher education and thus contributing to increased up-
take and diversity of physics and to some extent STEM 2.1.2 Role of teachers
(cf. Archer et al., 2020). Through working with teachers,
PRiSE also aims to develop their professional practice and The involvement of teachers at all stages is of paramount im-
build long-term university–school relationships that raise the portance in terms of delivery and safeguarding. They have
profile of science and mitigate biases/stereotypes associated helped shape the design of the programme, inform how we
with physics within these schools, generally making them en- update it each year, and serve as our liaison to schools and the
vironments which nurture and enhance all students’ science students involved. It is the teachers that decide who projects
capital (cf. IOP, 2014). Our rationale for these particular out- are offered to within their school, with us simply advising
comes is left to the Supplement. This section summarises the that the projects should be suitable for all A-Level (16–18-
PRiSE framework, discussing the ethos of the programme year-old) students as well as high-ability GCSE (14–16-year-
and the roles played by the schools and university as well old) students (further contextual information on the UK ed-
as outlining the various activity stages, interventions, and re- ucation system is given in Appendix A). These recommen-
sources which it consists of. In-depth practical details aimed dations were made based on the basic background knowl-
at practitioners looking to replicate the framework are given edge required to meaningfully engage with the research. In-
in the Supplement. variably teachers choose to involve older age groups, with
79±1 % of PRiSE students being aged 16–18 (and so far only
2.1 Approach
one student below our recommended ages has been involved,
being 13–14). Unfortunately, we do not have any specific in-
PRiSE takes the “research in schools” approach to schools formation on exactly how teachers go about selecting stu-
engagement, whereby students are given the opportunity to dents. However, the average number of students per school
lead and tackle open-ended scientific investigations in ar- each year is around 12, which compared to the national av-
eas of current research. Therefore, the PRiSE projects were erage class size in A-Level physics of 16 (RAE et al., 2015)
developed to transform current scientific research methods, indicates teachers involve a significant majority (or in many
making them accessible and pertinent to school students so cases the entirety) of their cohorts in PRiSE. We allow teach-
that they could experience, explore, and undertake scientific ers to determine how best to integrate the projects within
research themselves. their school though provide advice on this. We also aim,
through our resources and communications, to equip teach-
2.1.1 Ethical considerations
ers to manage the day-to-day aspects of the projects without
overly burdening them – their role is chiefly one of encourag-
Since the programme intends to influence school students ing their students to persist, providing what advice they can,
and teachers, a number of ethical considerations have been and then communicating with the university.
taken into account, following the BERA (2018) guidance for Teachers’ involvement at all stages also presents opportu-
educational research, with regard to safeguarding and to en- nities to them for continuing professional development, help-
sure that no harm results. Firstly, to ensure equality of ac- ing them nurture and cultivate STEM aspirations among stu-
cess to the programme, we do not charge schools to be in- dents throughout their school (i.e. not just PRiSE students).
volved (cf. Harrison and Shallcross, 2010; Jardine-Wright, This is implemented informally and integrated within the
2012) and try to provide them with all the physical resources programme in the form of both bespoke resources and on-
they need for their project, thereby removing potential bar- going dialogues between teachers and researchers. These are
riers to entry for less resourced schools. Our targeting takes aimed at enhancing teachers’ knowledge about the under-
into account several school-level metrics (type of school, stu- lying science and how they link to curriculum-based topics
dents on free school meals, indices of multiple deprivation, where appropriate, their skills and confidence surrounding
gender balance, etc.) to ensure diversity. We aim for the pro- current research topics and methods, and their pedagogy in
gramme to be equitable with all schools being offered the mentoring independent project work.
same interventions/opportunities, taking into account and be-
ing flexible to their specific needs where necessary. We work 2.1.3 Role of the university
with as many schools as we have capacity to do so each year
and do not withhold interventions from any students for the It was recognised that teachers in general likely will not
purpose of having control groups. The projects are optional have the skills or experience in research to manage projects
and presented as an opportunity that students can take ad- without expert assistance (Shah and Martinez, 2016; Bennett
vantage of which will be supported by their teacher and the et al., 2016, 2018). Therefore, PRiSE was designed to be sup-
Geosci. Commun., 4, 147–168, 2021 https://doi.org/10.5194/gc-4-147-2021
M. O. Archer et al.: Physics research in school environments 151
Table 1. A summary of the existing PRiSE projects at QMUL.
Project Abbreviation Years Field Description
Scintillator Cosmic Ray SCREAM 2014–2020 Cosmic rays Scintillator–photomultiplier
Experiments into Atmospheric tube detector usage
Muons
Magnetospheric Undulations MUSICS 2015–2020 Magnetospheric physics Listening to ultra-low
Sonified Incorporating Citizen frequency waves and analysing
Scientists in audio software
Planet Hunting with Python PHwP 2016–2020 Exoplanetary transits Learning computer
programming, applying this to
NASA Kepler and TESS data
ATLAS Open Data ATLAS 2017–2020 Particle physics Interacting through an online
tool with LHC statistical data
on particle collisions
ported by active researchers equipped with the necessary ex- be used as criteria for academic promotions (cf. Hillier et al.,
pertise to draw upon in offering bespoke, tailored guidance to 2019). Physics researchers though are largely unmotivated
the students and teachers. Well-defined roles within the uni- in delivering curriculum content as part of their engagement
versity have been established for each of the PRiSE projects work, valuing instead aspects relating to their research and
to provide this support. role as a researcher (Thorley, 2016). PRiSE thus also aligns
– Outreach Officer. Manages the entire programme, in- with this direction. Ultimately, researcher buy-in is vital to
cluding university–school relationships, communica- the delivery of protracted research-based engagement pro-
tions, intervention/event co-ordination, programme fi- grammes such as PRiSE.
nances, and evaluation. It is clear from Table 1 that the topics and activities of
current PRiSE projects vary considerably. This suggests that
– Project Lead. Visible figurehead for the project to a wide range of fields and project ideas might be able to
schools, typically an academic member of staff. adopt the PRiSE framework. How projects have been devel-
– Researcher. Providing advice and guidance to students oped has also varied (further explored in the Supplement),
and teachers throughout the programme. Can be dele- though we have adopted a pragmatic approach in taking ad-
gated to or shared with early career researchers. vantage of opportunities (grant funding, internships, etc.) and
adapting existing materials where possible, since creating a
Since the primary focus of PRiSE is (unlike typical citi- project from scratch is a significant undertaking far beyond
zen science) on the participants rather than the research, re- what most academics (unfortunately) have the capacity to do
searchers should not consider students contributing to novel (cf. Thorley, 2016).
research to be their rationale for being involved. Our position
is that it is rather unreasonable to expect investigations that
2.2 Structure
are motivated by school students themselves (an established
element of good practice in independent research projects, PRiSE runs from the start of the UK academic year to
e.g. Dunlop et al., 2019) to be able to make meaningful con- just before the spring/Easter break, which teachers had in-
tributions to the physics research as a matter of course. We formed us during the pilot stage is manageable and largely
note that in some exceptional cases PRiSE students’ work fits around exams/other activities for most (but not neces-
has arrived at promising preliminary results, though these sarily all) schools (Archer, 2017). The structure has evolved
have required significant follow-up work by professional re- naturally from the pilot to that shown in Fig. 1.
searchers to transform the results into publishable research
(e.g. Archer et al., 2018) and thus should not be considered
2.2.1 Activity stages
the archetype. Instead, researchers are enticed by the possi-
bility of societal impact underpinned by their research. This Students work in research groups of typically five people
is something which is increasingly called upon from funders and they are advised to try and work on the project on av-
(e.g. National Coordinating Centre for Public Engagement, erage for 1–2 h a week. The bulk of this is done outside of
2020) and is notoriously difficult for areas of “blue skies” regular physics lessons, though some schools integrate the
research such as physics. Furthermore, significant contribu- projects within their timetabled “science clubs” or required
tion to a coordinated departmental outreach programme can extra-curricular blocks, whereas other teachers arrange a reg-
https://doi.org/10.5194/gc-4-147-2021 Geosci. Commun., 4, 147–168, 2021152 M. O. Archer et al.: Physics research in school environments
Figure 1. Graphic summarising the PRiSE framework including a timeline of the different project activity stages (rectangles), interventions
and stakeholders’ roles within them (rounded rectangles), and resources provided (document shapes). Arrows indicate over what dates
interventions (identified by colour) typically occur.
ular slot for students to work on the projects or leave it up 2.2.2 Interventions
to the students to arrange (though this latter approach often
Several different interventions form the structure and support
proves unsuccessful). PRiSE projects are split up into three
behind the activity stages.
activity stages (see Fig. 1).
– Assignment. The opportunity is advertised to schools via
– Prescribed work. Given that independent research in existing teacher networks and teachers apply to partic-
STEM is probably unfamiliar to the students, rather than ipate in the following academic year. Schools are as-
expecting them to be able to come up with their own signed a project before the summer break.
avenues of investigation in an unfamiliar research topic – Kick-off. Typically on-campus event featuring an intro-
straight away, we instead give them an initial prescribed ductory science talk, outline of how the project will
stage of research. work, and a hands-on workshop.
– Visit. Researchers visit schools to mentor students (and
their teachers) on their project work in a student-driven
– Independent project. Groups are encouraged to set their
intervention.
own research questions and undertake different projects
in the topic area when ready. This enables every group – Webinars. Drop-in online sessions similar to school vis-
to explore something different so that students gain a its but allowing students and teachers to gain further
sense of independence and ownership of their work. support.
– Ad hoc. Further asynchronous support via email as re-
quired.
– Writing up. Near the end of the project students produce
either a scientific poster or talk to be presented at an – Comments. Students are offered the opportunity to re-
annual conference. ceive comments on their draft slides or posters.
Geosci. Commun., 4, 147–168, 2021 https://doi.org/10.5194/gc-4-147-2021M. O. Archer et al.: Physics research in school environments 153
– Conference. Students present the results of their projects However, PRiSE’s efficiency in researcher time is reflected
as oral or poster presentations at a special conference in its reach as shown in Table 2, demonstrating the model
at the university, attended by researchers as well as the now serves around 30 schools per year having reached a to-
students’ teachers, peers, and family. tal of 67 schools and over 1300 students with the direct in-
volvement of 88 teachers as of 2020. We note that a minor-
Stakeholders’ roles within these interventions are given in ity of schools do not complete the full programme and oth-
Fig. 1, with photos depicting some of them displayed in ers do not return for subsequent years; however, we do not
Fig. 2. All stages of the programme and the processes in- provide a full analysis of the types of schools involved and
volved are communicated to teachers via email to pass on to their retention within the programme in this paper. Compar-
their students. ing the number of schools to other physical science “research
in schools” projects/programmes, University of Oxford re-
2.2.3 Resources searchers have interacted directly with only 14 students from
five schools through their Higgs Hunters IRIS project which
To enable the students to take part in PRiSE, the students and
commenced in 2016 (Oliver Moore, personal communica-
teachers are also provided with numerous resources. While
tion, 2020, though we note information regarding other IRIS
some projects require specific equipment, data and software,
projects is not available), ORBYTS (2019) reports collab-
here we discuss more common types of resources across the
orating with 17 schools since 2016, and HiSPARC (2018)
different projects as shown in Fig. 1.
discloses 140 schools since 2004 across their network of 13
– Project poster/flyer. Given to teachers to help advertise Dutch, Danish, and UK universities (van Dam et al., 2020),
projects in their school. i.e. around 11 per institution. Therefore, the reach through
the PRiSE framework by a single university department is
– Project guide. Each project has a student guide cov- considerable compared to the rest of the sector.
ering an introduction to the research field, back-
ground physics/theory, an explanation of the equip-
3 Methods
ment/data, discussion of analysis techniques, details of
the initial prescribed activity, suggested research ques- To determine the perceived value of PRiSE’s approach with
tions/methods for independent research, and links to its key stakeholders, namely participating students and teach-
other sources of information. Teachers are provided ers as well as those across the wider university sector, we
with the same guide but with extra guidance. have maintained regular collection of evaluative data (cf.
– Project webpage. These showcase anonymised exam- Rogers, 2014, and references therein) via various surveys
ples of good-quality talks/posters that previous students which we detail here. These data underpin our understand-
have produced and provide any links or videos relevant ing of PRiSE and have been collected securely to protect all
to the project. participants, in compliance with GDPR and in line with the
BERA (2018) guidelines for educational research.
– “How to” guide. General articles applicable to most re-
search projects, such as producing scientific talks and 3.1 Instruments and participants
posters.
We gathered feedback from participating students and teach-
ers via paper questionnaires handed out at our student con-
2.3 Scalability
ferences each year. The only exception to this was in 2020,
PRiSE’s framework attempts to find a balance between the where online forms were used due to the COVID-19 pan-
(necessarily competing) reach and significance of the inter- demic causing that year’s conference to be postponed. The
actions. For example, an academic staff member acting as questionnaire method was chosen so as to gather data from
both project lead and researcher within PRiSE can support as wide a range of students and teachers as possible, respect-
four schools’ participation (∼ 50 students), taking around 8 h ing the limited time/resources of all involved (both on the
over the course of 6 months (cf. Fig. 1). Using PhD students school and university sides). For ethics considerations all
or post-docs in the researcher role(s) makes even more effi- feedback was anonymous, with students and teachers only
cient use of time. In contrast, under the ORBYTS model each indicating their school (pseudonyms are used here to pro-
early career researcher can support only one school (four to tect anonymity) and which project they were involved with.
six students) with 10 h of their time. As noted in the introduc- Students were not asked to provide details of any protected
tion, mentorship from active researchers throughout does not characteristics (such as gender or race) or sensitive infor-
appear prevalent in other “research in schools” programmes mation (such as socio-economic background). Both students
at present. and teachers were informed via an ethics statement on the
Programmes of repeat interventions with schools will nec- form that the information was being collected for the pur-
essarily have a smaller reach than various one-off events. pose of evaluating and improving the programme and that
https://doi.org/10.5194/gc-4-147-2021 Geosci. Commun., 4, 147–168, 2021154 M. O. Archer et al.: Physics research in school environments
Figure 2. Photos of various stages of the PRiSE framework: students participating in an on-campus kick-off workshop (a), students inter-
acting during the poster session at a conference (b), a group of students display their prizes won at a conference along with their teacher (c),
and a group presents a talk at a conference (d).
Table 2. Number of schools, students, and teachers involved in PRiSE by academic year as the programme has grown.
Academic year 2014/2015 2015/2016 2016/2017 2017/2018 2018/2019 2019/2020
Schools 1 6 18 29 27 33
Number per year Students 20 115 163 310 311 407
Teachers 1 7 25 29 31 38
Schools 1 6 20 39 50 67
Unique cumulative total Students 20 135 298 608 919 1326
Teachers 1 7 28 44 63 88
they could leave any question they felt uncomfortable an- ered throughout. There is no indication that the respondents
swering blank (this functionality was also implemented on differed in any substantive way from the wider cohorts par-
the online form for consistency). ticipating in the programme. While ideally one would also
The open and closed questions concerning participants’ gather feedback from schools which dropped out during the
experience of the programme, which varied slightly year to year, a similar formal feedback process has not been viable
year, are given in Appendix B. While we attempted to col- bar in a few cases where only the teachers responded.
lect responses from all participants in attendance, invariably Feedback from the university sector came from a session
only a fraction did so, yielding results from 153 students and at the 2019 Interact symposium that presented the challenges
45 teachers across 37 schools. A breakdown of the number to STEM outreach practice highlighted by recent educational
of respondents and their schools per year is given in Table 3, research and the need for deeper programmes of engage-
where the numbers of participants and schools in attendance ment with young people and then summarised the PRiSE
at our conferences are also indicated. We do not have reliable approach as one possible example (Archer, 2019). Through-
information on how many students, teachers, and schools out this workshop an anonymous interactive online survey
would have successfully completed the programme in 2020 was used for interactivity and to collect data presented in this
due to the COVID-19 disruption. Students and teachers did paper. The survey included both closed and open questions
not always answer all of the questions asked, and hence we as listed in Appendix C. Attendees were fairly evenly split
indicate the number of responses for each question consid- between UK university researchers and engagement profes-
Geosci. Commun., 4, 147–168, 2021 https://doi.org/10.5194/gc-4-147-2021M. O. Archer et al.: Physics research in school environments 155
Table 3. Response rates to questionnaires at PRiSE student conferences.
2015 2016 2017 2018 2019 2020
Students 13/26 (50 %) 21/70 (30 %) 46/92 (50 %) 38/97 (39 %) 35/?
Teachers 1/1 (100 %) 6/6 (100 %) 6/11 (55 %) 9/16 (56 %) 6/16 (38 %) 17/?
Schools 1/1 (100 %) 6/6 (100 %) 11/11 (100 %) 13/15 (87 %) 11/15 (73 %) 19/?
sionals (gauged in-person by attendees raising their hands 3. Thematic review. Codes are used to generate themes and
when asked), with 19 people participating in the survey and identify associated data.
only 7 not doing so. Participants were allocated a unique
number by the online survey itself, which did not distinguish 4. Application. Codes are reviewed through application to
between researchers and engagement professionals. the full data set.
5. Analysis. Thematic overview of the data is confirmed,
3.2 Analysis with examples chosen from the data to highlight the
themes.
Both qualitative and quantitative approaches were utilised
in data analysis, as the open- and closed-ended questions
present in the questionnaires produced different types of data. 4 Feedback from participants
For all quantitative data, standard (i.e. 68 %) con-
fidence intervals are presented throughout. For propor- In this section we use the feedback from participating stu-
tions/probabilities these are determined through the Clopper dents and teachers to evaluate the provision offered within
and Pearson (1934) method, a conservative estimate based on the PRiSE framework, specifically assessing their experience
the exact expression for the binomial distribution, and there- and the level of support offered.
fore represent the expected variance due to counting statis-
tics only. Several statistical hypothesis tests are used with
4.1 Experience
effect sizes and two-tailed p values being quoted, with the
required significance level being α = 0.05. In general we opt Firstly from 2016 onwards we asked both students (n = 150)
to use nonparametric tests as these are more conservative and and teachers (n = 42) “Have you been happy with the re-
suffer from fewer assumptions (e.g. normality, interval scal- search project overall?”, giving options on a five-point Lik-
ing) than their parametric equivalents such as t tests (Hollan- ert scale, which we coded to the values 1–5. This scale and
der and Wolfe, 1999; Gibbons and Chakraborti, 2011). The the results are displayed in Fig. 3, revealing that 91 ± 3 %
Wilcoxon signed-rank test is used to compare single sam- of students and 95 ± 5 % of teachers rated their experience
ples to a hypothetical value, testing whether differences in as positive (scores of 4–5), with only three students giving a
the data are symmetric about zero in rank. When comparing negative reaction (scores of 2). Teachers tended to rank this
unpaired samples a Wilcoxon rank-sum test is used, which question somewhat higher (their mean score was 4.50±0.09,
tests whether one sample is stochastically greater than the where uncertainties refer to the standard error in the mean)
other (often interpreted as a difference in medians). Finally, than students (mean of 4.17 ± 0.05), with p = 0.002 in a
for proportions we use a binomial test, an exact test based on Wilcoxon rank-sum test. The PHwP project scored slightly
the binomial distribution of whether a sample proportion is higher (average of 4.59 ± 0.11, p = 8 × 10−4 ) than the over-
different from a hypothesised value (Howell, 2007). For ease all results with students, whereas ATLAS scored slightly
of reference, further details about the quantitative analyses lower with both students (3.92 ± 0.09, p = 0.012) and teach-
are incorporated into the relevant sections of the findings. ers (3.80 ± 0.20, p = 0.017) than their respective means. No
Qualitative data were analysed using thematic analysis obvious trends were present by school.
(Braun and Clarke, 2006). Instead of using a priori codes, the While suggestive of extremely positive experiences with
themes were allowed to emerge naturally from the data us- PRiSE, one also needs to compare these distributions against
ing a grounded theory approach (Robson, 2011; Silverman, the typical responses of students and teachers for schools’
2010) as follows. STEM engagement programmes. We use the results of Ven-
nix et al. (2017) as such a benchmark, which surveyed
1. Familiarisation. Responses are read and initial thoughts 729 high-school students and 35 teachers about 12 different
noted. STEM outreach activities in the USA and Netherlands. This
comparison reveals that PRiSE seems to be perceived consid-
2. Induction. Initial codes are generated based on review erably more positively than usual by both students (bench-
of the data. mark average 3.66 ± 0.01, p = 1 × 10−15 in a one-sample
https://doi.org/10.5194/gc-4-147-2021 Geosci. Commun., 4, 147–168, 2021156 M. O. Archer et al.: Physics research in school environments
I learned so much! I would recommend it to all
the younger kids at my school! (Student 111, St
Trinians, SCREAM 2019)
I definitely learnt many new and interesting things
and it helped me to develop my understanding of
particle physics while aiding my A-Level knowl-
edge. (Student 140, Jedi Academy, ATLAS 2020)
They found the projects’ content and methods interesting
(47 responses).
It brought my interests in programming, physics,
maths and space together. (Student 100, Octavian
Country Day School, PHwP 2019)
Figure 3. Distribution of students’ (blue) and teachers’ (green)
It has been very interesting to work with actual data
overall happiness with their PRiSE projects. Error bars denote stan-
and plan our own research project. (Student 116, St
dard (1σ ) Clopper and Pearson (1934) intervals.
Trinians, SCREAM 2019)
It was very interesting to learn more in regards to
astrophysics and the MUSICS project was a very
Wilcoxon signed-rank test) and teachers (benchmark average safe space to do so. We got lots of support and it
3.84 ± 0.08, p = 1 × 10−7 ). was fun. (Student 119, Pokémon Technical Insti-
Secondly, students (n = 135) were asked for adjectives de- tute, MUSICS 2020)
scribing their experience of the projects overall. They were Coming into the Planet Hunting With Python
free to use any words they wanted and were not given a pre- project, my interests were mainly focused on the
selected list. Teachers (n = 38) were similarly asked to in- physics side of understanding brightness-curves
dicate observations of their students’ experience also. Since and finding equations to solve for planetary pa-
2016 this has resulted in 88 unique adjectives, with both stu- rameters. However, in this project, my eyes were
dents and teachers typically writing two to three words each. opened to the many uses of coding to analyse data,
We present the results as the word cloud in Fig. 4, where and it was a wonderful experience to learn about
students and teachers have been given equal prominence by such an interesting area through a combination of
normalising their counts by their respective totals. We have theory and practical coding. (Student 120, Octa-
indicated by colour from which group(s) the words origi- vian Country Day School, PHwP 2020)
nated, generally showing a lot of agreement between stu- They enjoyed the style of working in and with research
dents’ thoughts and teachers’ observations. The most cited which differs from their regular school experience (33 re-
adjectives were (in descending order) interesting, challeng- sponses).
ing, exciting, inspiring, and fun, similarly to those from the
pilot (Archer, 2017), with the top two adjectives being sig- It was quite interesting to broaden our views and
nificantly greater than the subsequent ones. While in the pi- experience high level education. (Student 90, Hill
lot stage only positive adjectives were expressed, since then Valley High School, ATLAS 2019)
a few negative experiences have been conveyed, such as I enjoyed the opportunity to do science instead of
time-consuming, frustrating, and stressful. These constitute a just learning it. (Student 101, Octavian Country
small minority of experiences though (6 ± 1 %), and in most Day School, PHwP 2019)
cases the same students also listed positive adjectives, apart It was very nice to work with friends and work to-
from only four individuals. gether to produce something. (Student 106, Boston
Following on from these quantitative analyses, we quali- Bay College, PHwP 2019)
tatively explore the potential reasons behind the results. The
most common themes that emerged from students’ (n = 110) It was very fun to do our own research and I appre-
responses to open questions about their experience were that ciated that help was always available even though it
they feel they learnt a lot (62 responses). is very independent. It also shows how challenging
research can actually be but also how rewarding it
is once you start making progress. (Student 143,
Sunnydale High School, MUSICS 2020)
We have learnt so many new things relating to the
magnetosphere and waves and we have developed Neutral or negative experiences tended to be due to stu-
new skills. (Student 3, Xavier’s Institute for Higher dents finding the projects’ content or open-ended way of
Learning, MUSICS 2016) working difficult or confusing (seven responses).
Geosci. Commun., 4, 147–168, 2021 https://doi.org/10.5194/gc-4-147-2021M. O. Archer et al.: Physics research in school environments 157
Figure 4. Word cloud of students’ experiences. Colours indicate words identified by students (blue), teachers (green), or both (cyan). Students
and teachers have been given equal total weight.
I did not understand most of the project or what I Excellent project that is open-ended allowing stu-
was supposed to do. (Student 147, Jedi Academy, dents to take it where they want. (Teacher 27, Hog-
ATLAS 2020) warts, SCREAM 2019)
The vast majority of students seemed to ultimately enjoy This year I had an extremely motivated, enthusi-
this challenge though. astic and well-organised group of 7 students who
fully immersed themselves into the project and
The project gave us a lot of freedom and chal- quickly took it in a direction outside my own un-
lenged us to think in different ways. (Student 122, derstanding of this area of science. This is exactly
Jedi Academy, ATLAS 2020) the experience I wanted them to have, and they
[It] made me more willing to take a go at chal- were able to discover some genuinely novel pro-
lenges and what I deem hard. (Student 130, Bend- cesses that had not been observed before – the hall-
ing State College, PHwP 2020) mark of great scientific research! (Teacher 44, Sun-
nydale High School, MUSICS 2020)
Teachers agreed that the learning curve involved with the
projects was advantageous for students. Therefore, both quantitative and qualitative data suggest
students and teachers had much more positive and reward-
When the students got the hang of it they really en-
ing experiences participating in PRiSE projects than is typi-
joyed it. (Teacher 16, Tree Hill High School, MU-
cal for schools engagement programmes from universities in
SICS 2018)
general.
The students found it hard to identify what to do a
project on and would have liked guidance on that, 4.2 Support and resources
but I felt this was a good experience. They have
developed grappling with open ended and difficult We originally asked students whether they felt they had re-
material. (Teacher 23, Smeltings, ATLAS 2019) ceived adequate support, finding overall positive results on
a five-point Likert scale (Archer, 2017). However, students’
Teachers’ feedback on their experience (n = 34) tended to
qualitative responses explaining their answers often revealed
praise how the projects allow their students to access and ex-
a conflation of the support provided by Queen Mary with
plore beyond the curriculum (29 responses).
that offered by their teacher. Therefore, from 2019 onwards
A great framework to explore physics beyond the we explicitly separated these two aspects. Students (n = 68)
syllabus but still accessible. (Teacher 26, Octavian were asked “Do you feel that support from your teacher
Country Day School, PHwP 2019) was provided/available during the project?”, which yielded
https://doi.org/10.5194/gc-4-147-2021 Geosci. Commun., 4, 147–168, 2021158 M. O. Archer et al.: Physics research in school environments
the following results: Strongly Agree (30), Agree (34), Nei- My own experience with research was handy but I
ther Agree or Disagree (3), and Disagree (1). The aver- felt that without this the students would still have
age response is 4.37 ± 0.08, which is considerably greater been supported (Teacher 2, Hogwarts, SCREAM
than the benchmark on teacher support reported by Vennix 2016)
et al. (2017) of 3.60 ± 0.03 (p = 4 × 10−10 ). Students’ com- only through Martin’s support (Teacher 4, Sweet
ments explaining their ratings (n = 56) revealed that teachers Valley High School, MUSICS 2016)
provided them with advice, encouragement, and enthusiasm
(49 responses). the teacher version of the handout was useful,
but otherwise I could only give generic advice
My teacher has been very supportive and has (Teacher 16, Tree Hill High School, MUSICS
helped us when we didn’t understand something as 2018)
well as encouraging us to taking a more innovative
approach. (Student 124, Quirm College for Young Several teachers raised pressure on their time, with one
Ladies, MUSICS 2020) teacher using this to justify their negative response.
If we had a question, teachers were probably not sheer time pressure – big limiting factor
useful. But if we did not know what to do or we (Teacher 13, St Trinians, SCREAM 2017)
were stuck, here teachers were really useful and
that was what we needed. (Student 145, Sunnydale Another said this somewhat limited their ability to support
High School, MUSICS 2020) students.
They arranged regular sessions for students to meet and not as much as I would have liked (lack of avail-
visits or calls from the university when required (seven re- able time) (Teacher 3, Xavier’s Institute for Higher
sponses). Learning, MUSICS 2016)
Our teacher arranged a Skype call with a profes- However, most said it was manageable.
sor from QMUL when we needed to ask questions
about how certain parts of the data were calculated. The autonomous group work, with very little input
(Student 126, Harbor School, ATLAS 2020) from me, was great to see (Teacher 1, Hogwarts,
SCREAM 2015)
[Our] teacher would often ask us about it and hold
meetings to catch up with us on our progress. (Stu- Students were quite self-sufficient so if I made
dent 143, Sunnydale High School, MUSICS 2020) suggestions they were able to do the leg work
(Teacher 15, Spence Academy for Young Ladies,
[Our] teacher answered some of our questions and
MUSICS 2018)
organised a weekly meet-up where students could
ask each other questions and work together. (Stu- Teachers’ ability and confidence in supporting the projects
dent 148, Jedi Academy, ATLAS 2020) were other themes that emerged. Even with the teacher-
Neutral or negative responses tended to be explained by specific resources provided, some felt they did not have the
their teacher lacking specific knowledge about the research specific knowledge or skills to support the projects.
in response to students’ queries (two responses). [Unable to support due to a] lack of knowledge of
Although they were always ready to help, some- Python (Teacher 19, Boston Bay College, PHwP
times they didn’t know the answers to our ques- 2018)
tions. (Student 122, Jedi Academy, ATLAS 2020)
Other teachers reflected that, similarly to their students,
[They] didn’t understand the content of the project. they too had experienced a learning curve through their in-
(Student 139, Jedi Academy, ATLAS 2020) volvement.
This is something we do not expect of teachers (cf. Shah [I found it] difficult at first (Teacher 8, Coal Hill
and Martinez, 2016; Bennett et al., 2016, 2018), hence why School, MUSICS 2017)
support from the university is also offered.
Teachers’ (n = 18) responses on a yes/no scale (chosen They ultimately became more determined and confident
due to expected small number statistics) on whether they with time and in subsequent years.
felt able to support their students were also highly posi-
tive, with only two negative responses, a significant major- First time we’ve done this – I will do better next
ity (p = 0.001 in a two-tailed binomial test). Bear in mind, time (Teacher 17, Sunnydale High School, MU-
however, that these responses were in light of the support SICS 2018)
provided from the university, something which a few teach- Second year that I ran it I feel more confident
ers referenced in explaining their answers. (Teacher 21, Hogwarts, SCREAM 2018)
Geosci. Commun., 4, 147–168, 2021 https://doi.org/10.5194/gc-4-147-2021M. O. Archer et al.: Physics research in school environments 159
The final theme raised was that for successful participation
teachers believed the students needed the external motivation
to come from the university rather than having project deliv-
ery being solely teacher-driven.
Dr Archer was a great external lead to have. If I
had been pushing them myself they would have
taken it less seriously (Teacher 17, Sunnydale High
School, MUSICS 2018)
Therefore, the comments from both students and teach-
ers indicate that teachers alone would likely not have been
able to successfully support these research projects in their
schools without both the resources and external motiva-
tion/mentoring provided as part of the PRiSE framework.
We now consider the specific elements of support shown
in Fig. 1. From 2019 onwards we investigated participants’
thoughts on each of these various aspects offered. Students
(n = 68) and teachers (n = 23) were asked to rate the use- Figure 5. Usefulness of support provided to teachers (T, n = 23)
fulness of these as either “unimportant”, “helpful”, “essen- and students (S, n = 68). Results are divided (black lines and asso-
tial”, or “unsure”. This was chosen over a five-point Likert ciated error bars) into negative (red) and positive responses, with the
scale due to an expected low number of responses, particu- latter subdivided (grey lines and error bars) into “essential” (blue)
larly from teachers. Any unsure or blank responses are ne- and “helpful” (yellow) elements. Error bars denote standard (1σ )
glected, yielding 326 (out of a potential 408) student and 156 Clopper and Pearson (1934) intervals.
(out of 161) teacher responses. We divide these responses
into negatives (“unimportant”) and positives (“helpful” or
essary. This has been further elaborated on in teacher feed-
“essential”), though we acknowledge some may consider the
back.
“helpful” response to be neutral, and thus our analysis takes
both interpretations into account. The results are displayed It is very well set up and open ended and the sup-
in Fig. 5 for the individual elements as well as overall re- port received is magnificent. (Teacher 33, Boston
sults obtained from totalling all responses. Both students and Bay College, PHwP 2020)
teachers overall rated the elements positively – coding the
responses to values of 1 (negative) to 3 (essential), the over- Martin’s guide to help the students was a very good
all means were 2.62 ± 0.04 for teachers and 2.23 ± 0.03 for balance of useful guidance and allowing them to
students. The majority of teachers tended to give “essen- find their own way through. (Teacher 38, Royal
tial” ratings to most aspects, and while these majorities are Dominion College, MUSICS 2020)
not statistically significant in a two-tailed binomial test, the Truly excellent support from the Queen Mary
average value for each element was greater than 2 to high team! They have visited us multiple times and have
confidence (p < 0.002 in one-sample Wilcoxon signed-rank been so generous with their time. Students have
tests). Students, on the other hand, mostly rated each ele- learnt a great deal from them! (Teacher 44, Small-
ment as “helpful” as well as stating slightly more negative ville High School, PHwP 2020)
responses than teachers, though again all elements’ mean
scores (apart from the kick-off workshop at 2.15 ± 0.08)
were significantly greater than 2 (p < 0.023). While there 5 Feedback from the university sector
are some variations in scoring amongst the different support
elements, such as students and teachers respectively rating Based on the highly positive results from participants, we
researcher visits and communications (the latter of which in- think there is potential for the PRiSE framework to spread
cludes webinars and ad hoc emails) as the most essential, beyond QMUL and be applied to other institutions’ own ar-
these differences to each group’s overall results are slight and eas of physics (and perhaps even STEM more generally) re-
not statistically significant. One interpretation of this might search. We therefore wanted to assess how it is perceived by
be that most respondents answered unreflectively, ticking the those from the university sector with interests and/or exper-
same boxes for each item. However, no students and only tise in schools engagement. Feedback from our partner or-
three teachers gave the same answer in every category. This ganisations seemed promising, for example with the South
therefore suggests that all of the elements of support pro- East Physics Network including PRiSE in their public en-
vided as part of PRiSE are almost equally important and nec- gagement strategy (SEPnet, 2017), several of their member
https://doi.org/10.5194/gc-4-147-2021 Geosci. Commun., 4, 147–168, 2021You can also read
