NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
NSTA RGO Evaluation
September 21, 2010
By Hilarie B. Davis, Ed.D.
Bradford T. Davey, M.A.
Technology for Learning Consortium Inc.
1
Table of Contents
I. Overview and Purpose 3
II. Method 3
III. Findings
A. Target population for RGO 6
B. Promotion 15
C. Application process 21
D. Selection process 30
E. Prior to flight 32
F. Flight week 42
G. Follow up after flight week 47
H. Overall Timeline for RGO participants 51
I. Evaluation of Results 54
IV. Options for Future TFS RGO efforts 61
Workspace Documents
See http://rgotfs.pbworks.com for documents and detailed information cited
throughout this report (you must be registered to view this site)
2
I. Overview and Purpose
The purpose of this evaluation of the RGO program for K-12 teachers is to determine
effective practices and obstacles to recruitment, support, implementation, follow up and
evaluation with the resources available. Is it achieving what we want? Does the benefit
merit the cost? In Education Flight Projects, RGO is listed as a student project. Teachers
need to report in their 120-day follow up that they used this with their students. This
report addresses the core question – how does RGO need to be designed to be get the
teachers involved who will use it powerfully with their students and their schools over
time?
Timeframe of the Evaluation
From May to September 30, 2010
As a part of this report, full documents and details have been cataloged on in a private
web-based workspace at http://rgotfs.pbworks.com
II. Method
The evaluation will be conducted using a mixed methods approach.
Existing NES survey data will be located and analyzed.
Interviews with a random sample of participants (NASA Explorer School
teachers, NEAT RGO teachers, NSTA teachers-2010) will also be conducted to
better understand their motivations for participating, how they integrated the work
into their curriculum, their perceptions of the effect on students and what they
would say to other teachers considering this opportunity.
NASA staff involved with the program will also be interviewed for their
perceptions of the program structure, participants and for suggestions.
NSTA staff managing the 2010 program will be interviewed about the promotion,
application, and support structures.
RGO staff will be interviewed about the their perceptions of the participants and
program and suggestions for the future.
The selection panel for the 2010 cohort of past NSTA presidents will be surveyed
and a few interviewed for their perceptions of the application and review process.
A representative of Zero-G will be interviewed about how they promote and
operate their program. Teachers who participate in that model will also be
interviewed (2-3) about the effects of participation.
3
Sample questions to be addressed in the evaluation study
Target Which K-12 teachers can most benefit from this experience?
population What are their characteristics that make them interested and successful?
for RGO What are the benefits to the teachers who participate from their
perspective and from the staff’s perspective?
What are the barriers to their participation? (timeline, funding for travel,
timeframe -summer vs. school year, other) How do they overcome them?
What can we learn from other programs such as downlinks?
Promotion How has the program been promoted in the past?
How can the program be promoted to them without eliciting too many
applications that will result in a low percentage of acceptances?
What is the target number of applications per flight week opportunity? (5
to 1 ratio for 9-10 teams?) What is the best way to target the teachers who
will make the most of it?
How is the website structured? What information does it provide? How
often is it accessed? Where is it linked from and to? Do teachers find out
about the program from the website?
http://microgravityuniversity.jsc.nasa.gov/
How can it be improved to get a larger pool of applicants?
Application How can the application process be used to get the right people to apply?
process With adequate rigor and follow through? In others words, how can the
application process be rigorous enough to require an investment by the
applicant, and yet be doable by the teachers who are most likely to benefit
from it?
Would a more rigorous application be a deterrent? (NSTA application is
watered down undergraduate version)
Is what they are proposing beneficial to the schools? Become part of
classroom lessons or change how they teach?
What is a reasonable timeline for teachers from when they hear about it to
when they apply, to when they fly, to when they complete the follow up?
Are canned experiments the way to go? (like the Zero-G flights?)
Selection How are applications reviewed and accepted?
process Does the selection process result in the projects that are worthy and
teachers that follow through?
What are the characteristics of teachers and their applications who do not
participate, drop out, or fail to complete elements of the experience?
Support What kind of support do teachers need prior to flight week to be
prior to prepared?
flight How has the NSTA webinar approach been working for the teachers and
staff? Is it effective? Doable? Motivating? Learning from them? Take too
much time?
How does NES support teachers ahead of time?
Flight How closely aligned are teacher expectations and the actual experience?
week Did teachers achieve their objectives? Science? Personal? Were they able
to engage their students with it?
How does each of the activities contribute to the goals for the teachers?
Were there any missing pieces? Something they would have liked to
4
occur? Speak with more engineers? Scientists? Prep time, more or less?
Some groups are ready to go, others have to scramble to get ready.
Follow up What do teachers do with students and their schools after the flight week?
support What support do they need, or would appreciate for those activities?
after flight Webinars with NSTA participation? Value? NES follow up?
week Who do they share the experience with?
Evaluation What are the effects on the teachers, their schools and their students?
How are the effects currently evaluated and reported?
How do we ensure cooperation in evaluation from the participants?
Data Collection Sources
Surveys N=119
45 NES RGO teachers 2009-10
40 TFS NSTA team members 2010
7 NEAT RGO
27 NEAT teachers
Interviews N=46
9 NES and NEAT teachers
1 NES professional development coordinator
15 TFS NSTA applicants, drops, finishers
3 NSTA staff
3 NSTA selection panelists
2 Zero-G staff and participants
4 RGO staff
15 NSTA 2010 RGO teachers
3 NSTA staff
10 NSTA review panel
Ad hoc analysis N=230
230 NES end of experience surveys form 2004-2010
Analysis of documents
3 Websites, NES, Undergraduate, SEED
3 Annual reports 2008, 2009, 2010
Observation
Flight week Activities related to experiments, professional development
Webinars NSTA March 4, May 10, May 17, May 20, July 8
Data Analysis
Interview and survey data will be summarized and analyzed for themes across
stakeholders with particular attention to how to get teachers involved who will greatly
benefit from this unique experience. Discourse analysis techniques (Johnstone, 2002, and
Schiffrin et al, 2002)1 will be used to analyze qualitative data. Likert type question data
will be analyzed for mean, median, mode and frequency by item.
1
Johnstone, B. (2002). Discourse Analysis. Oxford: Blackwell.
Schiffrin, D. Tannen, D. & Hamilton, H. E. (eds.). (2001). Handbook of Discourse Analysis.
Oxford: Blackwell.
5
III. Findings
As presented in the methodology section, we made very attempt to get the ideas and
opinions of NES teachers who have participated, NEAT teachers who have not
participated, RGO staff, NSTA staff and past presidents who reviewed the applications,
and EFP/TFS staff. We analyzed the undergraduate programs (design and systems
versions) because they have a long history of success and the pre-college experiences
were based on the undergraduate model. We also analyzed the private experience (Zero-
G Corporation) for ideas on how to structure the program. In all our interviews we asked
people for their ideas on how to make this worthwhile for them and others. We observed
the TFS NSTA flight week to better understand the nature, purpose and benefits of the
experience. In this section, we discuss each program area based on the data we collected.
In the next section, we discuss how what we learned can be applied to the future of RGO
as sponsored by the Teaching from Space office.
A. Target population for RGO
A fundamental question for this project is to identify the teachers who can best take
advantage of this unique and high cost experience to enhance their students’ knowledge,
understanding and enthusiasm for STEM content and careers, their own understanding of
the science research process, and their school’s and colleagues enthusiasm for NASA and
STEM. Questions guiding our evaluation included the following:
• Which K-12 teachers can most benefit from this experience?
• What are their characteristics that make them interested and successful?
• What are the benefits to the teachers and students who participate?
Goals and Objectives
Clear goals and objectives are integral to identifying the target population. We can use
both an inductive and a deductive reasoning approach to clarify this area. In the inductive
approach, we look at past projects to see who benefitted from the program and in what
ways, and based on that derive goals and objectives. Deductively we can identify the
goals and objectives the experience could support, and identify the target populations that
would benefit.
A goal is defined by Miriam-Webster dictionary as, “the end toward which effort is
directed.” An objective is defined as, “involving or deriving from sense perception or
experience with actual objects, conditions, or phenomena.” In other words, goals are what
it is hoped will be achieved; objectives are the measurable components needed to achieve
that end. For example:
Goal: Knows about the bones in the human body.
Objectives:
• Will be able to name all of the bones in the human body as stated in the
6
medical textbook "The Human Body."
• Will be able to point out where the bones are in the body and describe their
function(s) in that location.
• Will be able to accurately describe the structure and function of the skeletal
system in relationship to the other systems in the body.
Inductive Approach
Which K-12 teachers can most benefit from this experience? What are their
characteristics that make them interested and successful? Who can benefit from the RGO
project? The answer to this question from all the groups surveyed and interviewed
focused on individuals that have a lot of drive to involve students in meaningful research-
based experiences. These teachers are highly motivated to integrate this experience into
their curriculum and into the life of their schools. They are adventurous with their
curriculum and in their outlook on life. They recognize that RGO is an unusual
opportunity that requires creativity, and great attention to detail, as well as the ability to
seek out and use input from others on their team, their mentor, and other experts to
improve the research. As the teachers put it,
They recognize what their district and NASA are contributing so that they might
be able to perform real research so they take this experience back to their district
and peers to enhance their programs overall.
The ideal candidate is a leader or mentor to his/her peers and looks to improve
the science and math programs of their school or district and not merely their
own classroom.
Everyone, especially those teachers who have done RGO, emphasize the need for
energetic teachers who are willing to work hard, put in long hours with their students
designing and testing their experiment, and pay close attention to detailed NASA
guidelines every step of the way, e.g.
They must be organized and come with a plan. However, they must recognize that
their plan will mostly likely change
Hard working and organized person able to follow commands from NASA
Naturally curious, patient, good listener and flexible, appreciates teamwork
Grade level and content expertise seem to be less important than a willingness to learn
and make a strong connection to the curriculum and the school goals, e.g. Knowledgeable
yet wanting to know more. Elementary teachers make the case that they can get students
excited early so they pursue STEM careers. Middle school teachers point out that have
more flexible curriculum, and high school teachers teach many sophisticated concepts
that can be made real through experiments in microgravity.
Adventurous with their curriculum, the type of teacher that isn't stuck in a rigid
curriculum plan
7
All grade levels benefit provided the teachers have sufficient background
The teachers need to be committed to NASA and its goals, and have the time and
flexibility in their curriculum to include the preparation and testing with students,
or do it as an after school program.
For students to get the most out of the RGO experience, the teacher needs to be someone
who supports student inquiry. They may be into problem-based learning, competitions, or
design challenges. Teachers put it this way:
A real listener, relates to kids, able to get them involved in problem solving
A teacher researcher familiar with problem-based learning
One who cares about students and subject matter. This really is about them so
they need to be involved in all aspects of preparation and the design process
Is used to working with student teams on projects
We also asked teachers who had participated, “What are the benefits to the teachers
and students who participate?” All the teachers who have participated would do it
again and recommend it to others because of its value as a research experience.
This is inquiry at its best! Real science, real data, real world!
Teachers are involved first-hand in the scientific process, and their students
experience this along with them. The parameters of the variables are so specific
and narrow, that it makes it very complex.
The kids used technology and a Wiki space I created to follow daily events -they
understood why a hypotheses was important and could see and participate in the
scientific method as we went through the process.
They describe the excitement students have when they realize they are designing an
experiment with NASA.
It was a real life experience for our elementary students. The excitement caught
like a fever- throughout the elementary schools and to the parents and
community. It brought everyone together and everyone was involved. It was the
chance of a lifetime, better than meeting Mickey Mouse!
This was an invaluable experience for me and my students! The science that we
delved into was fascinating and engaging.
Teachers develop a much better understanding of NASA, a greater appreciation
of the space program, and get excited about being involved with NASA on a
project.
8
For teachers, the research experience may be their first so they feel it enhances their
ability to teach the scientific process.
Many teachers do not have research experience so they get to experience what
they teach.
Teachers need to coordinate many things in order to be successful. Teachers and
students get to learn more about the work of NASA and STEM careers, and also
about forces and motion.
The complexity of the process engages everyone is detailed and collaborative work that is
not only engaging but also “fascinating” as one teacher put it.
Combining ideas to put more rigor into our classes. Cross-curricular
connections.
This is a practical application of theory. Kids see what's going on outside
classroom
From an inductive perspective, the teachers who teachers who feel they are successful
and benefit are knowledgeable about NASA and committed to engaging students in an
authentic design, build, and test research experience. From the inductive analysis, the
goals and objectives of the program could read:
Goals:
• Teachers use RGO to engage students, faculty and community to motivate them
to learn more about NASA STEM topics, processes and careers.
• Students learn how to do research by participating in a reduced gravity design,
build, data collection and analysis, and sharing of results for a reduced gravity
experiment.
Objectives
• Teachers will teach students about NASA, microgravity, experimental methods
and prior RGO experiments to facilitate student development of a proposal for a
RGO experiment.
• Teachers will facilitate student building and testing of an apparatus for the
experiment and ground tests of it in 1g for students to collect data, refine their
hypotheses, and improve the apparatus.
• Teachers will work with a NASA mentor throughout the process, connecting the
mentor with the students to improve the experiment.
• Teachers, with support from the NASA mentor, will conduct the experiment and
collect data in micro and hyper gravity, and share the results with students.
• Teachers will facilitate student analysis of the data, drawing conclusions and
sharing the results with wider audiences.
• Teachers will provide access and information about further NASA STEM
resources and opportunities to students, faculty and community.
9
Deductive Approach
For the deductive approach, we examine the goals, objectives and/or descriptions and
requirements of the TFS, NSTA TFS, Undergraduate and SEED RGO programs to
understand the potential target populations for each.
The goals and objectives stated on the NES RGO website2 are:
Goals
• Provide educators and student the opportunity to experience developing an
investigation just like scientists and engineers.
• Provide the students and teachers the opportunity to test their investigation in
a reduced gravity environment.
Objectives
• To provide students the opportunity to design, fabricate and evaluate an
investigation to be tested in reduced gravity environment.
• Provide the opportunity for educators to experience reduced gravity.
The TFS NSTA RGO announcement described the program:
NASA reduced gravity flight experiences offer educators the opportunity to
successfully propose, design, and fabricate a reduced gravity investigation of their
choice with their students; fly the experiment; conduct research in a microgravity
environment; and evaluate the investigation. Educators then share their findings
with their students (who are not permitted to fly) and emulate the nature of
inquiry to the larger education arena via a community experience of learning and
future flight participants. The opportunity offers educators the opportunity to
participate in first class immersive inquiry learning experiences and to engage,
educate, and inspire their students in the STEM disciplines using NASA unique
content and resources.
The application requires a description of the experiment, student hypotheses, correlation
to the classroom curricula, community and media outreach effort, community and family
involvement, professional development/research dissemination, letter of commitment
from the school administrator, and an optional 1-2 minute video. Teachers agree to
participate in pre and post flight web seminars and activities. From this information,
additional information from RGO staff and observations and interviews, the goals and
objectives could read:
Goals
• Students and teachers experience an immersive inquiry learning experience
using NASA unique content and resources
• Teachers are prepared to support students in doing inquiry
• Students and teachers share their inquiry experience with their school,
families and community
2
http://microgravityuniversity.jsc.nasa.gov/nes
10Objectives
• Teachers will facilitate students doing inquiry in the microgravity
environment with the support of a NASA mentor
• Students will design and fabricate a reduced gravity experiment, communicate
with teachers who fly the experiment and evaluate the investigation
• Teachers and administrators will facilitate sharing the investigation experience
and results with the community
The undergraduate design website implies a similar set of goals and objectives for
students in the description under the “About the Program.”3 These teams are also required
to have a faculty member but they are able to conduct the experiment in micro and hyper
gravity themselves.
The Reduced Gravity Student Flight Opportunities Program provides a unique
academic experience for undergraduate students to successfully propose, design,
fabricate, fly and evaluate a reduced gravity experiment of their choice over the
course of four-six months. The overall experience includes scientific research,
hands-on experimental design, test operations and educational/public outreach
activities.
Given this description and other information on the website, the goals and objectives for
this program could read:
Goals
• Undergraduate students learn how to do scientific research, hands-on
experimental design, test operations, and educational/public outreach activities
• Undergraduates have a motivating STEM career-related experience.
• Undergraduate faculty use RGO as a motivator for engaging students in NASA
STEM career studies.
Objectives
• The undergraduate RGO team will successfully propose, design, fabricate, fly and
evaluate a reduced gravity experiment of their choice over the course of four-six
months
• The undergraduate RGO team will do outreach to other students, faculty and the
community based on their experiment.
• The undergraduate RGO team will prepare a report on their RGO experiment.
• The undergraduate professor (RGO team advisor) will facilitate the student team
in the process; providing information, guidance, reviews and ensuring adherence
to NASA guidelines.
• The undergraduate professor (RGO team advisor) and RGO team will support
subsequent year’s teams from their institution by sharing information, lessons
learned, apparatus and results.
3
http://microgravityuniversity.jsc.nasa.gov/
11The undergraduate SEED website provides this description4 for the Systems Engineering
Educational Discovery program using the RGO.
This project offers a nationwide solicitation of student application aimed at
addressing systems engineering challenges within a microgravity environment.
NASA has identified ongoing projects that are systems engineering and reduced
gravity related.
The overall experience includes scientific research, hands-on investigational
design, test operations and educational/public outreach activities. SE
investigations will adhere to the same processes and procedures governing NASA
research and test flights so that SE students and faculty gain insight into the
workings of NASA and ensure participant and investigation safety guidelines are
followed.
The students attached to the selected proposals would then work with a NASA
principal investigator lead for that project, to prepare the experiment for flight. In
addition to student involvement, one university/college faculty member will be
invited to fly with each team. This will help to provide faculty members with
teaching materials in their classroom and can be used as a motivator to increase
their students' interest in systems engineering.
In addition to the student-based research, they will participate in a number of
Digital Learning Network events (videoconferences). Incorporated as part of the
NASA experience, and working in conjunction with other NASA and engineering
organizations, student teams will participate in up to three videoconferences as
part of a systems engineering design challenge through the Digital Learning
Network (DLN).
Conditions permitting, each investigation will be flown twice so that there can be
replication of the investigation and any problems encountered during the first
flight can be corrected during the second.
After returning to their respective campuses, flight team members conduct
technical outreach events for younger students. As part of the experience, teams
will be assigned mentors with specialties in systems engineering.
This program is clearly focused on systems engineering with students working with
NASA principal investigators on something NASA needs. Students are expected to
mentor younger university students in systems engineering (one student team member
must be a senior). This program’s goals and objectives are somewhat different than the
undergraduate design RGO experience. They might include:
4
http://microgravityuniversity.jsc.nasa.gov/se
12Goals
• Students address NASA systems engineering challenges within a microgravity
environment
• Students and faculty gain insights into the workings of NASA by working with a
principal investigator to design an experiment to be conducted in microgravity
that will provide data about a NASA identified need.
• Undergraduate faculty experience conducting an experiment in micro and hyper
gravity to use in their teaching to increase student interest in systems engineering
Objectives
• Students work with a NASA principal investigator lead to prepare the experiment
for flight, then analyze results and draw conclusions
• Students will learn from mentors and experts using videoconferencing
• Undergraduate student teams will conduct technical outreach events for younger
students at their institutions and act as mentors in systems engineering
• The undergraduate professor (RGO team advisor) will facilitate the student team
in the process; providing information, guidance, reviews and systems engineering
expertise
• The undergraduate professor (RGO team advisor) will use their RGO experience
in their teaching
Summary for Target Population Findings
Goals and objectives were defined for each program based on inductive and deductive
reasoning approaches. These goals and objectives define the target population as those
who can successfully accomplish the objectives. Based on the goals and objectives, the
application, requirements, and support for each program can also be defined. These clear
expectations will attract applications from teachers who feel they can meet them and also
help define the portfolio of opportunities available to teachers to use with their students
year after year, and the pipeline of NASA opportunities for students during their
academic years.
Teaching from Space and Education Flight Projects are based on the hypothesis that:
“… teachers and students can benefit from experience with flight-related projects
going on at NASA. Offering unique, NASA-only, experience can excite, inspire
and engage teachers to incorporate NASA resources into their curricula and
students to develop, pursue and sustain an interest in STEM topics and careers.”
(2009 Evaluation Report, p. 4).
The outcome objectives are: (2009 Evaluation Report, p. 7).
1. Teachers learn NASA related STEM content
2. Teachers connect NASA content to STEM literacy
3. Teachers and students conduct investigations with NASA resources
4. Students experience using tools for investigation
5. Students reflect on their NASA experience
RGO supports the TFS goals and objectives in that teachers use NASA resources to
prepare students to develop a proposal for a microgravity experience, learn about the
13investigation process in NASA, and support their students in conducting investigations
using tools and NASA resources (including the mentor) and analyzing the data from the
experiment in micro and hyper gravity. Students then reflect on their NASA experience
through their reports and presentations in their communities. If we refine the goals and
objectives created from the current description of the program (deductive approach) with
the insights gained from the interviews and surveys (inductive approach) and in light of
the TFS/EFP goals and objectives, they can be more focused for elementary, middle and
high school audiences leading up to participation in the undergraduate program. We will
expand on this idea of a developmental approach to RGO after reviewing the components
of the current experiences and formulating objectives for each level in the future. For
elementary, the goals are focused on building interest and enthusiasm for research and
NASA STEM. Middle school teachers report that the experience is very motivational for
their students and that they can also use it to teach research skills and content. High
school teachers use RGO for building interest, teaching curriculum concepts in an
authentic context, and for a career experience. Goals for each level could read:
Elementary Goals
• Teachers use RGO to engage students, faculty and the community to motivate
them to learn more about NASA STEM topics, processes and careers.
• Students learn what it is like to be a researcher by participating in a reduced
gravity experiment.
Middle School Goals
• Teachers use RGO to enhance their curriculum by engaging students in a
microgravity experiment.
• Students are motivated to learn more about NASA STEM topics, processes and
careers related to gravity.
• Teachers and students share their experience with their school and community to
motivate to others to learn more about NASA and STEM.
High School Goals
• Students and teachers experience an immersive inquiry learning experience using
NASA unique content and resources about gravity
• Teachers are prepared with knowledge and NASA resources to support students
in doing inquiry about gravity through designing and conducting an experiment in
hyper and microgravity
• Students and teachers share their inquiry experience and results with their school,
families and community to motivate to others to learn more about NASA and
STEM.
14B. Promotion
In this section we look at the ways the various RGO programs have been promoted, and
what we can learn from other programs. We also asked teachers how they found out
about it.
Questions that guided our investigation included:
• How has the program been promoted in the past?
• How can the program be promoted without eliciting too many applications that will
result in a low percentage of acceptances?
• What is the target number of applications per flight week opportunity? (5 to 1 ratio
for 9-10 teams?) What is the best way to target the teachers who will make the most
of it?
• How is the website structured? What information does it provide? How often is it
accessed? Where is it linked from, and to? Do teachers find out about the program
from the website? http://microgravityuniversity.jsc.nasa.gov/
TFS NSTA 2010 Promotion
NSTA was contracted to do only limited promotion of TFS RGO pilot opportunity
through NSTA internal networks. The first calls for applications were distributed through
the 18 district directors at the beginning of February 2010 (See Appendix A for copies of
the emails). Additional emails were sent to 300 or more people/groups on their Chapters
and Affiliated Groups (CAGs) lists with requests for distribution through their various
networks. In addition, NSTA selected certain schools (approximately 50 with multiple
teachers from each school on the distribution list) with a potential interest or existing
programs in aerospace education and sent another wave of emails to that target group. A
notice was also distributed through the NSTA Science Matters network. This resulted in
about 50 inquiries. The feedback NSTA received was that it was difficult to keep team
members involved when they found out they would be paying expenses for the 10-day
Houston trip.
The teachers who were selected reported the following sources:
Number of times Source
10 Colleague
10 NSTA email
5 DuPont (sponsor of Delaware teams)
2 Forwarded email (from outside source)
2 NSTA affiliate
2 STEM conference
2 My team leader
1 Regional in-service, NEAT, Florida Space Report, T-STEAM
program, twitter, science specialist from my school district, NASA
staff member
15Most of the teachers found out from a fellow teacher or from an email from NSTA. Many
of the teachers only received the last email that went to all the membership in March so
they had to move quickly to put a team together and submit an application.
It was expected that the personal invitations from NSTA board of directors and council
members would result in a good number of quality applications. NSTA chapters and
associated groups such as state science teacher associations were approached. In the
undergraduate program, RGO expects at least 70 applications for 14 team slots. When the
networking approach did not result in many applications, NSTA advertised to broader
and broader audiences.
The promotion timeline was also a factor and will be discussed in more detail in its own
section in the context of the whole timeline. Both NSTA and teaches would have liked to
have had more time to promote the program to get the word out about the opportunity.
NSTA is publicizing this year’s program through its website. Teams will be presenting
their research through a series of NSTA webinars in December 2010 that will be open to
the entire membership.
One of the concerns raised by the Teaching from Space staff was having too many
applications, causing disappointment for a large number of people. When we asked
teachers about it, almost all of them said they were used to competitions being difficult.
I told my students we were going to learn about microgravity and think about
what we would like to know about it, and that we might get to do our experiment,
and we might not, but either way we would learn a lot.
I think as long as it is clearly laid out how many spots there are and there is a
clear, fair process for selecting teams, it is not a problem to have a lot of
competition for a few slots. You know what you are getting into to.
I think this is such an amazing unusual opportunity; it should be hard to get in.
One person we interviewed described another STEM program on nanotechnology that
she saw as similar to RGO in that it is a narrow, sophisticated topic that requires creative,
flexible teachers to integrate it into their curricula. The nanotechnology program also has
a large time commitment; three years teachers including summer workshops and
implementation in their classrooms during the academic year. It was advertised to state
science supervisors and all 300,000+ members of NSTA nationally. As a result, 110
applications for 20 slots were received.
TFS RFO NSTA Past President reviewers also commented on the number of
applications, the application timeline and how to limit the number of applications by
having very clearly defined expectations:
There clearly were not enough applications. From discussion I had with some
folks who applied and those who decided not to apply, the time frame was simply
too short. More advance notice is needed. To be able to develop a project of this
caliber the applicants need more time to be able to collaborate.
16Certainly if more applications are received the number of quality applications
will increase and the percentage of success will be less, but I don't see that as a
deterrent.
The key thing is to be clear what are you looking for in the applicants. If it is
broad-based, not narrowly enough defined, you will get too many. If it is fits in
high school physical science courses (chem., physics, engineering, math,
technology courses), don’t be afraid to say so.
Some of the teachers said they knew about RGO and were always keeping an eye out for
when applications might open up, and they still heard about this one at the last minute
and only because of the NSTA email. When we asked teachers how to effectively
promote the program, they offered the following suggestions:
• Email, listservs, NASA e-blasts (to local teacher association representatives,
department heads, science teachers, content supervisors, principals)
• Web - the ones seeking out new opportunities will find it. Make sure there is a
website that comes up at the top of the search list. Put all the information about
applying on the website. Have applications open up at the same time every year
so we can plan.
• Through professional organizations such as AAPT, NABT, NCTM, ASA, NSTA,
National FFA, NEST, NAAE, NEST
• Presentations and booths at STEM teacher conferences or teacher workshops,
regional and national
• Have teachers who have done it “tell everyone”
• Regional in-service centers
Past NSTA presidents suggested promoting the program through NSTA publications,
such as Science Scope, The Science Teacher and NSTA Reports and the Journal of
Physics Teachers Association. Part of team’s follow up might be to contribute to an
article about the program.
This is pretty specific to a small group of interested teachers at the high school
level so I would say focus the contact list on those who are most appropriate and
able to use this. For example, biophysics would work, but not general biology.
Physics teachers or pre-engineering teachers could use it in their classes.
NES Promotion
Because NASA Explorer Schools have an ongoing relationship with NASA, the
leadership team in the school receives weekly “eblasts” on upcoming opportunities. RGO
was one of several “special opportunities” they could apply to in their second year of
participation beginning in 2004. The program began in 2003 with 50 schools and grew to
250 schools at its peak in 2006. While the number of applications for available slots has
not been recorded over the years, the staff reports it is generally 50% more than the
available slots. So for 10 team slots, they might get 16 applications. Some schools have
applied and been successful in participating more than once. New schools are encouraged
to apply and given preference over returning teams. All expenses are paid for travel to
Houston. Teams must pay for their own materials and expenses associated with building
17their experimental apparatus. Participation in NES has varied over the years depending
on funding more than interest.
Year 2004 2005 2006 2007 2008 2009 2010
# Teachers 17 * 30 70 48 40 45
*In 2005 the plane was not available to NES teachers
NES also has a website on its RGO professional development opportunity. A search on
“reduced gravity opportunity” results in the NES website showing up at the top of the list
followed by the undergraduate program website.
http://microgravityuniversity.jsc.nasa.gov/nes/
The experience of NES is instructive in that the barrier of cost of the Houston trip is
removed, and it is part of a strong professional development program; two assets in
recruiting teachers. Yet, they still do not have a large number of applicants. When asked,
teachers report that the time commitment, intensity, sophisticated content, and being out
of school for 10 days make it a program that only a few teachers are interested in or
qualified to follow through with.
It is important to note that NSTA partners with NES through Oklahoma State University
in implementing professional development. NSTA sends out the weekly eblasts prepared
by NES, manages registration and travel, and tracks participation.
NEAT teachers reported that they would like to be involved in the program if they could.
For example, several of them are no longer in the classroom, but they would be willing to
work with a team. Others have flown before and are under the impression they are not
allowed to participate again.
18Undergraduate Program Promotion
As mentioned earlier, the undergraduate program generally receives at least four to five
as many applications as there are slots. The program began in 1995 with four teams of
college seniors from Texas’ Rice and A&M Universities (called SURF Student
Understanding Reduced Gravity Flight). It was renamed in 1996, the “Reduced Gravity
Student Flight Opportunities Program (RGSFOP).
Year Time of yr # Teams # States # Institutions
1995 Summer 4 1 2 UG
1996 Summer 4 1 4 UG
1997 Spring 23 15 28 UG
1998 ? 46 25 37 UG
1999 Spring & Summer 76 29 33 UG
2000* Spring 47 25 38 UG
2001 Spring 82 37 63 UG
2002 Spring & Summer 50 28 43 UG
2003 Spring & Summer 68 31 54 UG
2004† Spring & Summer 68, 3 28,3 44 UG, 3 NES
2005* Spring 10 8 10 UG
2006† Spring & Summer 60, 11, 7, 43, 8 6, 6 26 UG, 11 NES,
6 7 Informal 6 WYP
2007† Spring & Summer 31, 20, 4 19, 17, 1 28 UG, 14 NES,
4 Special Opportunities
2008† Spring & Summer 40, 14, 11, 23, 14, 9, 33 UG, 14 NES,
10 9 11 NEAT, 10 SEED
2009† Spring & Summer 19, 10, 8, 17, 9, 6, 8 19 UG, 10 NES, 7 SEED,
13 13 Special Opportunities
2010† Spring & Summer 14, 13, 13, 12, 11, 10, 13 UG, 12 SEED,
11, 2, 1 8, 1, 1 18 NES/SEMAA/MUST,
11 TFS NSTA, 2 DOE,
1 HUNCH
Based on 2009 Annual Report, Appendix B plus info from RGO office
* Teams were shifted into next flight campaign year due to aircraft maintenance and delays
† Numbers are in order of the program list in the institution column
Over time, a program develops a history with institutions that promote it, and have an
oral history among the students. Many of the undergraduate institutions that participate
have aerospace degree programs so they see this as a pipeline opportunity for their
students. Their goals are aligned with the goals of RGO.
To promote the program, the website has information about how to apply and what the
program involves.
Microgravity University – http://microgravityuniversity.jsc.nasa.gov/
19In addition to the website, RGO does a mass mailing to universities with engineering and
science departments, their deans and presidents. RGO has compiled a database of
contacts at universities (over 3000 contacts). Past participants, USRP and other students
who have expressed an interest also receive emails. Community colleges are also
contacted.
Summary for Promotion Findings
The promotion of the RGO program needs to be both targeted and broad, and develop a
legacy with K-12 as it has done with undergraduates.
NSTA’s membership is a good place to start promoting RGO since science teachers
are the primary participants to date. Most of the successful teams had a strong science
person as the lead who had recruited other teachers to be members of the team. RGO
actually recommends a physical science teacher since that is the strongest content
connection. The RGO staff and the teachers who have participated in the past are both
quick to point out, however, that any teacher can be successful who understands the
content, is willing to work hard under the rigorous conditions required by NASA, and is
enough of a leader in his or her own school to involve the school and community in the
experience. From this year’s pilot, it would seem that starting broad, and starting early
would appear to be more effective than networking through smaller groups to find
appropriate teachers.
Mailings to Principals and Directors of Curriculum can get early institutional support
as we saw with several of the TFS NSTA teams. When the supervisor recruits the
teachers, he/she wants them to be successful.
20The risk of having too many applications seems to be minimal. None of the RGO
programs have had that problem. Other sophisticated programs, such as the
nanotechnology example given here, also do not seem to be overwhelmed with
applications. The time and cost barriers may also mitigate the number of applications.
Having programs at different times of the year (January to December and August –
June) may also increase the number of applications. Different teachers seem to prefer a
spring or summer flight based on their curriculum and testing schedules, and their
districts’ policy about absences. Some districts now require teachers to pay their own
substitutes when they are absent for opportunities like this. Other districts do not allow
their teachers to be absent from school (other than sickness) during test preparation
periods in the spring. Coaching schedules also eliminates travel for some teachers in
different seasons.
Targeting teachers who do this kind of activity in their schools may be a good strategy.
Who does other competitions? Who does NASA activities? These teachers have the
interest and skill set to consider an involved project like RGO. Creating partnerships
within and outside of NASA for co-advertising, or finding participant lists on the web
could promote RGO. Listservs for science teachers, particularly physical science
teachers, or for competitions for students are other potential avenues for promotion.
Identifying where RGO fits in the NASA portfolio of activities would support promotion
by feeder programs, and subsequent opportunities. Creating mentors for new teams could
lead to teachers who have participated recruiting other teachers for the chance to
participate again.
Creating buzz about the program will contribute to promotion. A website for TFS NSTA
like the NES website will make the information easier to find. It could have new
information all the time that even unsuccessful applicants would find useful and be able
to use in their classrooms. Requiring participants to reach out to other teachers could
create interest. Having local press involved (as in the undergraduate program where a
journalist can go with the team) might create more interest. Some participants talk about
having a huge contingent of people meeting them at the airport like they were rock stars.
Others say they could not seem to interest the press. More support for how the
participants can create buzz all along could help promote the program. NSTA articles and
news stories about the program at each stage of the process will build awareness. Having
a presence in social media such as Facebook builds a support base of teachers that other
teachers can tap into if they are thinking of applying.
Making year after year opportunities available will promote institutional interest. Like
the undergraduate program, institutions like programs they can do year after year. Maybe
different opportunities at each grade level could be identified, leading to a team doing
RGO. For example, students could do sounding rockets, or balloon sat, or DIME
Dropping in a Microgravity Environment. Or different levels of the program could be
offered. The lead teacher might participate over and over again but bring in new teachers
for the team, and new students. The lead provides the accumulated knowledge that
supports the team so more teachers and students are involved who might not attempt it on
their own.
21C. Application process
The application for TFS NSTA RGO was adapted to provide more scaffolding for
teachers and students than the undergraduate program offers. One of the objectives for
the undergraduates is to learn how to develop a full technical proposal. In the SEED
program, they write an essay that shows their knowledge of systems engineering since the
issue is defined by the NASA principal investigator (PI). Some of the questions we asked
about the application were:
• How can the application process be used to get the right people to apply? With
adequate rigor and follow through? In others words, how can the application process
be rigorous enough to require an investment by the applicant, and yet be doable by
the teachers who are most likely to benefit from it?
• Would a more rigorous application be a deterrent? (TFS NSTA application is
scaffolded)
• Should previously conducted successful experiments or apparatus be offered to
teams?
In this section we review the application forms and processes for TFS NSTA, NES,
Undergraduates and SEED and consider them in terms of the goals and objectives of each
program.
TFS NSTA Application
The application was part of the announcement sent out by NSTA so teachers received it
as part of an email. The application form was not online on a website or in forms based
tool. The application asked for the following information:
• Teacher team information
• The student experiment
• Student hypotheses and expected outcomes of the investigation in microgravity
• Correlation to classroom
• Community and media outreach effort
• Community and family involvement
• Professional development/research dissemination
• Letter of Commitment from the School Administrator
• Supplemental information - optional one-two minute video
For the full announcement and application see the workspace:
http://nasatfs.pbworks.com/RGO-Application
This version was developed to obtain the information RGO needs to evaluate the merit,
safety and technical requirements of each proposal. They expect to see procedures and
“down in the weeds” detail to evaluate its feasibility and merit.
The application also needs to demonstrate that the team understands the concept of
microgravity, the test conditions on board, and experimental research. In the webinars
and in interviews, the RGO staff emphasized that the investigations need to be
experimental with controls and variables that require data to be collected in micro and
22hyper gravity to test a hypothesis about how something works, under what conditions and
in what ways.
The experimental method is a systematic and scientific approach to research in
which the researcher manipulates one or more variables, and controls and
measures any change in other variables. http://www.experiment-
resources.com/experimental-research.html
Observational experiments are not appropriate; ones that ask the question, “What will
happen?” do not meet the goals of the program. This requirement for using the
experimental method is to ensure that the teams understand gravity and its affects, and
are investigating a question that can be answered in the Zero-G plane flight protocol. It
also supports the goal of students developing an understanding of how to do experimental
research.
Sections 7 & 8 address the experiment:
Part 7: The Student Experiment.
• What is the title of your investigation?
• How did you and your students come up with this investigation?
• What scientific concepts are being tested?
• How is gravity as a variable relevant to this investigation?
• List the tasks/phases of the project to be completed by students (examples – investigation build,
scientific research, planning trip to Houston, post-flight analyze, publish final report).
• Describe your testing procedure. What steps will you perform when the plane is in free fall?
What steps will you perform when the plane is in hyper-gravity?
• What do you expect to happen?
• Is this investigation a free-float?
• Has your school or school district flown this investigation in a prior flight on a NASA
Microgravity Aircraft?
Part 8: Student Hypotheses and Expected Outcomes of the Investigation in Microgravity.
Student Hypothesis/Outcome 1:
Student Hypothesis/Outcome 2:
Student Hypothesis/Outcome 3:
Teachers from the TFS NSTA group gave mixed reviews of the application. The biggest
concern was the time of year and short notice.
Great, but short notice this year (heard about it on spring break, busy time of
year)
Easy to follow - good communication
Long but important
I think the application process was fine
The process was much clearer and simpler when I did the program in 2007
23Concerns were about the redundancy of the some of the information required and the
length.
Some of the application process was unclear and repetitive
Too long and repetitive -very wordy. We need to read a sentence in the 15
seconds we have to check email and understand what we need to do. Use simple
language.
Make it clear and don't ask the same questions over and over. Have some of us
alumni look it over and offer suggestions. Perhaps we could be included in the
review process.
Make the application simple, look for teachers who reach a wider audience, and
get short administrator references
The teachers’ comments about redundancy may indicate a lack of understanding of the
experimental method and what is required in each section. Since one of the goals of the
program is to have teachers participate in a research experience to develop their
understanding of it and incorporate into it their teaching, it may be necessary to better
define what is required in each section, give detailed examples of successful applications,
offer proposal preparation sessions, and provide for questions in the application period
that are open to everyone. The National Science Foundation (NSF) uses this process for
their grant applications to ensure applicants have an adequate understanding of the goals
of the grant and the proposal format as well as cut down on the proposals that do not
conceptually fit the grant, or do not provide the information needed to evaluate a
proposal.
NSTA Past Presidents who reviewed the applications this year for TFS RGO were asked
about the applications. Their comments about the applications focus on the need for
clearly thinking through the experiment:
The application was sufficient. The applicants' ability to filling out the application
with major details was the major obstacle or their lack of imagination for having
a really good project.
My overall impression was that the people involved were very much committed to
the project and tried to write good applications. However, some were able to
articulate their project better than others.
Teams need to think through the whole plan and how it would react in 0g. They
need to get info earlier. This year, the timeline was too short. Teachers and
students need to get consultation ahead of time (engineering, practicality, safety,
solid physics, pre-engineering) and use local resources to actually put the
experiments together. It takes time. A lot of the applications seemed rushed.
To get further insight into the application process we asked teachers if a more rigorous
application would be a deterrent. About 40% said no, another 40% said yes, and about
2420% maybe. The teachers who felt a rigorous application was a good thing said it would
ensure you get people who are committed to dong all the work.
It will ensure enthusiasm about the program.
Make proposals mini TEDPs so you have a start on that (Technical Experiment
Data Package that the teachers and mentor complete before flying)
It needs to be difficult because of the nature of the program but not so hard that
would deter teachers because we are so busy. It needs to demonstrate proficiency
in technology and the content.
Teachers who said the current application was fine and should not be more rigorous
thought this provided enough information to understand the experiment and the
hypotheses.
It was doable but sufficient to get the commitment
The teachers who said “maybe” pointed out that the time it takes to complete a rigorous
application would be hard to justify if they are unlikely to get accepted. It was suggested
that a pre-proposal be submitted with feedback on whether they should pursue a full
proposal. This is similar to what many grants do with a letter of intent, followed by a pre-
proposal which is reviewed and either recommended for full proposal submission,
recommended with reservations or not recommended.
It depends. It is hard to invest that much time in an application for something
that may not be accepted. Perhaps there should be a pre-application, then a
formal application.
Many of the teachers we interviewed were eager for feedback on the quality of their ideas
and wanted more time in the development process.
Make a first announcement in fall. Have an early deadline for proposals with
lowest level of detail. Increase level of detail with subsequent deadlines and give
feedback.
Give more time from the announcement to the application deadline – it helps to
involve the students more.
Make it more about the students; get them more involved in the application
process.
To improve the process, teachers suggested that electronic applications, directions to
involve the whole team, not just the team lead, streamlining the forms, processes,
providing examples of prior successful applications and providing a timeline.
Make the application to be done by more than one principal applicant.
25Streamline the barrage of emails early on in the application process.
Put out a checklist of deadlines for the whole process so we know how to plan and
can write to that in the application.
We need more information on what we need to do, the initial presentation was not
enough. (There was a March 4th webinar on what to expect for people interested
in proposing)
Connect previous RGO schools with those involved currently to act as mentors or
guides early in the process.
Find way to take the edge off for students and teachers to complete an experiment
proposal - website with intro video designed to encourage proposals; examples of
proposals - builds - actual experiment
TFS RFO NSTA Past President reviewers had some suggestions for the application
process.
There may be a lead role for someone who teaches physics. Make it clear that
there are substantive roles for other people so it is not limited to upper level, or
AP students. Encourage applicants to looking for collaborators who have an
interesting role and can fit it into their classrooms.
It would be an obvious advantage to have more applications in relationship to the
quality of the proposals accepted. Also, the increased number of applicants
would help justify continued funding of the program. An increased number of
applicants also result in more students and teachers collaborating and thinking
"out of the box."
NES provides teachers and students with links to abstracts for prior experiments and to
resources to come up with ideas for their experiments5:
Teams may find researching topics below useful in deciding on a experiment area to study in Microgravity:
Physics, Physical Science, Chemistry, and Models (Designs). This link to Prior Campaigns may also help
show what kinds of experiments have been performed in the previous years.
Educator Guides to Microgravity may also serve as a guide to choosing an experiment:
• Microgravity: A Teacher’s Guide With Activities in Science, Mathematics, and Technology
• International Toys in Space: Science on the Station
• NASA – Microgravity Resources from CORE
• PSA (Personal Satellite Assistant) (Grades 5-8)
Teams are not limited to these areas but are encouraged to look for experiments that can give measurable,
repeatable data collection. Emphasize science in the proposal – there should be some theory, postulate, data
or calculation that leads you to believe that the phenomena you are investigating will react differently in
microgravity than in your 1-G lab. Simply proposing to do something “to see what happens” will probably
weaken your proposal’s technical merit. A well-stated hypothesis with underlying rationale on why you
believe something will happen will probably strengthen your proposal’s technical merit.
5
http://microgravityuniversity.jsc.nasa.gov/nes/theApplication/advice.cfm
26You can also read
