NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.

 
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
NSTA RGO Evaluation
 September 21, 2010

       By Hilarie B. Davis, Ed.D.
       Bradford T. Davey, M.A.
Technology for Learning Consortium Inc.

                   1
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
Table of Contents

I. Overview and Purpose                                                      3

II. Method                                                                   3

III. Findings
    A. Target population for RGO                                              6
    B. Promotion                                                             15
    C. Application process                                                   21
    D. Selection process                                                     30
    E. Prior to flight                                                       32
    F. Flight week                                                           42
    G. Follow up after flight week                                           47
    H. Overall Timeline for RGO participants                                 51
    I. Evaluation of Results                                                 54

IV. Options for Future TFS RGO efforts                                       61

Workspace Documents
See http://rgotfs.pbworks.com for documents and detailed information cited
throughout this report (you must be registered to view this site)

                                           2
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
I. Overview and Purpose

The purpose of this evaluation of the RGO program for K-12 teachers is to determine
effective practices and obstacles to recruitment, support, implementation, follow up and
evaluation with the resources available. Is it achieving what we want? Does the benefit
merit the cost? In Education Flight Projects, RGO is listed as a student project. Teachers
need to report in their 120-day follow up that they used this with their students. This
report addresses the core question – how does RGO need to be designed to be get the
teachers involved who will use it powerfully with their students and their schools over
time?

Timeframe of the Evaluation
From May to September 30, 2010

As a part of this report, full documents and details have been cataloged on in a private
web-based workspace at http://rgotfs.pbworks.com

II. Method

The evaluation will be conducted using a mixed methods approach.
    Existing NES survey data will be located and analyzed.
    Interviews with a random sample of participants (NASA Explorer School
       teachers, NEAT RGO teachers, NSTA teachers-2010) will also be conducted to
       better understand their motivations for participating, how they integrated the work
       into their curriculum, their perceptions of the effect on students and what they
       would say to other teachers considering this opportunity.
    NASA staff involved with the program will also be interviewed for their
       perceptions of the program structure, participants and for suggestions.
    NSTA staff managing the 2010 program will be interviewed about the promotion,
       application, and support structures.
    RGO staff will be interviewed about the their perceptions of the participants and
       program and suggestions for the future.
    The selection panel for the 2010 cohort of past NSTA presidents will be surveyed
       and a few interviewed for their perceptions of the application and review process.
    A representative of Zero-G will be interviewed about how they promote and
       operate their program. Teachers who participate in that model will also be
       interviewed (2-3) about the effects of participation.

                                             3
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
Sample questions to be addressed in the evaluation study
 Target      Which K-12 teachers can most benefit from this experience?
 population What are their characteristics that make them interested and successful?
 for RGO     What are the benefits to the teachers who participate from their
             perspective and from the staff’s perspective?
             What are the barriers to their participation? (timeline, funding for travel,
             timeframe -summer vs. school year, other) How do they overcome them?
             What can we learn from other programs such as downlinks?
 Promotion How has the program been promoted in the past?
             How can the program be promoted to them without eliciting too many
             applications that will result in a low percentage of acceptances?
             What is the target number of applications per flight week opportunity? (5
             to 1 ratio for 9-10 teams?) What is the best way to target the teachers who
             will make the most of it?
             How is the website structured? What information does it provide? How
             often is it accessed? Where is it linked from and to? Do teachers find out
             about the program from the website?
             http://microgravityuniversity.jsc.nasa.gov/
             How can it be improved to get a larger pool of applicants?
 Application How can the application process be used to get the right people to apply?
 process     With adequate rigor and follow through? In others words, how can the
             application process be rigorous enough to require an investment by the
             applicant, and yet be doable by the teachers who are most likely to benefit
             from it?
             Would a more rigorous application be a deterrent? (NSTA application is
             watered down undergraduate version)
             Is what they are proposing beneficial to the schools? Become part of
             classroom lessons or change how they teach?
             What is a reasonable timeline for teachers from when they hear about it to
             when they apply, to when they fly, to when they complete the follow up?
             Are canned experiments the way to go? (like the Zero-G flights?)
 Selection   How are applications reviewed and accepted?
 process     Does the selection process result in the projects that are worthy and
             teachers that follow through?
             What are the characteristics of teachers and their applications who do not
             participate, drop out, or fail to complete elements of the experience?
 Support     What kind of support do teachers need prior to flight week to be
 prior to    prepared?
 flight      How has the NSTA webinar approach been working for the teachers and
             staff? Is it effective? Doable? Motivating? Learning from them? Take too
             much time?
             How does NES support teachers ahead of time?
 Flight      How closely aligned are teacher expectations and the actual experience?
 week        Did teachers achieve their objectives? Science? Personal? Were they able
             to engage their students with it?
             How does each of the activities contribute to the goals for the teachers?
             Were there any missing pieces? Something they would have liked to

                                           4
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
occur? Speak with more engineers? Scientists? Prep time, more or less?
                   Some groups are ready to go, others have to scramble to get ready.
    Follow up      What do teachers do with students and their schools after the flight week?
    support        What support do they need, or would appreciate for those activities?
    after flight   Webinars with NSTA participation? Value? NES follow up?
    week           Who do they share the experience with?
    Evaluation     What are the effects on the teachers, their schools and their students?
                   How are the effects currently evaluated and reported?
                   How do we ensure cooperation in evaluation from the participants?

Data Collection Sources
         Surveys                 N=119
            45                   NES RGO teachers 2009-10
            40                   TFS NSTA team members 2010
             7                   NEAT RGO
            27                   NEAT teachers
       Interviews                N=46
             9                   NES and NEAT teachers
             1                   NES professional development coordinator
            15                   TFS NSTA applicants, drops, finishers
             3                   NSTA staff
             3                   NSTA selection panelists
             2                   Zero-G staff and participants
             4                   RGO staff
            15                   NSTA 2010 RGO teachers
             3                   NSTA staff
            10                   NSTA review panel
     Ad hoc analysis             N=230
           230                   NES end of experience surveys form 2004-2010
  Analysis of documents
             3                   Websites, NES, Undergraduate, SEED
             3                   Annual reports 2008, 2009, 2010
      Observation
       Flight week               Activities related to experiments, professional development
     Webinars NSTA               March 4, May 10, May 17, May 20, July 8

Data Analysis
Interview and survey data will be summarized and analyzed for themes across
stakeholders with particular attention to how to get teachers involved who will greatly
benefit from this unique experience. Discourse analysis techniques (Johnstone, 2002, and
Schiffrin et al, 2002)1 will be used to analyze qualitative data. Likert type question data
will be analyzed for mean, median, mode and frequency by item.

1
 Johnstone, B. (2002). Discourse Analysis. Oxford: Blackwell.
Schiffrin, D. Tannen, D. & Hamilton, H. E. (eds.). (2001). Handbook of Discourse Analysis.
Oxford: Blackwell.

                                                5
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
III. Findings

As presented in the methodology section, we made very attempt to get the ideas and
opinions of NES teachers who have participated, NEAT teachers who have not
participated, RGO staff, NSTA staff and past presidents who reviewed the applications,
and EFP/TFS staff. We analyzed the undergraduate programs (design and systems
versions) because they have a long history of success and the pre-college experiences
were based on the undergraduate model. We also analyzed the private experience (Zero-
G Corporation) for ideas on how to structure the program. In all our interviews we asked
people for their ideas on how to make this worthwhile for them and others. We observed
the TFS NSTA flight week to better understand the nature, purpose and benefits of the
experience. In this section, we discuss each program area based on the data we collected.
In the next section, we discuss how what we learned can be applied to the future of RGO
as sponsored by the Teaching from Space office.

A. Target population for RGO

A fundamental question for this project is to identify the teachers who can best take
advantage of this unique and high cost experience to enhance their students’ knowledge,
understanding and enthusiasm for STEM content and careers, their own understanding of
the science research process, and their school’s and colleagues enthusiasm for NASA and
STEM. Questions guiding our evaluation included the following:
    • Which K-12 teachers can most benefit from this experience?
    • What are their characteristics that make them interested and successful?
    • What are the benefits to the teachers and students who participate?

Goals and Objectives
Clear goals and objectives are integral to identifying the target population. We can use
both an inductive and a deductive reasoning approach to clarify this area. In the inductive
approach, we look at past projects to see who benefitted from the program and in what
ways, and based on that derive goals and objectives. Deductively we can identify the
goals and objectives the experience could support, and identify the target populations that
would benefit.

A goal is defined by Miriam-Webster dictionary as, “the end toward which effort is
directed.” An objective is defined as, “involving or deriving from sense perception or
experience with actual objects, conditions, or phenomena.” In other words, goals are what
it is hoped will be achieved; objectives are the measurable components needed to achieve
that end. For example:

       Goal: Knows about the bones in the human body.
       Objectives:
       • Will be able to name all of the bones in the human body as stated in the

                                             6
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
medical textbook "The Human Body."
       •   Will be able to point out where the bones are in the body and describe their
           function(s) in that location.
       •   Will be able to accurately describe the structure and function of the skeletal
           system in relationship to the other systems in the body.

Inductive Approach
Which K-12 teachers can most benefit from this experience? What are their
characteristics that make them interested and successful? Who can benefit from the RGO
project? The answer to this question from all the groups surveyed and interviewed
focused on individuals that have a lot of drive to involve students in meaningful research-
based experiences. These teachers are highly motivated to integrate this experience into
their curriculum and into the life of their schools. They are adventurous with their
curriculum and in their outlook on life. They recognize that RGO is an unusual
opportunity that requires creativity, and great attention to detail, as well as the ability to
seek out and use input from others on their team, their mentor, and other experts to
improve the research. As the teachers put it,

       They recognize what their district and NASA are contributing so that they might
       be able to perform real research so they take this experience back to their district
       and peers to enhance their programs overall.

       The ideal candidate is a leader or mentor to his/her peers and looks to improve
       the science and math programs of their school or district and not merely their
       own classroom.

Everyone, especially those teachers who have done RGO, emphasize the need for
energetic teachers who are willing to work hard, put in long hours with their students
designing and testing their experiment, and pay close attention to detailed NASA
guidelines every step of the way, e.g.

       They must be organized and come with a plan. However, they must recognize that
       their plan will mostly likely change

       Hard working and organized person able to follow commands from NASA

       Naturally curious, patient, good listener and flexible, appreciates teamwork

Grade level and content expertise seem to be less important than a willingness to learn
and make a strong connection to the curriculum and the school goals, e.g. Knowledgeable
yet wanting to know more. Elementary teachers make the case that they can get students
excited early so they pursue STEM careers. Middle school teachers point out that have
more flexible curriculum, and high school teachers teach many sophisticated concepts
that can be made real through experiments in microgravity.

       Adventurous with their curriculum, the type of teacher that isn't stuck in a rigid
       curriculum plan

                                              7
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
All grade levels benefit provided the teachers have sufficient background

       The teachers need to be committed to NASA and its goals, and have the time and
       flexibility in their curriculum to include the preparation and testing with students,
       or do it as an after school program.

For students to get the most out of the RGO experience, the teacher needs to be someone
who supports student inquiry. They may be into problem-based learning, competitions, or
design challenges. Teachers put it this way:

       A real listener, relates to kids, able to get them involved in problem solving

       A teacher researcher familiar with problem-based learning

       One who cares about students and subject matter. This really is about them so
       they need to be involved in all aspects of preparation and the design process

       Is used to working with student teams on projects

We also asked teachers who had participated, “What are the benefits to the teachers
and students who participate?” All the teachers who have participated would do it
again and recommend it to others because of its value as a research experience.

       This is inquiry at its best! Real science, real data, real world!

       Teachers are involved first-hand in the scientific process, and their students
       experience this along with them. The parameters of the variables are so specific
       and narrow, that it makes it very complex.

       The kids used technology and a Wiki space I created to follow daily events -they
       understood why a hypotheses was important and could see and participate in the
       scientific method as we went through the process.

They describe the excitement students have when they realize they are designing an
experiment with NASA.

       It was a real life experience for our elementary students. The excitement caught
       like a fever- throughout the elementary schools and to the parents and
       community. It brought everyone together and everyone was involved. It was the
       chance of a lifetime, better than meeting Mickey Mouse!

       This was an invaluable experience for me and my students! The science that we
       delved into was fascinating and engaging.

       Teachers develop a much better understanding of NASA, a greater appreciation
       of the space program, and get excited about being involved with NASA on a
       project.

                                             8
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
For teachers, the research experience may be their first so they feel it enhances their
ability to teach the scientific process.

       Many teachers do not have research experience so they get to experience what
       they teach.

       Teachers need to coordinate many things in order to be successful. Teachers and
       students get to learn more about the work of NASA and STEM careers, and also
       about forces and motion.

The complexity of the process engages everyone is detailed and collaborative work that is
not only engaging but also “fascinating” as one teacher put it.

       Combining ideas to put more rigor into our classes. Cross-curricular
       connections.

       This is a practical application of theory. Kids see what's going on outside
       classroom

From an inductive perspective, the teachers who teachers who feel they are successful
and benefit are knowledgeable about NASA and committed to engaging students in an
authentic design, build, and test research experience. From the inductive analysis, the
goals and objectives of the program could read:

   Goals:
   • Teachers use RGO to engage students, faculty and community to motivate them
      to learn more about NASA STEM topics, processes and careers.
   • Students learn how to do research by participating in a reduced gravity design,
      build, data collection and analysis, and sharing of results for a reduced gravity
      experiment.

   Objectives
   • Teachers will teach students about NASA, microgravity, experimental methods
      and prior RGO experiments to facilitate student development of a proposal for a
      RGO experiment.
   • Teachers will facilitate student building and testing of an apparatus for the
      experiment and ground tests of it in 1g for students to collect data, refine their
      hypotheses, and improve the apparatus.
   • Teachers will work with a NASA mentor throughout the process, connecting the
      mentor with the students to improve the experiment.
   • Teachers, with support from the NASA mentor, will conduct the experiment and
      collect data in micro and hyper gravity, and share the results with students.
   • Teachers will facilitate student analysis of the data, drawing conclusions and
      sharing the results with wider audiences.
   • Teachers will provide access and information about further NASA STEM
      resources and opportunities to students, faculty and community.

                                             9
NSTA RGO Evaluation September 21, 2010 - By Hilarie B. Davis, Ed.D. Bradford T. Davey, M.A. Technology for Learning Consortium Inc.
Deductive Approach
For the deductive approach, we examine the goals, objectives and/or descriptions and
requirements of the TFS, NSTA TFS, Undergraduate and SEED RGO programs to
understand the potential target populations for each.

The goals and objectives stated on the NES RGO website2 are:

          Goals
          • Provide educators and student the opportunity to experience developing an
             investigation just like scientists and engineers.
          • Provide the students and teachers the opportunity to test their investigation in
             a reduced gravity environment.

          Objectives
          • To provide students the opportunity to design, fabricate and evaluate an
             investigation to be tested in reduced gravity environment.
          • Provide the opportunity for educators to experience reduced gravity.

The TFS NSTA RGO announcement described the program:

          NASA reduced gravity flight experiences offer educators the opportunity to
          successfully propose, design, and fabricate a reduced gravity investigation of their
          choice with their students; fly the experiment; conduct research in a microgravity
          environment; and evaluate the investigation. Educators then share their findings
          with their students (who are not permitted to fly) and emulate the nature of
          inquiry to the larger education arena via a community experience of learning and
          future flight participants. The opportunity offers educators the opportunity to
          participate in first class immersive inquiry learning experiences and to engage,
          educate, and inspire their students in the STEM disciplines using NASA unique
          content and resources.

The application requires a description of the experiment, student hypotheses, correlation
to the classroom curricula, community and media outreach effort, community and family
involvement, professional development/research dissemination, letter of commitment
from the school administrator, and an optional 1-2 minute video. Teachers agree to
participate in pre and post flight web seminars and activities. From this information,
additional information from RGO staff and observations and interviews, the goals and
objectives could read:

          Goals
          • Students and teachers experience an immersive inquiry learning experience
             using NASA unique content and resources
          • Teachers are prepared to support students in doing inquiry
          • Students and teachers share their inquiry experience with their school,
             families and community

2
    http://microgravityuniversity.jsc.nasa.gov/nes
                                               10
Objectives
          • Teachers will facilitate students doing inquiry in the microgravity
             environment with the support of a NASA mentor
          • Students will design and fabricate a reduced gravity experiment, communicate
             with teachers who fly the experiment and evaluate the investigation
          • Teachers and administrators will facilitate sharing the investigation experience
             and results with the community

The undergraduate design website implies a similar set of goals and objectives for
students in the description under the “About the Program.”3 These teams are also required
to have a faculty member but they are able to conduct the experiment in micro and hyper
gravity themselves.

          The Reduced Gravity Student Flight Opportunities Program provides a unique
          academic experience for undergraduate students to successfully propose, design,
          fabricate, fly and evaluate a reduced gravity experiment of their choice over the
          course of four-six months. The overall experience includes scientific research,
          hands-on experimental design, test operations and educational/public outreach
          activities.

Given this description and other information on the website, the goals and objectives for
this program could read:

      Goals
      • Undergraduate students learn how to do scientific research, hands-on
         experimental design, test operations, and educational/public outreach activities
      • Undergraduates have a motivating STEM career-related experience.
      • Undergraduate faculty use RGO as a motivator for engaging students in NASA
         STEM career studies.

      Objectives
      • The undergraduate RGO team will successfully propose, design, fabricate, fly and
         evaluate a reduced gravity experiment of their choice over the course of four-six
         months
      • The undergraduate RGO team will do outreach to other students, faculty and the
         community based on their experiment.
      • The undergraduate RGO team will prepare a report on their RGO experiment.
      • The undergraduate professor (RGO team advisor) will facilitate the student team
         in the process; providing information, guidance, reviews and ensuring adherence
         to NASA guidelines.
      • The undergraduate professor (RGO team advisor) and RGO team will support
         subsequent year’s teams from their institution by sharing information, lessons
         learned, apparatus and results.

3
    http://microgravityuniversity.jsc.nasa.gov/
                                              11
The undergraduate SEED website provides this description4 for the Systems Engineering
Educational Discovery program using the RGO.

          This project offers a nationwide solicitation of student application aimed at
          addressing systems engineering challenges within a microgravity environment.
          NASA has identified ongoing projects that are systems engineering and reduced
          gravity related.

          The overall experience includes scientific research, hands-on investigational
          design, test operations and educational/public outreach activities. SE
          investigations will adhere to the same processes and procedures governing NASA
          research and test flights so that SE students and faculty gain insight into the
          workings of NASA and ensure participant and investigation safety guidelines are
          followed.

          The students attached to the selected proposals would then work with a NASA
          principal investigator lead for that project, to prepare the experiment for flight. In
          addition to student involvement, one university/college faculty member will be
          invited to fly with each team. This will help to provide faculty members with
          teaching materials in their classroom and can be used as a motivator to increase
          their students' interest in systems engineering.

          In addition to the student-based research, they will participate in a number of
          Digital Learning Network events (videoconferences). Incorporated as part of the
          NASA experience, and working in conjunction with other NASA and engineering
          organizations, student teams will participate in up to three videoconferences as
          part of a systems engineering design challenge through the Digital Learning
          Network (DLN).

          Conditions permitting, each investigation will be flown twice so that there can be
          replication of the investigation and any problems encountered during the first
          flight can be corrected during the second.

          After returning to their respective campuses, flight team members conduct
          technical outreach events for younger students. As part of the experience, teams
          will be assigned mentors with specialties in systems engineering.

This program is clearly focused on systems engineering with students working with
NASA principal investigators on something NASA needs. Students are expected to
mentor younger university students in systems engineering (one student team member
must be a senior). This program’s goals and objectives are somewhat different than the
undergraduate design RGO experience. They might include:

4
    http://microgravityuniversity.jsc.nasa.gov/se
                                                12
Goals
   • Students address NASA systems engineering challenges within a microgravity
      environment
   • Students and faculty gain insights into the workings of NASA by working with a
      principal investigator to design an experiment to be conducted in microgravity
      that will provide data about a NASA identified need.
   • Undergraduate faculty experience conducting an experiment in micro and hyper
      gravity to use in their teaching to increase student interest in systems engineering

   Objectives
   • Students work with a NASA principal investigator lead to prepare the experiment
      for flight, then analyze results and draw conclusions
   • Students will learn from mentors and experts using videoconferencing
   • Undergraduate student teams will conduct technical outreach events for younger
      students at their institutions and act as mentors in systems engineering
   • The undergraduate professor (RGO team advisor) will facilitate the student team
      in the process; providing information, guidance, reviews and systems engineering
      expertise
   • The undergraduate professor (RGO team advisor) will use their RGO experience
      in their teaching

Summary for Target Population Findings
Goals and objectives were defined for each program based on inductive and deductive
reasoning approaches. These goals and objectives define the target population as those
who can successfully accomplish the objectives. Based on the goals and objectives, the
application, requirements, and support for each program can also be defined. These clear
expectations will attract applications from teachers who feel they can meet them and also
help define the portfolio of opportunities available to teachers to use with their students
year after year, and the pipeline of NASA opportunities for students during their
academic years.

Teaching from Space and Education Flight Projects are based on the hypothesis that:
       “… teachers and students can benefit from experience with flight-related projects
       going on at NASA. Offering unique, NASA-only, experience can excite, inspire
       and engage teachers to incorporate NASA resources into their curricula and
       students to develop, pursue and sustain an interest in STEM topics and careers.”
       (2009 Evaluation Report, p. 4).

The outcome objectives are: (2009 Evaluation Report, p. 7).
       1. Teachers learn NASA related STEM content
       2. Teachers connect NASA content to STEM literacy
       3. Teachers and students conduct investigations with NASA resources
       4. Students experience using tools for investigation
       5. Students reflect on their NASA experience

RGO supports the TFS goals and objectives in that teachers use NASA resources to
prepare students to develop a proposal for a microgravity experience, learn about the
                                            13
investigation process in NASA, and support their students in conducting investigations
using tools and NASA resources (including the mentor) and analyzing the data from the
experiment in micro and hyper gravity. Students then reflect on their NASA experience
through their reports and presentations in their communities. If we refine the goals and
objectives created from the current description of the program (deductive approach) with
the insights gained from the interviews and surveys (inductive approach) and in light of
the TFS/EFP goals and objectives, they can be more focused for elementary, middle and
high school audiences leading up to participation in the undergraduate program. We will
expand on this idea of a developmental approach to RGO after reviewing the components
of the current experiences and formulating objectives for each level in the future. For
elementary, the goals are focused on building interest and enthusiasm for research and
NASA STEM. Middle school teachers report that the experience is very motivational for
their students and that they can also use it to teach research skills and content. High
school teachers use RGO for building interest, teaching curriculum concepts in an
authentic context, and for a career experience. Goals for each level could read:

Elementary Goals
   • Teachers use RGO to engage students, faculty and the community to motivate
      them to learn more about NASA STEM topics, processes and careers.
   • Students learn what it is like to be a researcher by participating in a reduced
      gravity experiment.

Middle School Goals
   • Teachers use RGO to enhance their curriculum by engaging students in a
      microgravity experiment.
   • Students are motivated to learn more about NASA STEM topics, processes and
      careers related to gravity.
   • Teachers and students share their experience with their school and community to
      motivate to others to learn more about NASA and STEM.

High School Goals
   • Students and teachers experience an immersive inquiry learning experience using
      NASA unique content and resources about gravity
   • Teachers are prepared with knowledge and NASA resources to support students
      in doing inquiry about gravity through designing and conducting an experiment in
      hyper and microgravity
   • Students and teachers share their inquiry experience and results with their school,
      families and community to motivate to others to learn more about NASA and
      STEM.

                                          14
B. Promotion

In this section we look at the ways the various RGO programs have been promoted, and
what we can learn from other programs. We also asked teachers how they found out
about it.

Questions that guided our investigation included:
• How has the program been promoted in the past?
• How can the program be promoted without eliciting too many applications that will
   result in a low percentage of acceptances?
• What is the target number of applications per flight week opportunity? (5 to 1 ratio
   for 9-10 teams?) What is the best way to target the teachers who will make the most
   of it?
• How is the website structured? What information does it provide? How often is it
   accessed? Where is it linked from, and to? Do teachers find out about the program
   from the website? http://microgravityuniversity.jsc.nasa.gov/

TFS NSTA 2010 Promotion
NSTA was contracted to do only limited promotion of TFS RGO pilot opportunity
through NSTA internal networks. The first calls for applications were distributed through
the 18 district directors at the beginning of February 2010 (See Appendix A for copies of
the emails). Additional emails were sent to 300 or more people/groups on their Chapters
and Affiliated Groups (CAGs) lists with requests for distribution through their various
networks. In addition, NSTA selected certain schools (approximately 50 with multiple
teachers from each school on the distribution list) with a potential interest or existing
programs in aerospace education and sent another wave of emails to that target group. A
notice was also distributed through the NSTA Science Matters network. This resulted in
about 50 inquiries. The feedback NSTA received was that it was difficult to keep team
members involved when they found out they would be paying expenses for the 10-day
Houston trip.

The teachers who were selected reported the following sources:
Number of times Source
       10         Colleague
       10         NSTA email
        5         DuPont (sponsor of Delaware teams)
        2         Forwarded email (from outside source)
        2         NSTA affiliate
        2         STEM conference
        2         My team leader
        1         Regional in-service, NEAT, Florida Space Report, T-STEAM
                  program, twitter, science specialist from my school district, NASA
                  staff member

                                           15
Most of the teachers found out from a fellow teacher or from an email from NSTA. Many
of the teachers only received the last email that went to all the membership in March so
they had to move quickly to put a team together and submit an application.

It was expected that the personal invitations from NSTA board of directors and council
members would result in a good number of quality applications. NSTA chapters and
associated groups such as state science teacher associations were approached. In the
undergraduate program, RGO expects at least 70 applications for 14 team slots. When the
networking approach did not result in many applications, NSTA advertised to broader
and broader audiences.

The promotion timeline was also a factor and will be discussed in more detail in its own
section in the context of the whole timeline. Both NSTA and teaches would have liked to
have had more time to promote the program to get the word out about the opportunity.
NSTA is publicizing this year’s program through its website. Teams will be presenting
their research through a series of NSTA webinars in December 2010 that will be open to
the entire membership.

One of the concerns raised by the Teaching from Space staff was having too many
applications, causing disappointment for a large number of people. When we asked
teachers about it, almost all of them said they were used to competitions being difficult.

       I told my students we were going to learn about microgravity and think about
       what we would like to know about it, and that we might get to do our experiment,
       and we might not, but either way we would learn a lot.

       I think as long as it is clearly laid out how many spots there are and there is a
       clear, fair process for selecting teams, it is not a problem to have a lot of
       competition for a few slots. You know what you are getting into to.

       I think this is such an amazing unusual opportunity; it should be hard to get in.

One person we interviewed described another STEM program on nanotechnology that
she saw as similar to RGO in that it is a narrow, sophisticated topic that requires creative,
flexible teachers to integrate it into their curricula. The nanotechnology program also has
a large time commitment; three years teachers including summer workshops and
implementation in their classrooms during the academic year. It was advertised to state
science supervisors and all 300,000+ members of NSTA nationally. As a result, 110
applications for 20 slots were received.

TFS RFO NSTA Past President reviewers also commented on the number of
applications, the application timeline and how to limit the number of applications by
having very clearly defined expectations:

       There clearly were not enough applications. From discussion I had with some
       folks who applied and those who decided not to apply, the time frame was simply
       too short. More advance notice is needed. To be able to develop a project of this
       caliber the applicants need more time to be able to collaborate.
                                             16
Certainly if more applications are received the number of quality applications
       will increase and the percentage of success will be less, but I don't see that as a
       deterrent.

       The key thing is to be clear what are you looking for in the applicants. If it is
       broad-based, not narrowly enough defined, you will get too many. If it is fits in
       high school physical science courses (chem., physics, engineering, math,
       technology courses), don’t be afraid to say so.

Some of the teachers said they knew about RGO and were always keeping an eye out for
when applications might open up, and they still heard about this one at the last minute
and only because of the NSTA email. When we asked teachers how to effectively
promote the program, they offered the following suggestions:

   •   Email, listservs, NASA e-blasts (to local teacher association representatives,
       department heads, science teachers, content supervisors, principals)
   •   Web - the ones seeking out new opportunities will find it. Make sure there is a
       website that comes up at the top of the search list. Put all the information about
       applying on the website. Have applications open up at the same time every year
       so we can plan.
   •   Through professional organizations such as AAPT, NABT, NCTM, ASA, NSTA,
       National FFA, NEST, NAAE, NEST
   •   Presentations and booths at STEM teacher conferences or teacher workshops,
       regional and national
   •   Have teachers who have done it “tell everyone”
   •   Regional in-service centers

Past NSTA presidents suggested promoting the program through NSTA publications,
such as Science Scope, The Science Teacher and NSTA Reports and the Journal of
Physics Teachers Association. Part of team’s follow up might be to contribute to an
article about the program.

       This is pretty specific to a small group of interested teachers at the high school
       level so I would say focus the contact list on those who are most appropriate and
       able to use this. For example, biophysics would work, but not general biology.
       Physics teachers or pre-engineering teachers could use it in their classes.

NES Promotion
Because NASA Explorer Schools have an ongoing relationship with NASA, the
leadership team in the school receives weekly “eblasts” on upcoming opportunities. RGO
was one of several “special opportunities” they could apply to in their second year of
participation beginning in 2004. The program began in 2003 with 50 schools and grew to
250 schools at its peak in 2006. While the number of applications for available slots has
not been recorded over the years, the staff reports it is generally 50% more than the
available slots. So for 10 team slots, they might get 16 applications. Some schools have
applied and been successful in participating more than once. New schools are encouraged
to apply and given preference over returning teams. All expenses are paid for travel to
Houston. Teams must pay for their own materials and expenses associated with building
                                            17
their experimental apparatus. Participation in NES has varied over the years depending
on funding more than interest.

                     Year 2004 2005 2006 2007 2008 2009 2010
               # Teachers     17 *          30     70      48    40    45
                  *In 2005 the plane was not available to NES teachers

NES also has a website on its RGO professional development opportunity. A search on
“reduced gravity opportunity” results in the NES website showing up at the top of the list
followed by the undergraduate program website.

http://microgravityuniversity.jsc.nasa.gov/nes/

The experience of NES is instructive in that the barrier of cost of the Houston trip is
removed, and it is part of a strong professional development program; two assets in
recruiting teachers. Yet, they still do not have a large number of applicants. When asked,
teachers report that the time commitment, intensity, sophisticated content, and being out
of school for 10 days make it a program that only a few teachers are interested in or
qualified to follow through with.

It is important to note that NSTA partners with NES through Oklahoma State University
in implementing professional development. NSTA sends out the weekly eblasts prepared
by NES, manages registration and travel, and tracks participation.

NEAT teachers reported that they would like to be involved in the program if they could.
For example, several of them are no longer in the classroom, but they would be willing to
work with a team. Others have flown before and are under the impression they are not
allowed to participate again.

                                            18
Undergraduate Program Promotion
As mentioned earlier, the undergraduate program generally receives at least four to five
as many applications as there are slots. The program began in 1995 with four teams of
college seniors from Texas’ Rice and A&M Universities (called SURF Student
Understanding Reduced Gravity Flight). It was renamed in 1996, the “Reduced Gravity
Student Flight Opportunities Program (RGSFOP).

  Year          Time of yr           # Teams        # States                # Institutions
 1995             Summer                 4             1                        2 UG
 1996             Summer                 4             1                        4 UG
 1997              Spring                23            15                      28 UG
 1998                ?                   46            25                      37 UG
 1999        Spring & Summer             76            29                      33 UG
 2000*             Spring                47            25                      38 UG
 2001              Spring                82            37                      63 UG
 2002        Spring & Summer             50            28                      43 UG
 2003        Spring & Summer             68            31                      54 UG
 2004†       Spring & Summer           68, 3          28,3                 44 UG, 3 NES
 2005*             Spring                10            8                       10 UG
 2006†       Spring & Summer         60, 11, 7,    43, 8 6, 6             26 UG, 11 NES,
                                         6                              7 Informal 6 WYP
 2007†       Spring & Summer         31, 20, 4       19, 17, 1        28 UG, 14 NES,
                                                                  4 Special Opportunities
 2008†       Spring & Summer           40, 14, 11, 23, 14, 9,             33 UG, 14 NES,
                                           10             9            11 NEAT, 10 SEED
 2009†       Spring & Summer           19, 10, 8, 17, 9, 6, 8 19 UG, 10 NES, 7 SEED,
                                           13                     13 Special Opportunities
 2010†       Spring & Summer           14, 13, 13, 12, 11, 10,           13 UG, 12 SEED,
                                        11, 2, 1       8, 1, 1       18 NES/SEMAA/MUST,
                                                                      11 TFS NSTA, 2 DOE,
                                                                             1 HUNCH
Based on 2009 Annual Report, Appendix B plus info from RGO office
* Teams were shifted into next flight campaign year due to aircraft maintenance and delays
† Numbers are in order of the program list in the institution column

Over time, a program develops a history with institutions that promote it, and have an
oral history among the students. Many of the undergraduate institutions that participate
have aerospace degree programs so they see this as a pipeline opportunity for their
students. Their goals are aligned with the goals of RGO.

To promote the program, the website has information about how to apply and what the
program involves.

Microgravity University – http://microgravityuniversity.jsc.nasa.gov/

                                              19
In addition to the website, RGO does a mass mailing to universities with engineering and
science departments, their deans and presidents. RGO has compiled a database of
contacts at universities (over 3000 contacts). Past participants, USRP and other students
who have expressed an interest also receive emails. Community colleges are also
contacted.

Summary for Promotion Findings
The promotion of the RGO program needs to be both targeted and broad, and develop a
legacy with K-12 as it has done with undergraduates.

NSTA’s membership is a good place to start promoting RGO since science teachers
are the primary participants to date. Most of the successful teams had a strong science
person as the lead who had recruited other teachers to be members of the team. RGO
actually recommends a physical science teacher since that is the strongest content
connection. The RGO staff and the teachers who have participated in the past are both
quick to point out, however, that any teacher can be successful who understands the
content, is willing to work hard under the rigorous conditions required by NASA, and is
enough of a leader in his or her own school to involve the school and community in the
experience. From this year’s pilot, it would seem that starting broad, and starting early
would appear to be more effective than networking through smaller groups to find
appropriate teachers.

Mailings to Principals and Directors of Curriculum can get early institutional support
as we saw with several of the TFS NSTA teams. When the supervisor recruits the
teachers, he/she wants them to be successful.

                                            20
The risk of having too many applications seems to be minimal. None of the RGO
programs have had that problem. Other sophisticated programs, such as the
nanotechnology example given here, also do not seem to be overwhelmed with
applications. The time and cost barriers may also mitigate the number of applications.

Having programs at different times of the year (January to December and August –
June) may also increase the number of applications. Different teachers seem to prefer a
spring or summer flight based on their curriculum and testing schedules, and their
districts’ policy about absences. Some districts now require teachers to pay their own
substitutes when they are absent for opportunities like this. Other districts do not allow
their teachers to be absent from school (other than sickness) during test preparation
periods in the spring. Coaching schedules also eliminates travel for some teachers in
different seasons.

Targeting teachers who do this kind of activity in their schools may be a good strategy.
Who does other competitions? Who does NASA activities? These teachers have the
interest and skill set to consider an involved project like RGO. Creating partnerships
within and outside of NASA for co-advertising, or finding participant lists on the web
could promote RGO. Listservs for science teachers, particularly physical science
teachers, or for competitions for students are other potential avenues for promotion.
Identifying where RGO fits in the NASA portfolio of activities would support promotion
by feeder programs, and subsequent opportunities. Creating mentors for new teams could
lead to teachers who have participated recruiting other teachers for the chance to
participate again.

Creating buzz about the program will contribute to promotion. A website for TFS NSTA
like the NES website will make the information easier to find. It could have new
information all the time that even unsuccessful applicants would find useful and be able
to use in their classrooms. Requiring participants to reach out to other teachers could
create interest. Having local press involved (as in the undergraduate program where a
journalist can go with the team) might create more interest. Some participants talk about
having a huge contingent of people meeting them at the airport like they were rock stars.
Others say they could not seem to interest the press. More support for how the
participants can create buzz all along could help promote the program. NSTA articles and
news stories about the program at each stage of the process will build awareness. Having
a presence in social media such as Facebook builds a support base of teachers that other
teachers can tap into if they are thinking of applying.

Making year after year opportunities available will promote institutional interest. Like
the undergraduate program, institutions like programs they can do year after year. Maybe
different opportunities at each grade level could be identified, leading to a team doing
RGO. For example, students could do sounding rockets, or balloon sat, or DIME
Dropping in a Microgravity Environment. Or different levels of the program could be
offered. The lead teacher might participate over and over again but bring in new teachers
for the team, and new students. The lead provides the accumulated knowledge that
supports the team so more teachers and students are involved who might not attempt it on
their own.

                                             21
C. Application process

The application for TFS NSTA RGO was adapted to provide more scaffolding for
teachers and students than the undergraduate program offers. One of the objectives for
the undergraduates is to learn how to develop a full technical proposal. In the SEED
program, they write an essay that shows their knowledge of systems engineering since the
issue is defined by the NASA principal investigator (PI). Some of the questions we asked
about the application were:
• How can the application process be used to get the right people to apply? With
    adequate rigor and follow through? In others words, how can the application process
    be rigorous enough to require an investment by the applicant, and yet be doable by
    the teachers who are most likely to benefit from it?
• Would a more rigorous application be a deterrent? (TFS NSTA application is
    scaffolded)
• Should previously conducted successful experiments or apparatus be offered to
    teams?

In this section we review the application forms and processes for TFS NSTA, NES,
Undergraduates and SEED and consider them in terms of the goals and objectives of each
program.

TFS NSTA Application
The application was part of the announcement sent out by NSTA so teachers received it
as part of an email. The application form was not online on a website or in forms based
tool. The application asked for the following information:
    • Teacher team information
    • The student experiment
    • Student hypotheses and expected outcomes of the investigation in microgravity
    • Correlation to classroom
    • Community and media outreach effort
    • Community and family involvement
    • Professional development/research dissemination
    • Letter of Commitment from the School Administrator
    • Supplemental information - optional one-two minute video

For the full announcement and application see the workspace:
http://nasatfs.pbworks.com/RGO-Application

This version was developed to obtain the information RGO needs to evaluate the merit,
safety and technical requirements of each proposal. They expect to see procedures and
“down in the weeds” detail to evaluate its feasibility and merit.

The application also needs to demonstrate that the team understands the concept of
microgravity, the test conditions on board, and experimental research. In the webinars
and in interviews, the RGO staff emphasized that the investigations need to be
experimental with controls and variables that require data to be collected in micro and

                                            22
hyper gravity to test a hypothesis about how something works, under what conditions and
in what ways.

        The experimental method is a systematic and scientific approach to research in
        which the researcher manipulates one or more variables, and controls and
        measures any change in other variables. http://www.experiment-
        resources.com/experimental-research.html

Observational experiments are not appropriate; ones that ask the question, “What will
happen?” do not meet the goals of the program. This requirement for using the
experimental method is to ensure that the teams understand gravity and its affects, and
are investigating a question that can be answered in the Zero-G plane flight protocol. It
also supports the goal of students developing an understanding of how to do experimental
research.

Sections 7 & 8 address the experiment:
Part 7: The Student Experiment.
     • What is the title of your investigation?
     • How did you and your students come up with this investigation?
     • What scientific concepts are being tested?
     • How is gravity as a variable relevant to this investigation?
     • List the tasks/phases of the project to be completed by students (examples – investigation build,
         scientific research, planning trip to Houston, post-flight analyze, publish final report).
     • Describe your testing procedure. What steps will you perform when the plane is in free fall?
         What steps will you perform when the plane is in hyper-gravity?
     • What do you expect to happen?
     • Is this investigation a free-float?
     • Has your school or school district flown this investigation in a prior flight on a NASA
         Microgravity Aircraft?

Part 8: Student Hypotheses and Expected Outcomes of the Investigation in Microgravity.
          Student Hypothesis/Outcome 1:
          Student Hypothesis/Outcome 2:
          Student Hypothesis/Outcome 3:

Teachers from the TFS NSTA group gave mixed reviews of the application. The biggest
concern was the time of year and short notice.

        Great, but short notice this year (heard about it on spring break, busy time of
        year)

        Easy to follow - good communication

        Long but important

        I think the application process was fine

        The process was much clearer and simpler when I did the program in 2007

                                                    23
Concerns were about the redundancy of the some of the information required and the
length.

       Some of the application process was unclear and repetitive

       Too long and repetitive -very wordy. We need to read a sentence in the 15
       seconds we have to check email and understand what we need to do. Use simple
       language.

       Make it clear and don't ask the same questions over and over. Have some of us
       alumni look it over and offer suggestions. Perhaps we could be included in the
       review process.

       Make the application simple, look for teachers who reach a wider audience, and
       get short administrator references

The teachers’ comments about redundancy may indicate a lack of understanding of the
experimental method and what is required in each section. Since one of the goals of the
program is to have teachers participate in a research experience to develop their
understanding of it and incorporate into it their teaching, it may be necessary to better
define what is required in each section, give detailed examples of successful applications,
offer proposal preparation sessions, and provide for questions in the application period
that are open to everyone. The National Science Foundation (NSF) uses this process for
their grant applications to ensure applicants have an adequate understanding of the goals
of the grant and the proposal format as well as cut down on the proposals that do not
conceptually fit the grant, or do not provide the information needed to evaluate a
proposal.

NSTA Past Presidents who reviewed the applications this year for TFS RGO were asked
about the applications. Their comments about the applications focus on the need for
clearly thinking through the experiment:

       The application was sufficient. The applicants' ability to filling out the application
       with major details was the major obstacle or their lack of imagination for having
       a really good project.

       My overall impression was that the people involved were very much committed to
       the project and tried to write good applications. However, some were able to
       articulate their project better than others.

       Teams need to think through the whole plan and how it would react in 0g. They
       need to get info earlier. This year, the timeline was too short. Teachers and
       students need to get consultation ahead of time (engineering, practicality, safety,
       solid physics, pre-engineering) and use local resources to actually put the
       experiments together. It takes time. A lot of the applications seemed rushed.

To get further insight into the application process we asked teachers if a more rigorous
application would be a deterrent. About 40% said no, another 40% said yes, and about
                                            24
20% maybe. The teachers who felt a rigorous application was a good thing said it would
ensure you get people who are committed to dong all the work.

       It will ensure enthusiasm about the program.

       Make proposals mini TEDPs so you have a start on that (Technical Experiment
       Data Package that the teachers and mentor complete before flying)

       It needs to be difficult because of the nature of the program but not so hard that
       would deter teachers because we are so busy. It needs to demonstrate proficiency
       in technology and the content.

Teachers who said the current application was fine and should not be more rigorous
thought this provided enough information to understand the experiment and the
hypotheses.

       It was doable but sufficient to get the commitment

The teachers who said “maybe” pointed out that the time it takes to complete a rigorous
application would be hard to justify if they are unlikely to get accepted. It was suggested
that a pre-proposal be submitted with feedback on whether they should pursue a full
proposal. This is similar to what many grants do with a letter of intent, followed by a pre-
proposal which is reviewed and either recommended for full proposal submission,
recommended with reservations or not recommended.

       It depends. It is hard to invest that much time in an application for something
       that may not be accepted. Perhaps there should be a pre-application, then a
       formal application.

Many of the teachers we interviewed were eager for feedback on the quality of their ideas
and wanted more time in the development process.

       Make a first announcement in fall. Have an early deadline for proposals with
       lowest level of detail. Increase level of detail with subsequent deadlines and give
       feedback.

       Give more time from the announcement to the application deadline – it helps to
       involve the students more.

       Make it more about the students; get them more involved in the application
       process.

To improve the process, teachers suggested that electronic applications, directions to
involve the whole team, not just the team lead, streamlining the forms, processes,
providing examples of prior successful applications and providing a timeline.

       Make the application to be done by more than one principal applicant.

                                            25
Streamline the barrage of emails early on in the application process.

          Put out a checklist of deadlines for the whole process so we know how to plan and
          can write to that in the application.

          We need more information on what we need to do, the initial presentation was not
          enough. (There was a March 4th webinar on what to expect for people interested
          in proposing)

          Connect previous RGO schools with those involved currently to act as mentors or
          guides early in the process.

          Find way to take the edge off for students and teachers to complete an experiment
          proposal - website with intro video designed to encourage proposals; examples of
          proposals - builds - actual experiment

TFS RFO NSTA Past President reviewers had some suggestions for the application
process.
       There may be a lead role for someone who teaches physics. Make it clear that
       there are substantive roles for other people so it is not limited to upper level, or
       AP students. Encourage applicants to looking for collaborators who have an
       interesting role and can fit it into their classrooms.

          It would be an obvious advantage to have more applications in relationship to the
          quality of the proposals accepted. Also, the increased number of applicants
          would help justify continued funding of the program. An increased number of
          applicants also result in more students and teachers collaborating and thinking
          "out of the box."

NES provides teachers and students with links to abstracts for prior experiments and to
resources to come up with ideas for their experiments5:
Teams may find researching topics below useful in deciding on a experiment area to study in Microgravity:
Physics, Physical Science, Chemistry, and Models (Designs). This link to Prior Campaigns may also help
show what kinds of experiments have been performed in the previous years.

Educator Guides to Microgravity may also serve as a guide to choosing an experiment:
• Microgravity: A Teacher’s Guide With Activities in Science, Mathematics, and Technology
• International Toys in Space: Science on the Station
• NASA – Microgravity Resources from CORE
• PSA (Personal Satellite Assistant) (Grades 5-8)

Teams are not limited to these areas but are encouraged to look for experiments that can give measurable,
repeatable data collection. Emphasize science in the proposal – there should be some theory, postulate, data
or calculation that leads you to believe that the phenomena you are investigating will react differently in
microgravity than in your 1-G lab. Simply proposing to do something “to see what happens” will probably
weaken your proposal’s technical merit. A well-stated hypothesis with underlying rationale on why you
believe something will happen will probably strengthen your proposal’s technical merit.

5
    http://microgravityuniversity.jsc.nasa.gov/nes/theApplication/advice.cfm

                                                    26
You can also read
Next slide ... Cancel