Putting the Teaching Quality Framework Initiative - no animation for full ...

 
Putting the Teaching Quality Framework Initiative - no animation for full ...
Putting the Teaching Quality Framework Initiative
                   Into Action:
A case-study comparison of three departments engaged
          in transforming teaching evaluation

   Sarah E. Andrews, Jessica Keating, Joel Corbo, Mark Gammon,
               Daniel Reinholz, and Noah Finkelstein

               ASCN Transforming Institutions Conference
                                      April 4, 2019
                           sarah.andrews-1@colorado.edu
                                  @sarahfoofoo79
                               #ASCNHigherEdTI2019
                                               Visit us at:
                  https://www.colorado.edu/teaching-quality-framework/ (TQF)
                    http://teval.net/ (multi-institutional umbrella project TEval)
Putting the Teaching Quality Framework Initiative - no animation for full ...
Why change current teaching evaluation practices?
         Take 45 seconds to talk to your neighbor:

          What is one reason to change practices?
                             OR
 What is one thing you would change about current teaching
                    evaluation practices?
Putting the Teaching Quality Framework Initiative - no animation for full ...
Why change current teaching evaluation practices?

  • Scholarly approach            • Multiple voices (from students,
  • Improve reputation              from peers, from self-reflection)
  • Help faculty navigate RT&P    • Add qual/de-emphasize quant
    processes                     • Evidence-based, reliable
  • Help faculty become better      measures
    teachers/support faculty in   • De-emphasize SETs
    their growth                  • Eliminate or reduce bias
  • Improve student outcomes      • Consistency/ability to compare
  • Align values to evaluation      growth over time
Putting the Teaching Quality Framework Initiative - no animation for full ...
Putting the Teaching Quality Framework Initiative - no animation for full ...
Putting the Teaching Quality Framework Initiative - no animation for full ...
The Teaching Quality
    Framework:
A model of improved
teaching assessment

     Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Putting the Teaching Quality Framework Initiative - no animation for full ...
The Teaching Quality
    Framework:
A model of improved
teaching assessment

                                                                                 Improved Evaluation System
     Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Putting the Teaching Quality Framework Initiative - no animation for full ...
A case study example: The Juniper* Dept

• Dept readiness

• Flexible process

• Early successes

• Iterative process
                                                                      DAT = Departmental Action Team (Corbo et al., 2015)

* All dept names are fictitious
         Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Putting the Teaching Quality Framework Initiative - no animation for full ...
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Putting the Teaching Quality Framework Initiative - no animation for full ...
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
The Teaching Quality
    Framework:
A model of improved
teaching assessment

     Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Brainstorming Activity:                                                           Card Sorting Activity:
             Ideal Teacher                                                 Literature-based components of effective teaching

Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Peer Observation Protocol
                        Peer (Adapted
                               Coursefrom    the UTeach
                                          Evaluation     Observation Protocol, UTOP 1)
                                                      Plan
                                                                Peer Evaluation of Courses/Teaching
You have been asked to Overview/purpose
                         complete a peer observation forPurpose  a [DEPT] faculty member for inclusion in
                        The [x] Department has collectively committed to developing best practices in teaching. The process
their reappointment, comprehensive       review, promotion, This or tenure    case.provides
                                                                                     Please    see  theteaching
                                                                                                         [DEPT]     Peer aboutand
                                                                                                                                             Guideoutlined
                                                                                                                                                       for writinghere will peer classroom observation letters
                        utilize the voices of faculty peers towards   document
                                                                          a) improving          background
                                                                                          individual           information
                                                                                                                   development,        peerb) classroom    observation
                                                                                                                                              for evaluative     reporting,   and proposes a system and peer
Course Evaluation Plan [insert
                        such aslink    when finalized]Promotion,
                                   for Reappointment,           evaluation
                                                           for full           form whereby
                                                                     instructions
                                                                        and Tenure    on        faculty
                                                                                          the peer
                                                                                       Review.           can be evaluated
                                                                                                 It isevaluation
                                                                                                       designed     process.
                                                                                                                   with  severalongoals:
                                                                                                                                    their courses    and[DEPT]
                                                                                                                                            a) to provide teaching.      CLASSROOM INTERVIEW GUIDE
                                                                                                                                                            greater A copy of this document should be
                                                                attached to peer
                        consistency in the scheduling of observations          and inobservation
                                                                                        evaluating protocols
                                                                                                      componentsWhen  ofwriting
                                                                                                                  submitted     for your
                                                                                                                         teaching     that peer   classroom
                                                                                                                                     reappointment,    promotion,
                                                                                                                                            the department        observation
                                                                                                                                                                values    and      letter,
                                                                                                                                                                           as tenure       please address the following items1 (if they areMERIT Self-reflective teaching statement
                                                                                                                                                                                       review.
                        effective practices; b) to be formative and developmental for faculty at allrelevant).                To provide
                                                                                                                  ranks in improving          feedback
                                                                                                                                          teaching   overClassroom
                                                                                                                                                          on    thisand
                                                                                                                                                            time;           c) Interview
                                                                                                                                                                       guide,                Process
                                                                                                                                                                                 please respond                    Guide
                                                                                                                                                                                                    to the questions    onfor
                                                                                                                                                                                                                           thereflecting
                                                                                                                                                                                                                                2nd page andon your
                                                                                                                                                                                                                                                 returnteaching in preparation for writing your Faculty Statement on
  ❏ Before printing thistoform,  insert
                            foster        the instructor’s
                                    a departmental            ofBackground
                                                     culturelearning    goals
                                                                 scholarly      (e.g., Student
                                                                             teaching    and sharedLearning
                                                                                                        visions.Objectives
                                                                                                                  To  achieve these goals, the process                                                                                              Reflect on your Teaching
                                                                                                                                                                                                                                                                          teaching during the past calendar year. The PEC will use your statement as part of the
                                                                Unstructured      peerincluding
                                                                                        classroom               to
                                                                                                       observation,[X].
                                                                                                                      i.e., those   that are
                        implements
     (SLOs)) for this course  directlystrategies   that are
                                         into the space     backed
                                                          provided    by  research,
                                                                      in item    A1)   Learning    employing
                                                                                                     Goals.      a standard    protocol    fornot  based on
                                                                                                                                               classroom         a set of core criteria, can result in
                                                                                                                                                             observations
                                                                                                                                                         Option        A)department
                                                                                                                                                                            Print out (AAAS,
                                                                                                                                                                                         the classroom       interview    form below and have       evaluation      process.
                                                                                                                                                                                                                                                         each student          Your reflection
                                                                                                                                                                                                                                                                            complete           should address some or all of the following guiding questions. You
                                                                                                                                                                                                                                                                                        the form
                                                                inconsistency     and   do  not  always   address    teaching    practices
                        and incorporating those observations within a broader process of consultation and conversation (e.g., Brinko,         that are valued      by
                                                                                                                                                                  1993 a                          2012).
                                                                                                                                                                         ; of the class period observed  For  this                                  do not have to answer all of the questions.
  ❏ Use the “Evidence/Notes”      boxes 1 to make notes regarding
                                                                reason,  each
                                                                          the    question
                                                                               scholarly     during
                                                                                           literature  the
                                                                                                        on  observation.
                                                                                                           best  practices    in teaching    recommends  at thethat   end
                                                                                                                                                                      academic     units  spell out the best          (5-10
                                                                                                                                                                                                                   When
                                                                                                                                                                                                               teaching     minutes).
                                                                                                                                                                                                                          preparing       Collect
                                                                                                                                                                                                                                       to write yourallFaculty
                                                                                                                                                                                                                                                          responses       and summarize
                                                                                                                                                                                                                                                                   Statement     on Teachingthem
                                                                                                                                                                                                                                                                                              for reappointment, promotion,
                        AAAS, 2012) .                                                                           Review of syllabus and other course materials:
     Please check NA if a particular item did not apply topractices
                                                                  the classforyou      field and define core criteria to use in the observationinprocess
                                                                                 theirobserved.                                                               the classroom
                                                                                                                                                                         (AAAS, 2012,observation       protocol
                                                                                                                                                                                          Cornell). Feedback        in the
                                                                                                                                                                                                                 onand     space
                                                                                                                                                                                                                        tenure,  weprovided
                                                                                                                                                                                                                                     recommend   [insert
                                                                                                                                                                                                                                                    Please  link].
                                                                                                                                                                                                                                                     reflectinglimiton  theresponse
                                                                                                                                                                                                                                                                      your  following to
                                                                                                                                                                                                                                                                                       components
                                                                                                                                                                                                                                                                                         no more thanof effective teachingpages.
                                                                                                                                                                                                                                                                                                        2 single spaced    and
                                                                teaching can be more effective in promoting          ● growth
                                                                                                                          Did the  andsyllabus   clearlywhen
                                                                                                                                        improvement        describe        expectations,
                                                                                                                                                                   it focuses    on specificrequirements,
                                                                                                                                                                                              issues, containsand assessments for the course?
                        Evaluation types                                                                                                                                                                           incorporating them into your statement. While these components are based on foundational
  ❏ As soon as possible after the observation, review your           notesinformation,
                                                                concrete     and write aand   quick
                                                                                                 is   summary/key
                                                                                                     based   on  specific   data  (rather  than  general   impressions)        (Brinko, 1993).   To that
                                                                                                                                                         Option B) Dedicate 15-25 minutes of classroom   end,   a group
                        There are two types of evaluation: a full peer course evaluation consisting of a pre-observation, classroom                                                                                scholarshiptime
                                                                                                                                                                                                                                1
                                                                                                                                                                                                                                 , thisto a more
                                                                                                                                                                                                                                        guide  is in1)focus
                                                                                                                                                                                                                                                      the
                                                                                                                                                                                                                                                        How    group
                                                                                                                                                                                                                                                            early
                                                                                                                                                                                                                                                                didstages
                                                                                                                                                                                                                                                                     yourstyle interview:
                                                                                                                                                                                                                                                                            of development
                                                                                                                                                                                                                                                                           courses           andcomment
                                                                                                                                                                                                                                                                                    go? Please     we welcome    your feedback!
                                                                                                                                                                                                                                                                                                            on achievement   of course goals, level of student
     takeaways in the space   provided.                         of five instructors and faculty from [DEPT], partnering
                        observation(s), a post-observation discussion, and a formal report (see 3Technical       in Procedures
                                                                                                                                     with facilitators from
                                                                                                                               aspects/classroom                the NSF-funded Teaching Quality Framework
                                                                                                                                                         mechanics:
                                                                                                                                    below); and an abbreviated        a) Please ensure that the faculty member being observed                           leaves the student
                                                                                                                                                                                                                                                                        room before
                                                                (TQF) Initiative, have developed new standardized protocols for peer classroom observation that align with the TQF                                                                  engagement,                  learning outcomes. You may want to address aspects of your courses and/or
  ❏ There is additional space   at theconsisting
                        observation     end of thisofform   if you
                                                      a single       run out
                                                                classroom
                                                                Initiative and ofare
                                                                                   space
                                                                              observation
                                                                                      basedwithin
                                                                                              and
                                                                                               on an aestablished    ●measure,
                                                                                                        box.report (see
                                                                                                   formal                 Resources.
                                                                                                                          4 in Procedures  Resources
                                                                                                                                  the UTeach    below).  selected
                                                                                                                                                  Observation           forbeginning.
                                                                                                                                                                    Protocol the(UTOP),
                                                                                                                                                                                  class (e.g.,  boardatwork,
                                                                                                                                                                                          developed      the slides, handouts, etc.) were
                                                                                                                                                                                                                      Goals for student learningteaching that proved to be particularly effective and/or ineffective, as well as how you assessed
  ❏ Please retain the completed protocol for your records.
                                                       University of Texas at Austin.                                      educationally valuable, well executed, and beneficial for the students.● What knowledge, skills,                                             and attitudes
                                                                                                                                                                                                                                                                  efficacy.   You may are focusimportant
                                                                                                                                                                                                                                                                                                   on one for     student
                                                                                                                                                                                                                                                                                                              course        success
                                                                                                                                                                                                                                                                                                                       or several       in your
                                                                                                                                                                                                                                                                                                                                     courses.
                          Selection of observers                                                                                                                         b) Briefly explain the purpose of this process.
                                                                                                                       ● Organized. The instructor’s activities were well organized, structured, and                                         made good use of
                                                                                                                                                                                                                                        discipline?
                          The department chair, director of undergraduate studies, and chair of the mentoringofcommittee
                                                             How     the   protocols      will  be  used    for  evaluation        teaching in    will[DEPT]
                                                                                                                                                        meet twice annually
                                                                                                                           time.                                         c) Form         students into small groups, depending on class2)size.
 Instructor               (August and December) to determine Thesewhostandardized
                                                                          will be observedtools,inwhich    replace the
                                                                                                   the upcoming           unstructured
                                                                                                                       semester    based letters
                                                                                                                                             upon the   previously
                                                                                                                                                            frequency     detailedfor Reappointment, Promotion, ● Are course goals appropriately
                                                                                                                                                                      submitted                                                                                                  challenging
                                                                                                                                                                                                                                                                      What (if any)   changesfor   didthe
                                                                                                                                                                                                                                                                                                        you level of the course?
                                                                                                                                                                                                                                                                                                              introduce    in your classes and why?
                                                             and Tenure Review, contribute to the consistency          ● Accuracy.and rigorInstructor’s
                                                                                                                                              of our evaluation writtenof and      spoken
                                                                                                                                                                            teaching,     ourcontent
                                                                                                                                                                                                 comparisoninformation
                                                                                                                                                                                                                  of teachingwas accurate
                                                                                                                                                                                                                                   ● Whatand   areappropriately
                                                                                                                                                                                                                                                     you preparing students for?
                          below, and prioritizing those due for Reappointment, Promotion, and Tenure Review. The committee will also appoint                             d)     Pose     your       chosen      questions       to the   class.
                                                             practices      over  time,   and   our  development       of a  shared
                                                                                                                           thorough    vision   for  teaching
                                                                                                                                          (e.g.toinformation       practice
                                                                                                                                                                     written    in [DEPT].     In  addition,   it is  our hope
 Course Name              faculty to conduct evaluations andthat
                                                               to select    the appropriate
                                                                    the structured      form will evaluation    type (faculty
                                                                                                    better enable                members
                                                                                                                       more formative               be observed
                                                                                                                                            assessment                will be on board,
                                                                                                                                                               and self-reflection
                                                                                                                                                                                                  in hand-outs).
                                                                                                                                                                                         on teaching practices.
                                                                                                                                                                                                                                   ● What are key challenges in 2a)         the What
                                                                                                                                                                                                                                                                                 teaching-learning
                                                                                                                                                                                                                                                                                        adjustments process?
                                                                                                                                                                                                                                                                                                          did you make in response to prior course feedback (FCQs, peer
 Course Number/Section    consulted as part of this process). Generally, some observers will be assigned               ● from
                                                                                                                           Active     Learning
                                                                                                                                 within   the faculty   (ifmember’s
                                                                                                                                                             applicable).e) Have
                                                                                                                                                                        field    The
                                                                                                                                                                                 of instructor
                                                                                                                                                                                         each group    employed
                                                                                                                                                                                                              discuss  active
                                                                                                                                                                                                                            andlearning
                                                                                                                                                                                                                                  come to  strategies   appropriate
                                                                                                                                                                                                                                              a consensus        on eachobservation,
                                                                                                                                                                                                                                                                              question. FTEP observation, Qualtrics, surveys)?
                          study and some will be chosen from Foroutside
                                                                   reappointment,        promotion,
                                                                             fields. However,            and to
                                                                                                   in order    tenure   review,
                                                                                                                  balance  for    the size
                                                                                                                                 the
                                                                                                                             workload, instructor     under
                                                                                                                                             and structure
                                                                                                                                           schedules,          review
                                                                                                                                                             etc., of   submits
                                                                                                                                                                   thethe    class, all
                                                                                                                                                                                      andclassroom
                                                                                                                                                                                             in line with observation
                                                                                                                                                                                                               departmental       expectations.
                                                                                                                                                                                                                               Preparation     for teaching
 Date/Time/Room/Bldgdepartment chair will have final sayprotocols           completed
                                                                in the selection           by their peers to the PUEC. The instructor will also have the
                                                                                      of observers.                                                                      f) opportunity
                                                                                                                                                                                Have eachtogroup              share
                                                                                                                                                                                                     reflect on    theseout their responses to the whole class. Write these responses
                                                                                                                                                                                                                                   ● What preparation have you 2b)           done  to acquire
                                                                                                                                                                                                                                                                                What    steps havedeepyou knowledge
                                                                                                                                                                                                                                                                                                              taken to of    the subject or develop
                                                                                                                                                                                                                                                                                                                         enhance/further       build your knowledge about effective
                                                             protocols when writing their teaching statements for reappointment, promotion, and                                     tenureeveryone
                                                                                                                                                                                where        review, andcan   in writing   their whiteboard/chalkboard, PowerPoint slide).
                                                                                                                                                                                                                    see (e.g.,
 # of students enrolled/# of                                 self-reflection for annual merit review. What the instructor was doing/efforts made to engage students:                                                                    topics into your courses? teaching practices, methods, and/or materials? You may wish to consider factors such as
                          Procedure                                                                                                                                                                                                ● on How    do  you   know    whataskpractices   are   evidence     based    and what      are not? literature; attending FTEP, ASSETT, or COLTT
 students who attended                                                                                                 ● Content. The instructor choseg)examples                If time and youdetails
                                                                                                                                                                                                    may leadthat were      appropriate
                                                                                                                                                                                                                    a discussion           and worthwhile
                                                                                                                                                                                                                                            these   items      for
                                                                                                                                                                                                                                                             and/or         reading
                                                                                                                                                                                                                                                                             students in scholarship
                                                                                                                                                                                                                                                                                          to vote on       of teaching
                                                                                                                                                                                                                                                                                                           most          and    learning
                              1. All first-time observers, or observers who are new to this process, shall meet with the department chair to review
                                                                                                                           helpingprotocol
                                                                                                                                        students      learn         contentimportant              takeaways.                       ● In what ways do you incorporate               evidence-based
                                                                                                                                                                                                                                                                            sessions;                      teaching practices
                                                                                                                                                                                                                                                                                         engaging in discussion                     into youretc.
                                                                                                                                                                                                                                                                                                                        with colleagues;        courses?
                                                                                                                                                                                                                                                                                                                                                    How have these opportunities initiated
 Observer                                                    Table
                                   and become familiar with this      1. Alignment of [DEPT] Peer Classroom Observation
                                                                   process.                                                                      items   withthe
                                                                                                                                                               components        in effective
                                                                                                                                                                                 of  this course.
                                                                                                                                                                                                classroom    teaching from the
                                                             TQF rubric.                                               ● Motivation. The instructor provided context and made clear attempts●to How                                      pointdo
                                                                                                                                                                                                                                               outyouthe identify
                                                                                                                                                                                                                                                          relevancestudent    challenges?
                                                                                                                                                                                                                                                                      of reflection      about your teaching and changes in your teaching?
                                                                                                                                                                         h) Record            responses/key             takeaways      in
                                                                                                                                                                                                                                   ●realHowthedoclassroom
                                                                                                                                                                                                                                                   you work observation
                                                                                                                                                                                                                                                                to overcome       protocol
                                                                                                                                                                                                                                                                                 them         in the space
                                                                                                                                                                                                                                                                                         or address      them when they arise?
                              2. Once you have been assigned        the role of
                                                                 Component      aseffective
                                                                                    an observer,     it isteaching
                                                                                              classroom                    the material,
                                                                                                           your responsibility      to initiatee.g.,
                                                                                                                                                  the by     connecting
                                                                                                                                                        following    process. it to other      subjects,items
                                                                                                                                                                                        Corresponding       giving      examples,
                                                                                                                                                                                                                   in the Peer             world  applications,     etc.
                                                                                                                                                                                providedObservation
                                                                                                                                                                                                 [insert link].
                                                                                                                       ● Depth. The instructor delivered content and answered questions in a ●wayWhat                                          changes      have with
                                                                                                                                                                                                                                                                   you made 2c) to  yourmade
                                                                                                                                                                                                                                                                                           syllabus    and/or   course
                                                                                                                                                                                                                                                                                                                    were materials     that
                                                                                                                                                                                                                                                                                                                                          to reflect a what changes did you introduce in
     a) Did the syllabus include the Required Syllabus Statements (i.e. Disability                                                                                                                           Protocols                                                          If you           changes      that         not related       2a or 2b,
                                                                                                                                                                                                                                          that was    consistent
     Accommodation; Religious 3. Holidays;    Classroom
                                   Full peer course       Behavior;
                                                     evaluation         Sexual
                                                                  process Content,Misconduct,                  Yes         deep knowledge of the Classroom                                                                              commitment to careful planning            and meaningful
                                                                                                                                                                                                                                                                            your classes     and why?   assignments?
                                                                 Goals,             and Alignment                                                               subject.             Interview
                                                                                                                                                                                  A3) Learning Goals   Questions
     Discrimination, Harassment, and/or      Retaliation; and Honor
                                       a. Pre-observation        What are Code),
                                                                            studentsper  CU to learn from
                                                                                      expected            No the courses   taught? Are course goals appropriately                                                                  ● What efforts        have   you made      to design    your courses,       syllabi, and/or materials that show
                                                                 challenging? Is content aligned with the curriculum   ●andReasoning.         The instructor
                                                                                                                             integrate other topics            Select      2 orA6)
                                                                                                                                                                     highlighted
                                                                                                                                                       and/or courses?              3 Prior
                                                                                                                                                                                      ofthe
                                                                                                                                                                                          the     following
                                                                                                                                                                                            Knowledge
                                                                                                                                                                                              ideas    behind questions
                                                                                                                                                                                                                  the content to  andinclude    in your
                                                                                                                                                                                                                                       encouraged          classroom
                                                                                                                                                                                                                                                       students   to      interview     (ideally   make this
     Boulder policy?                                                                                                                                                                                                                    a commitment
                                              i.    Request and review the following items prior to the in person consultation:
                                                                                                                           increase      their   reasoning     decision
                                                                                                                                                                 ability.      in   consultation with the observed instructor).                        Youto   meaningful
                                                                                                                                                                                                                                                             may  3)tailor
                                                                                                                                                                                                                                                                      Whatany  assignments
                                                                                                                                                                                                                                                                             steps   have you
                                                                                                                                                                                                                                                                                 question       (e.g.,
                                                                                                                                                                                                                                                                                                  taken that
                                                                                                                                                                                                                                                                                               specificallyto connect
                                                                                                                                                                                                                                                                                                               evaluate
                                                                                                                                                                                                                                                                                                                to       to
                                                                                                                                                                                                                                                                                                                          thereal  worldofapplications,
                                                                                                                                                                                                                                                                                                                               quality      student understanding in a formative as well as
                                                        1. Syllabus,     including
                                                                 Preparation          course learning goals (e.g., departmental Student Learning
                                                                                for Teaching                                                                                      A2) Tools and Resources                               prior knowledge, and/or         future course
                                                                                                                                                                                                                                                                  summative      fashion? applications)?
                                                                                                                                                                                                                                                                                             Formative assessments are low-stakes opportunities to monitor student learning
     b) Did the syllabus clearly describe expectations and    requirements        for
                                                                 Did the instructor    the   course,
                                                                                     demonstrate               Yes
                                                                                                   sufficient content, ●   Informal
                                                                                                                       background,         Assessment.
                                                                                                                                     and pedagogical           the
                                                                                                                                                                The
                                                                                                                                                         knowledge?  instructor’s
                                                                                                                                                                      instructor
                                                                                                                                                                       Was                or Learning
                                                                                                                                                                                       took
                                                                                                                                                                                  A5) Active    to   your interest.
                                                                                                                                                                                                advantage      of   opportunities     to gauge   student
                                                            Objectives     (SLOs)) (required)
                                                                 the instructor well-prepared in terms of classroom mechanics (e.g. grading, prepping activities,                 A7) Depth (Content Courses)                      ● How do you prepareand          for are
                                                                                                                                                                                                                                                                        all aspects
                                                                                                                                                                                                                                                                             typicallyofnot
                                                                                                                                                                                                                                                                                          classroom
                                                                                                                                                                                                                                                                                             graded or    mechanics     (e.g.,points
                                                                                                                                                                                                                                                                                                             only assigned       grading,   use on
                                                                                                                                                                                                                                                                                                                                        based    of completion. Summative assessments are
     including departmental Student Learning Objectives        (SLOs),     iftech
                                                                               applicable?                No class handouts/exams,
                                                                                                                           understanding           (e.g.,tobytheasking
                                                        2. Optional     materials
                                                                 materials,          you
                                                                                  use,     may consider:
                                                                                       etc.)?                                                 access               course1)them What
                                                                                                                                                                                  A9)questions).
                                                                                                                                                                                         aspects
                                                                                                                                                                                      Accuracy           of [insert
                                                                                                                                                                                                   (Language  Courses)faculty member’s name] teaching were most effective in helping
                                                                                                                                                                                                                               technology, prepping activities, lectures,
                                                                                                                                                                                                                                                  high-stakes    measuresdemonstrations,
                                                                                                                                                                                                                                                                          used to evaluateetc.)?
                                                                                                                                                                                                                                                                                            student learning and are therefore graded for
                                                                         management system, prior FCQs, etc.                                                                            you    learn?
      *If no in (a) or (b), what was missing/unclear?                                                                                                                                     A4) Participation                                                     correctness/quality (e.g., quizzes, exams, projects).
                                                         ii.     Set up an in Methods
                                                                               person consultation
                                                                                         and Teaching Practices                What the students were doing/student engagement:           A5) Active Learning
                                                                              What assignments, assessments, and activities are implemented? Are methods appropriate for and                                                     Methods and teaching practices
                                                                     1. This aligned
                                                                               meeting     should    occur   before    any   classroom     observation,
                                                                                       with the learning environment, the student●population
                                                                                                                                          Engagement.        early  in
                                                                                                                                                 (inclusive ed,Students
                                                                                                                                                                        the         2)
                                                                                                                                                                             semester
                                                                                                                                                                course level)appeared
                                                                                                                                                                              and
                                                                                                                                                                                        What      aspects
                                                                                                                                                                                          A8) Critical
                                                                                                                                                                                        toA8)
                                                                                                                                                                                           beCorrectiveThinkingof
                                                                                                                                                                                                on taskFeedback   [insert
                                                                                                                                                                                                               (Content
                                                                                                                                                                                                           throughout       faculty
                                                                                                                                                                                                                        Courses)     member’s     name]  teaching     were least effective in helping
                                                                                                                                                                                                                         the class and engaged in learning. 4) Describe
                                                                                                                                                                                                                    (Language Courses)● What teaching methods         do you the
                                                                                                                                                                                                                                                                             use? teaching accomplishment(s) from the past year that you value the most.
                                                                     2. Purpose:     scheduling
                                                                              departmental,   course,the
                                                                                                       andclass
                                                                                                           studentvisit(s),
                                                                                                                    goals? understanding the goals of the course and/or the             you    learn? strengths/expertise
                                                                                                                                     ● Participation. The instructor established                    an environment that gave all
                                                                                                                                                                                          C1) Instructor                                students the opportunity   to
                                                                                                                                                                                                                                       ● How do these methods contribute to your goals for students?
                                                                         class session(s) you’ll be observing, and explaining/asking questions about course
                                                                              Presentation and Student Interaction
                                                                         materials.                                                       participate fully, including encouraging  3) How        their participation
                                                                                                                                                                                                would      you describe   in class.
                                                                                                                                                                                                                              [insert●faculty
                                                                                                                                                                                                                                          Why member’s     name]
                                                                                                                                                                                                                                               are these methods     level of interest
                                                                                                                                                                                                                                                                      appropriate
                                                                                                                                                                                                                                                                5) What  other          in contributions,
                                                                                                                                                                                                                                                                                   for use
                                                                                                                                                                                                                                                                                teaching   helping
                                                                                                                                                                                                                                                                                            in your discipline? For theorclassroom
                                                                                                                                                                                                                                                                                                          challenges,      concerns would you like to share?
                                                                                                                                                                                          A4) Participation
                                                                              Are methods from above implemented effectively? Are students supported (e.g. student/teacher
                                                                     3. During    this consultation, it is also recommended that the observer and instructor discuss
                                                                              interaction)?
                                                                                                                                                                                        students
                                                                                                                                                                                          A7) Engagementlearn?   Explain
                                                                                                                                                                                                            (Language       and provideenvironment
                                                                                                                                                                                                                       Courses)             example(s).(e.g., lecture, lab)? For the course level?
                                                                         the possibility of supplementing the course           Additional
                                                                                                                                     evaluation considerations:
                                                                                                                                                   process with additional forms of                                                    ● What assignments, assessments, and learning activities are implemented?
                                                                         data,Student
                                                                                such as:outcomes
                                                                                             student interviews (FTEP CLIP
                                                                                                                                                                                    4) What could students do to help improve this class?
                                                                                                                                     ● service   or a modified
                                                                                                                                          Observations             version
                                                                                                                                                             of what
                                                                              What impact do these courses have on learners? What evidence shows the level of student
                                                                                                                                                                          the -instructor
                                                                                                                                                                                see        did well.
                                                                                                                                                                                          A4) Participation
                                                                         addendum),       otherAreFTEP   services,    and/or(shiftASSETT’s     VIP service.
                                                                                                                                     ● Suggestions          for  the instructor     to improve      theirFeedback
                                                                                                                                                                                                           teaching.
                                                                              understanding?        measures   of learning        in student performance   as     5) How has [insert faculty member’s name] worked to ensure members of the class were
                                                                                                                                                              a result of                 A8) Corrective            (Language Courses)
                                                                                 class/instruction) aligned with goals?
                                                                                                                                                                     engaged
                                                                                                                                    ● Comments on your interactions with         during the semester?
                                                                                                                                                                         the instructor.           1
                                 1
                                                                                                                                                                                                                       Developed from the Rubric for Statements of Teaching Philosophy developed by Matt Kaplan, Chris O’Neal, Debbie Meizlish,
                                    American  Association
                                                      Marchfor2018
                                                               the from
                                                                   Advancement      of Science. 2012. Describing & Measuring Undergraduate STEM Teaching Practices.
1. The UTeach Observation Protocol (UTOP).  Retrieved                   https://utop.uteach.utexas.edu/.
                                            http://ccliconference.org/files/2013/11/Measuring-STEM-Teaching-Practices.pdf                                               6) The instructor has identified Rosario
                                                                                                                                                                                                             a keyCarillo,  and Diana Kardia (http://www.crlt.umich.edu/sites/default/files/resource_files/TeachingPhilosophyRubric.pdf) and
                                                                                                                                                                                                                      learning    goal for this course as [insert goal that the
                                                                                                                                                                                                            from foundational scholarship, including Scholarship Reconsidered (Boyer, 1990), Scholarship Assessed (Glassick, Huber, &
                                                                                 [Insert references
                                   Brinko, K.T. 1993. The practice of giving feedback                    cited] What is effective? The Journal of1Higher Education 64(5): 574-593.
                                                                                           to improve teaching:                                                             instructor would like assessed].      How
                                                                                                                                                                                                            Maeroff,      much
                                                                                                                                                                                                                     1997),        progress
                                                                                                                                                                                                                            and work              do you
                                                                                                                                                                                                                                     at the University       feel(e.g.
                                                                                                                                                                                                                                                       of Kansas   you Theare  making
                                                                                                                                                                                                                                                                           Evaluation      towards
                                                                                                                                                                                                                                                                                      of Faculty Teaching Rubric), and with sponsorship from
                                            https://www.jstor.org/stable/2959994                                                                                                                            the National Science Foundation (DUE-1725959).
                                                                                                                                                                            thisProtocol
                                                                                                                              1. Items were adapted from the UTeach Observation   goal?(UTOP). Retrieved March 2018 from
                                                                                                                              https://utop.uteach.utexas.edu/.

            Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Teaching Quality Framework (TQF) - Aligning Data Sources to TQF Assessment Rubric

                                                                                                                                                                                                       Sources of evidence you could collect

                                                                                                                      Professional (3)                                 Voice Rank

                                                         Goals, Content, and              Goals/content are well-articulated, high quality, up-to-date, and                             Self-reflection         Classroom               Student interviews/
                                                         Alignment                        appropriate                                                                                                           observation             CLIP
                                                         What are students expected                                                                                                     Teaching statement
                                                                                          Course materials/content are aligned with course goal, include
                                                         to learn from the courses                                                                                                                              Review of syllabus/     FCQs
                                                         taught? Are course goals
                                                                                          high-quality elements
                                                                                                                                                                                        Course portfolio        course materials
                                                         appropriately challenging?       Range/depth of course topics is appropriate, integrate other                                                                                  Student letters
                                                         Is content aligned with the      topics/courses                                                                                                        Review of portfolio
                                                         curriculum?
                                                                                          Some examples of innovation, connection to current issues,
                                                                                                                                                                                                                Review of learning
                                                                                          developments in field
                                                                                                                                                                                                                objectives
                                                         Preparation for                  Knows subject on deep level, including current research, interaction                          Self-reflection         Classroom               Student interviews/
                                                         Teaching                         with other topics                                                                                                     observation             CLIP

  Mapping data
                                                         Content/ Background                                                                                                            Teaching statement
                                                         Knowledge; Pedagogical           Teaching practices/methods or materials are evidence-based, shown                                                     Review of syllabus/     FCQs
                                                         Knowledge (i.e. teaching         to enable learning                                                                            Course portfolio        course materials
                                                         generally and teaching                                                                                                                                                         Student letters
                                                         subject material);
                                                                                          Can identify most common student challenges; activities/innovations

   sources and
                                                                                          to help students overcome common challenges                                                                           Review of portfolio
                                                         Classroom mechanics
                                                         preparation (e.g. grading,
                                                         prepping activities,             Syllabus, materials, and course material are well-planned, integrated,                                                Faculty interviews
                                                         materials, tech use, etc)        and reflect commitment to meaningful assignments

voices to the TQF
                                                         Methods and Teaching             Often uses effective or innovative evidence-based* methods to                                 Self-reflection         Classroom               Student interviews/
                                                         Practices                        improve understanding *See: NAS Indicators                                                                            observation             CLIP
                                                         What assignments,                                                                                                              Teaching statement
                                                         assessments, and activities      Activities provide opportunities for practice/ feedback on important
                                                                                                                                                                                                                Review of syllabus/     FCQs
                                                         are implemented? Are             skills and concepts                                                                                                   course materials
                                                                                                                                                                                        Course portfolio

framework rubric
                                                         methods appropriate for                                                                                                                                                        Student letters
                                                         environment and aligned for      Regular opportunities to practice skills
                                                                                                                                                                                        Inventory (e.g., TPI,   Review of portfolio
                                                         student population                                                                                                                                                             Survey (e.g.,
                                                         (inclusive ed, course level)     Students consistently engaged, w/ occasional high levels of                                   TBI)
                                                                                          engagement                                                                                                            COPUS                   SALG)
                                                         and goals (departmental,
                                                         course, student)
                                                                                          Implementation is high-quality/consistent                                                                             Inventory (e.g., TPI,   Inventory (e.g., TPI,
                                                                                                                                                                                                                TBI)                    TBI)
                                                         Presentation and                 Evidence that classroom climate is respectful, cooperative, and                               Self-reflection         Classroom               Student interviews/
                                                         Student Interaction              encourages motivation and engagement                                                                                  observation             CLIP
                                                         What are the students’ views                                                                                                   Teaching Statement
                                                         of the learning experience?      Student reports of instructor accessibility and interaction skills are                                                Review of portfolio     FCQs
                                                         How has student feedback         positive                                                                                      Course Portfolio
                                                         informed the teaching? Are                                                                                                                                                     Student letters
                                                         methods (#3) implemented         Students perceive that they are learning important skills or knowledge
                                                         effectively? Are students
                                                         supported (e.g.
                                                                                                                                                                                                                                        Survey (e.g.,
                                                         student/teacher                  Instructor gathers student feedback and articulates some lessons                                                                              SALG)
                                                         interaction)?                    learned

                                                                                        For more information about the Teaching Quality Framework, please visit our website:​ ​www.colorado.edu/teaching-quality-framework

    Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Departmental examples of adapting rubric mapping to their own available data sources

                                                                                                                                                                                                 What do we know about a given instructor’s teaching and how do we know (i.e., where do we look for this information)?

                                                                                                                                                                                                                                                                              How do we know/where do we look?
                                                                                             Corresponding items from:
                                                                                                                                                                                                     What do we know about an instructor’s                                                                   RT&P
                                                                                                                                                                                                                                                                            Merit
                                                                                                                                                                                                     teaching?1                                                                                   (everything in merit plus…)
  Component of effective classroom                                                                                                                                                   Your
                                                      Self-reflection          Peer Observation                          FCQs                    Syllabi and course materials
  teaching                                                                                                                                                                         Score (1-5)       Goals, Content, and Alignment                           FCQs:2                             Peer Observations3
                                                                                                                                                                                                     What are students expected to learn from the            - S3 (intellectual challenge)      - Review of syllabus
                                                                                                                                                                                                     courses taught? Are course goals appropriately          - CP5 (helpful materials)          - A1) Resources
  Goals, Content, and Alignment               High weight               High weight                       Low weight                           High weight
                                                                                                                                                                                                     challenging? Is content aligned with the curriculum     - OP1 (connections to other        - A5) Content
  What are students expected to learn                                   A3) Learning Goals                S3 (intellectual challenge)                                                                and integrate other topics and/or courses?              classes/life)                      - A6) Motivation
  from the courses taught? Are course                                   A6) Prior Knowledge               CP5 (helpful materials)
  goals appropriately challenging? Is                                                                                                                                                                                                                        FCQs:                              Peer Observations
  content aligned with the curriculum and                                                                                                                                                                                                                    - CP4 (invested in student         - Review of syllabus
                                                                                                                                                                                                     Preparation for Teaching                                success)                           - A1) Resources
  integrate other topics and/or courses?                                                                                                                                                             Did the instructor demonstrate sufficient content,      - CP5 (helpful materials)          - A2) Organized
                                                                                                                                                                                                     background, and pedagogical knowledge? Was the                                             - A3) Accuracy
                                                                                                                                                                                                     instructor well-prepared in terms of classroom                                             - A4) Active Learning
  Methods and Teaching Practices              Equal weight              Equal weight                      Equal weight                         Equal weight
                                                                                                                                                                                                     mechanics (e.g. grading, prepping activities,                                              - A5) Content
  What assignments, assessments, and                                    A4) Participation                 S3 (intellectual challenge)                                                                materials, tech use, etc.)?                                                                - A7) Depth
  activities are implemented? Are                                       A5) Active Learning               CP7 (opportunities for                                                                                                                                                                - A8) Reasoning
  methods appropriate for and aligned                                   A8) Critical Thinking (C)         discussion)                                                                                                                                                                           - A9) Informal Assessment
  with the learning environment, the                                    A8) Corrective Feedback (L)       CP8 (helpful feedback on work)
                                                                                                                                                                                                                                                             FCQs:                              Peer Observations
  student population (inclusive ed, course                              C1) Strengths/expertise                                                                                                      Methods and Teaching Practices                          - S3 (intellectual challenge)      - A4) Active Learning
  level) and departmental, course, and                                                                                                                                                               What assignments, assessments, and activities are       - CP7 (opportunities for           - A8) Reasoning
  student goals?                                                                                                                                                                                     implemented? Are methods appropriate for and            discussion)                        - A9) Informal Assessment
                                                                                                                                                                                                     aligned with the learning environment, the student      - CP8 (helpful feedback on work)   - A10) Engagement
                                                                                                                                                                                                     population (inclusive ed, course level) and                                                - A11) Participation
                                              Low weight                Mid weight                        High weight                          Low weight                                            departmental, course, and student goals?                                                   - C1) observations of what the
  Presentation and Student Interaction                                  A4) Participation                 S4 (how much learned)                                                                                                                                                                 instructor did well

  Are methods from above implemented                                    A7) Engagement (L)                S6 (encourages interest)                                                                                                                           FCQs:                              Peer Observations
  effectively? Are students supported (e.g.                                                               S7 (instructor availability)                                                               Presentation and Student Interaction                    - S4 (how much learned)            - A6) Motivation
  student/teacher interaction)?                                                                           S8 (respect for students)                                                                  Are methods from above implemented effectively?         - S6 (encourages interest)         - A9) Informal Assessment
                                                                                                          CP3 (respectful environ.)                                                                  Are students supported (e.g. student/teacher            - S7 (instructor availability)     - A10) Engagement
                                                                                                                                                                                                     interaction)?                                           - S8 (respect for students)        - A11) Participation
                                                                                                                                                                                                                                                             - CP3 (respectful environment)
                                              High weight               High weight                       Mid weight                           Mid weight
  Student outcomes
                                                                        A4) Participation                 S3 (intellectual challenge)                                                                Student Outcomes                                        FCQs:                              Peer Observations
  What impact do these courses have on                                                                                                                                                               What impact do these courses have on learners?          - S3 (intellectual challenge)      - A9) Informal Assessment
                                                                        A8) Corrective Feedback (L)       S4 (how much learned)
  learners? What evidence shows the level                                                                                                                                                            What evidence shows the level of student                - S4 (how much learned)            - A11) Participation
                                                                                                          CP4 (invested in student                                                                   understanding? Are measures of learning (shift in       - CP4 (invested in student
  of student understanding? Are
                                                                                                          success)                                                                                   student performance as a result of class/instruction)   success)
  measures of learning (shift in student                                                                                                                                                             aligned with goals?                                     - CP8 (helpful feedback on work)
                                                                                                          CP8 (helpful feedback on work)
  performance as a result of
                                                                                                          See also: Classroom Interview
  class/instruction) aligned with goals?                                                                                                                                                             Mentorship and Advising                                 FRPA Teaching Activities:
                                                                                                          Q2                                                                                         How effectively has the faculty member worked           - Advisees/mentees
                                                                                                                                                                                                     individually with undergraduate or graduate             - Students’ supported in
                                                                                                                                                                                                     students?                                               independent work
  Reflection, Development, & Teaching         High weight               NA                                NA                                   High weight
  Service/ Scholarship                                                                                                                                                                                                                                       FRPA Service Activities:
  How has the faculty member’s teaching                                                                                                                                                                                                                      - Attendance at teaching
  changed over time? To what extent has                                                                                                                                                              Reflection, Development, & Teaching Service/
                                                                                                                                                                                                                                                             professional development
                                                                                                                                                                                                                                                             activities (Service Activities)
  the teacher reflected on and improved                                                                                                                                                              Scholarship
                                                                                                                                                                                                                                                             - Participation in
  their own teaching?                                                                                                                                                                                How has the faculty member’s teaching changed
                                                                                                                                                                                                                                                             departmental/institutional
                                                                                                                                                                                                     over time? To what extent has the teacher reflected
                                                                                                                                                                                                                                                             teaching-related committees
                                                                                                                                                                                                     on and improved their own teaching?
                                                                                                                                  Enter the average score across all components:                                                                             FRPA Scholarly Works:
                                                                                                                                                                                                                                                             - Pedagogical publications

                                                                                                                                                                                                 1
                                                                                                                                                                                                 Seven components of effective teaching from the TQF rubric
                                                                                                                                                                                                 2
                                                                                                                                                                                                 For FCQs, S = standard questions, CP = core pilot questions, and OP are [DEPT]-specific optional pilot questions.
                                                                                                                                                                                                 3
                                                                                                                                                                                                 Peer observations are based on draft items currently under review.

                      Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
The Teaching Quality
    Framework:
A model of improved
teaching assessment

     Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Conclusions and Future Directions

Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Conclusions and Future Directions
                              • How does the dept teaching eval. process
                                compare to TQF framework before starting
                                DATs?

                              • What counts as movement toward TQF
                                framework?

                              • Do we see a shift in departmental culture
                                toward a more scholarly approach to
                                teaching?

Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Interested in learning more?!
Hear more about TQF, DATs, and other TEval affiliated
projects at these sessions later today:
   • 1:15pm-1:45pm, Oral Presentation Session F in Woodlawn I (DATs):
          • Characterizing departmental culture and assessing change with the DELTA survey
   • 1:15pm-2:45pm, Thematic Symposium in Fountainview:
          • National Academies' Roundtable on Systemic Change in Undergraduate STEM
            Education: Directions and Opportunities
   • 2:45pm-3:30pm, Poster Session in Admiral (Student Evaluations):
          • Building a better SET: An "outside-in" approach to developing and implementing
            improved student evaluations of teaching
   • 3:30pm-5pm, Thematic Symposium in Waterfront (Rubric, etc.):
          • Improving Learning by Transforming the Evaluation of Teaching: Resources,
            Challenges, and Change Processes
                                                sarah.andrews-1@colorado.edu
                                                    @sarahfoofoo79
                                                    #ASCNHigherEdTI2019
 Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Questions?
We are grateful for the support of the National Science Foundation (grant
DUE-1725959), Bayview Alliance, the Association of American Universities,
CU Boulder College of Arts and Sciences, College of Engineering & Applied
Science, and Leeds School of Business.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of
the authors and do not necessarily reflect the views of any of these funders.

   Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
References
Association of American Universities. 2014. Framework for Systemic Change in Undergraduate STEM Teaching and Learning.
https://www.aau.edu/sites/default/files/STEM%20Scholarship/AAU_Framework.pdf
Benton, S.L., & Young, S. June 2018. Best practices in the evaluation of teaching. IDEA Paper #69. IDEA Center, Inc., Manhattan, KS.
https://files.eric.ed.gov/fulltext/ED588352.pdf
Berk, R.A. 2018. Start spreading the news: Use multiple forms of evidence to evaluate teaching. The Journal of Faculty Development 32(1): 73-81.
Bernstein, D. 2016. Now is a good time to recognize teaching as serious intellectual work. InSight: A Journal of Scholarly Teaching 11: 9-13.
http://insightjournal.park.edu/wp-content/uploads/2016/07/2-Editorial-Bernstein.pdf
Corbo, J.C., Reinholz, D.L., Dancy, M.H., & Finkelstein, N. 2015. Departmental Action Teams: Empowering faculty to make sustainable change. In 2015
Physics Education Research Conference Proceedings, pp. 91-94.
Doerer, K. Jan 13 2019. Colleges are getting smarter about student evaluations. Here’s how. The Chronicle of Higher Education.
https://www.chronicle.com/article/Colleges-Are-Getting-Smarter/245457
Finkelstein, N.F., and Keating, J. Aug 13 2018. Promoting scholarly evaluation of teaching: Addressing the third rail of academia. ASCB Newsletter.
https://www.ascb.org/science-news/promoting-scholarly-evaluation-teaching-addressing-third-rail-academia/
Hansen, W.L. Summer 2014. Rethinking the student course evaluation: How a customized approach can improve teaching and learning. Liberal
Education Vol. 100, No. 3. https://www.aacu.org/publications-research/periodicals/rethinking-student-course-evaluation
Mills, M., and Hyle, H.E. 1999. Faculty evaluation: A prickly pair. Higher Education 38: 351-371. https://doi.org/10.1023/A:1003735227936
National Research Council. 2003. Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics.
Washington, DC: The National Academies Press. https://doi.org/10.17226/10024.
Research Corporation for Science Advancement. 2015. Searching for Better Approaches: Effective Evaluation of Teaching and Learning in STEM.
https://rescorp.org/gdresources/publications/effectivebook.pdf
Tunks, K.W. Aug 20 2012. Transforming teaching through supplementary evaluations. Faculty Focus: Higher Ed Teaching Strategies from Magna
Publications. https://www.facultyfocus.com/articles/faculty-evaluation/transforming-teaching-through-supplementary-evaluations/
You can also read
NEXT SLIDES ... Cancel