Holistic Teaching Evaluation
In 2019, in response to discussions and observations about the current teaching evaluation practices, a joint Senate-Administration Workgroup on Holistic Teaching Evaluation was formed. The overarching goal for this workgroup was to develop a holistic, transparent, and flexible evaluation process, and to provide the tools and support needed for faculty and CAP reviewers to have more effective evaluations.
The 2019 workgroup produced a report that recommended the following:
- Separate formative assessment from summative evaluation of teaching
- Change the structure of the (undergraduate) student feedback forms (CAPEs).
- Evaluate teaching holistically based on “a teaching portfolio, an instructor’s self-reflection, and the chair’s summary.”
In 2020 a second senate and administration group- the Holistic Teaching Evaluation Implementation Task Force, was charged with developing a plan to implement the first workgroup's recommendations. Their report defined the following:
- The structure of the Teaching Portfolio and Teaching Statement
- A new structure for student feedback
- A method for evaluating teaching holistically
- An implementation plan is overseen by a new Holistic Teaching Review Oversight Committee
In 2022, a standing committee- the Holistic Teaching Oversight Committee, was formed to communicate and oversee the campus transition to holistic teaching evaluation.
Timeline for Rollout
- Academic Year 2022-23:
-
- Review of teaching in 2022-2023 files remains unchanged;
- Start transition from CAPEs to a revised student feedback mechanism (starting spring 2023);
- Develop resources to help faculty create teaching portfolios and to help departments, schools and CAP evaluate them;
- Encourage faculty who will be reviewed in 2023-2024 to start to incorporate elements of a teaching portfolio into their files
- Academic year 2023-24:
-
- Transition to new student feedback mechanism completes
- Campus reviewers begin to comment on teaching statements and other materials, in addition to student feedback forms/CAPEs (residual) and other forms of student input
- Academic year 2024-25:
-
- Encourage more detailed teaching statements to match additional portfolio material (including student input, syllabi, and other evidence of teaching effectiveness)
- Campus reviewers include more elements in evaluating teaching effectiveness
- Academic year 2025-26:
-
- Holistic teaching evaluation reaches steady-state
Resources for Holistic Teaching Review
If you have opted into the Holistic Teaching & Mentoring Review for this coming 2023-2024 review cycle (and academic year), please find resources below to help you put together your file. These are drafts and the Holistic Teaching Oversight Committee would welcome any feedback you are willing/able to provide on these materials.
Student Evaluation Forms- these will replace CAPEs
We have developed a new student evaluation form called the Student Evaluation of Teaching (SET) form, which will replace CAPEs in undergraduate courses and the graduate evaluation forms in graduate courses. The goals of this new form are
- to ensure that student feedback is focused on student learning and teaching practices;
- to minimize bias in student evaluations; and,
- to ensure that student voices are well-positioned as part of the evaluation of teaching.
Below we give information about the new SET questions, as well as the timeline for the development and rollout of this form.
Review the new questions here.
Transition Timeline- from CAPEs to SET Forms
Fall 2022:
- (1) New Student Evaluation of Teaching (SET) forms developed based on 2019 WG report
- (2) SET forms piloted with 13 students (both undergrad and grad) who provided detailed feedback
Winter 2023:
- SET forms finalized and implemented in Eval System
Spring 2023:
- Faculty can opt-in to using SET forms instead of CAPE for undergrad courses.
Summer 2023:
- SET forms replace CAPE for all undergraduate courses
Fall 2023:
- SET IA forms replace existing IA evaluation forms
- SET Data will be available on the evaluations site
Winter 2024:
- SET forms replace grad course evaluation forms
Committee Members & Contact Info
2023-2024 Senate and Administration Standing Holistic Teaching Evaluation Oversight Committee
Christine Alvarado- Associate Dean Division of Undergraduate Education- Committee co-chair
Phoebe Bronstein- Associate Teaching Professor, Sixth College- Committee co-chair
Marisa Abrajano- Provost, Warren College
Adair Appleton- Graduate & Professional Student Association Representative
Jelena Bradic- Professor, Mathematics & HDSI
April Burcham- AP Analyst, Division of Undergraduate Education
Hailey Caraballo- Project Policy Analyst, Division of Undergraduate Education
Sekhar Chivukula- Professor, Physics
Maureen Feeley- Teaching Professor, Political Science
Vic Ferreira- Associate Vice Chancellor for Faculty Equity, Diversity, and Inclusion
Judy Kim- Associate Dean, Division of Graduate Education and Post-doctoral Affairs
Stanley Lo- Professor, Cell & Developmental Biology
Gilberto Mosqueda- Professor, Structural Engineering
Dana Nelkin- Chair, Philosophy
Douglas Nitz- Chair, Cognitive Science
Julia Partridge- Director, Academic Advancement and Compensation
Todd Pawlicki- Professor, Radiation Medicine
Carolyn Sandoval- Associate Director, Teaching +Learning Commons
Ulrike Schaede- Professor, School of Global Policy and Strategy
David Stegman- Associate Professor, SIO
Kabir Vats- Associated Students Representative
Shahrokh Yadegari- Professor, Music
FAQ
Questions about holistic teaching evaluation and portfolio development
- What is the goal of moving to a holistic evaluation process?
- Answer: There are two goals of moving to a holistic teaching evaluation process.
First, in reference to evaluation of teaching, APM-210 states: “More than one kind of evidence shall accompany each review file.” However, on the UC San Diego campus we have been overly reliant on student evaluation data for overall teaching evaluation. The holistic evaluation process will broaden and strengthen the teaching and mentoring evaluation process by incorporating more sources of evidence and providing a more complete picture of a candidate’s teaching.
Second, the holistic evaluation process will elevate the engagement and discussion around teaching and mentoring on our campus by asking faculty to reflect on their teaching and mentoring strengths and their opportunities to continually grow as a teacher and mentor.
- Answer: There are two goals of moving to a holistic teaching evaluation process.
- How is the new teaching evaluation process different from the current teaching evaluation process?
- Answer: The new evaluation process aims to paint a fuller picture of an individual faculty member’s teaching using multiple data points, rather than relying solely on CAPEs (SETs) to assess strong teaching. This is a chance for faculty to narrativize their teaching and better showcase their hard work.
- I’ve heard something about a teaching and mentoring statement and portfolio being required. What does that mean?
- Answer: With holistic teaching evaluation, all faculty will be asked to submit a teaching and/or mentoring statement and a teaching portfolio. The Holistic Teaching Evaluation Oversight Committee is finalizing the instructors for the statement and portfolio, and we will post information soon. In the meantime, you can get a sense for what this will look like by reading the Holistic Teaching Evaluation Implementation Task Force report.
- I’m being reviewed this year. Can I submit a teaching portfolio?
- Answer: Yes! Faculty are encouraged to start submitting a teaching portfolio at any point in this process. In fact, many campus faculty have already been submitting a portfolio. However, the precise instructions and expectations for portfolios are still being developed, so we ask that you keep an eye on this site (and your email) as you are getting ready to prepare your materials. We will be updating the materials and rolling out calls for people who are willing to officially pilot the new holistic review process in their next review. Don’t hesitate to reach out to us at holistic-teaching@ucsd.edu with any questions you have.
- How much more time will this take?
- Answer: There is no exact or precise answer here, but it will take some additional time if this is the first time pulling together the portfolio documents and writing about teaching. To cut down on the time it takes, faculty members can keep a running list of their accomplishments at UCSD and/or save their syllabi and teaching evaluation at the end of each quarter.
- Will the campus be providing additional staff support and resources for this additional work?
- Answer: The campus will not be providing additional staff support. However, the Commons has multiple resources to support faculty in developing their teaching and mentoring portfolios and statements. Additionally, AP staff have been active members of the Holistic Teaching Evaluation Oversight Committee (HTOC) committee and the committee will continue to work closely with those supporting the review process.
Faculty can also reach out to the Teaching + Learning Commons for support on putting together a portfolio or look at this useful workbook on documenting and reflecting on teaching (from T+LC).
Questions about new Student Evaluations of Teaching (SET)
- What is the rationale for developing the new SET questions? What was wrong with CAPE?
- Answer: CAPE was originally developed as a student-run service. The original questions were developed by students, for students, and not explicitly intended to evaluate the quality of an instructor’s teaching. Since then, the faculty review process has become (overly) reliant on CAPEs to evaluate teaching, which the CAPE form was never explicitly designed to support.
The transition to SET, which is managed by the Holistic Teaching Evaluation Oversight Committee–a committee that includes faculty, students and staff, will ensure that the forms fulfill the dual purposes of providing information to students to help them choose what classes to take, and providing information from students to help evaluate an instructor’s teaching.
- What is different about the SET questions compared to the CAPE questions?
- Answer: The SET questions are designed to focus on aspects of student learning and teaching practices. More notably, the “summary” questions (“Would you recommend this course/instructor?) have been removed as general questions like these are shown to lead to biased responses (e.g. based on the gender, race, age, etc. of the instructor).
- Will removing the quantitative summary questions have a negative impact on students’ or the department’s ability to evaluate the course and the instructor? What will be considered as a successful teaching evaluation?
- Answer: It is true that students and evaluators will no longer be able to use these “overall” scores to form a quick (but potentially biased) assessment of a course or instructor. However, the questions on the current form provide a more authentic insight about the quality of the course.
- When and where will the public SET data from students be available?
- Answer: It will be available by the beginning of Fall for the courses that used SET this summer and spring, on the evals site.
For any additional questions, contact holistic-teaching@ucsd.edu