Teaching Quality Feedback Reports (TQFR)

TQFR began replacing paper course evaluations for the Institute in 2004. It was implemented in phases, starting with a select group of courses. Currently, TQFR is the Institute's official student feedback system for all course offerings.

The purpose of TQFR is to collect data from students regarding courses' teaching quality. Students are an important source of information about the effectiveness of a course, its instructor, and its teaching assistants. Results from the surveys and a summary of numerical data are given anonymously to the instructor, TAs, and division chair following the end of the term. Portions of the results are published for general distribution to the campus community.

The process for reviewing TQFR comments is:

1. Administrative Tool - Student comments are reviewed by an internal tool that is programmed to look for profanity. The tool is part of the TQFR and will automatically change the profane word to "(omitted)".
2. Lead Analyst for Student Information Systems – All student comments are reviewed anonymously by the Registrar's Office's Analyst. The Analyst reviews for profanity and checks the legitimacy of changes made by the Administrative Tool. The tool occasionally flags words which are not profane in context. If appropriate, the Analyst changes the tool's "(omitted)" back to the original word.
3. The Analyst scans newly submitted comments to see if there are other items that may need review.
4. The Analyst forwards flagged comments (without the student's identity) to the Registrar for review. See below for more information on flagged comments.
5. The Registrar may remove comments temporarily or permanently as a precaution until the appropriate leadership has had the opportunity to investigate and advise.
6. Leadership contacts will work with the Registrar on how to temporarily or permanently handle the comment.
7. Record keeping – For transparency when comments are removed or edited, the Registrar will ensure that there is notation in the places where content was changed. Changes to comments (raw, public or administrative) are logged in a document on the Registrar's drive.
8. Faculty, staff, and students may report comments or request changes by emailing tqfr@caltech.edu. The email is checked by the Analyst who will work with the Registrar to address the specific concern.

Flagged Comments

1. Comment contains profanity – if minor, addressed by the Administrative Tool and reviewed by the Analyst. The Analyst logs changes to comments in a spreadsheet on the Registrar drive.
2. Comment contains content that the Analyst feels may need further review – the Analyst forwards the comment to the Registrar without the student's identity.
   Mildly vulgar content – If the Registrar feels it is appropriate, they will ask the Analyst for the student's name and will notify them that the comment was flagged. If the Registrar feels that the comment does not need further leadership intervention, they will ask the student if they would like to reword the comment. In almost every case, the student rewrites it in a more constructive manner.
   Content that may involve discrimination, harassment, or other highly sensitive items – the Registrar will send the anonymous comment to the Assistant Vice President for Equity and Equity Investigations, Title IX Coordinator and/or the Provost's Office delegate as appropriate. If the contact confirms that the comment does warrant their intervention, the Registrar will provide them with the student's name and contact.

Public vs. Internal – TQFR can only be accessed through access.caltech.edu. Therefore, only active students, faculty and staff will be able to see "Public" student comments. Internal comments are viewable to those attached to the course. There are four types of comments and each has its visibility explained to the student within the survey.

Course Comment – Internal comment for the course itself.

Instructor Comment – Internal comment for the instructor. Each instructor will have their own section.

TA Comment – Internal comment for a TA. Each TA will have their own section, and will be able to view their own comments, but not those of other TAs in the course. Instructors can view TA comments for all TAs in the course.

Public Comment – Comment that will be visible to anyone who can log into TQFR.

Raw Comment – TQFR stores the "raw" comment for historical purposes. Due to the fact that some of the comments are later deemed necessary for edits and omissions, individuals who have access to raw comments is restricted to a few individuals. The Analyst and developer can see the raw comment through the same tool that holds the student's name. Only the developer has the access to alter a raw comment.

* In some cases, Executive Officers and various Division Administrators have been given the same access as the Division Chair.

• For edits to comments, the Registrar may decide to share the original comment with administrators with internal access to the course (Registrar, Faculty, Chairs, Provost's Office, etc.).
• We do not allow students to write in their name or the names of their classmates. Comments that include personal identifiable information may be altered to preserve anonymity.
• TQFR forms and procedures will be reviewed annually by representatives from the Provost's Office, Registrar's Office, and CTLO.
• Former faculty and TA's can request copies of their own TQFR results from the Registrar's office who retains the requests on their secure shared drive.
• Tenure and other faculty review committees can request TQFR's of an individual faculty member.

For as long as possible, and as appropriate, anonymity is preserved through the above chain. Student identities can be provided if required for further investigation or for follow-up for clarification. Reasons for providing the student's identity to areas outside of the Registrar's office may include safety and legal concerns.