Oral assessments and discussion at scale: Rethinking the teaching fellow’s role
Motivation
Many instructors are considering bringing oral assessment back into their classrooms. Oral assesments can test skills that AI still cannot automate and to restore the human-to-human connection that is part of the university’s value proposition. A challenge, however, is scale. Oral assessments are labor-intensive: Can they be implemented in anything larger than a discussion seminar?
I chatted with statistics instructors at UC Berkeley on how they have, this year, managed to run oral assessments for hundreds of students.
The approach I’ll present here is to achieve scale by changing the expectations of a teaching fellow. At both Berkeley and Yale, teaching fellows (TFs, or GSIs) typically are paid for 20 hours a week of work. But in reality, I’d guess TFs spend less than 20 hours a week on the class if: (a) students do not come to TF’s office hours, (b) the material is easy for the TFs so section requires less prep than say, a literature class, and (c) discussion sections are not mandatory for students so they do not show up to section anyways. I think all three conditions are fairly common in a stats class today.
[Disclaimer: this post is about classes where TFs are available in the first place. I’m aware in many other institutions, PhD students need to teach their own class. The ideas here is a middle ground in that sense: One large class might provide returns to scale while still maintaining the small-group interaction.]
A 50-student class
Instead, perhaps a TF could run mandatory discussion sections that are with about 3-4 students each time, and run for about 1 hour (50 minutes). For example, for a 50-student class with two TFs, this could be one TF’s breakdown:
| Task | Hours / week / TF |
|---|---|
| Six 50 min tutorial sections, 4 students each | 6 |
| Preparation for the section | 3 |
| Attending lecture | 3 |
| Grading | 3 |
| Administrative work, meeting with instructor | 3 |
| Modifying problem sets, coming up with exam questions | 2 |
| Total | 20 |
Together, two TFs can service 48 students (see table below). This meets Yale’s rough target in a social science class that there should be one TF for 36 students, and also meets Princeton’s note that they expect TFs to have at most 6 hours of in-person teaching a week.
Notice in this breakdown there are no office hours, as the sections are small enough to handle student questions. I am also penciling few hours for grading because in an intro or intermediate-level statistics class, most of the grading could be sped up by automatic coding / AI assistance. In the past, classes like mine may have been built around high-frequency problem sets requiring a lot of grading, but I fear this model will go extinct soon as classes are increasingly only graded for completion, due to AI.
Six scheduled hours of in-person instruction is much more than what the typical TF would do in the schools where I’ve taught or studied. But what would we rather have teaching fellows do: Spend 6 hours a week grading statistics quizzes (and potentially AI-generated essays), or spend 6 hours a week conversing directly with students?
Scaling up to 200
At Berkeley, Andrew Bray pulls off not one but multiple oral assessments for each of his roughly 200 students in Stat 133, a required class for its stats majors on data science, data visualization, and computation (it has a great course website with material). Students in this class prepare five projects during the semester, in addition to seven problem sets. They must bring each completed project to a section and be able to answer questions about it.
He notes that each TF working 20 hours a week can cover 70 students, with the following once-every-two-week schedule:
- Every student meets in groups of three for 20-30 minutes with a TF for their graded session, which occurs once every two weeks
- A TF can teach twelve 30-minute sections a week, which is six hours of in-class time and covers 35 students a week
- TFs grade presentations in person during the section, checking off a standard rubric as each student presents.
TFs also hold 2 hours of office hours, grade open-ended questions in quizzes, may be asked to attend lectures, and help with tasks like releasing quizzes and assignments. Grading for closed-ended questions is done automatically by software.
Summary in Numbers
| Attribute | Class of 50 students | Lecture of 200 students |
|---|---|---|
| Student experience | ||
| Section length | 50 minutes | 30 minutes |
| Enrollment per section | 4 | 3 |
| Frequency | Weekly | Once / 2 weeks |
| TF workload | ||
| TF paid for | 20 hours / week | 20 hours / week |
| Weekly time allocated to tutorials per TF | 6 hours | 6 hours |
| Full tutorials one TF can teach each week | 6 | 12 (6 hours) |
| Students one TF can cover | 24 | 72 |
| Class staffing | ||
| Tutorial groups needed | 12-13 | 66-67 |
| Weekly tutorial contact hours across the class | 11 hours | 17 hours |
| TF equivalents needed | 50 ÷ 24 ≈ 2.1 | 200 ÷ 72 ≈ 2.8 |
| Staffing plan | 2 TFs | 3 TFs |
Note: These numbers are not meant to accurately describe any real class, but are constructed to provide a proof of concept and based loosely on my conversation with Andrew.
Conclusion
If oral, 1:1 human interactions are the critical part of an undergraduate learning experience, it makes sense to reorient the TF tasks that way. That might mean:
- Sections (tutorials) should be small, say 3-4 students to allow for in-person assessment and teaching
- They are also mandatory and graded on performance, not just showing up
- The TF reorients their budgeted hours to face-to-face time, rather than office hours (which seem increasingly unattended) or grading (which seem increasingly unimportant as problem sets lose their weight).