Last Monday, the faculty of Princeton University, where I teach, voted to reinstitute the proctoring of exams after more than 130 years of reliance on an honor code to prevent cheating.
As someone who has been working the problem of transactional attitudes towards school since before ChatGPT appeared I'm deeply sympathetic to these challenges. We know that students need to do the actual stuff or nothing is going to be learned.
But as you intuit, things like policing LLM use or oral exams or all in-person writing all run up against downsides that, really, are unacceptable. In my view, the only route towards a solution is to move away from the framework of "schooling" and toward a root-level examination of the experiences of "learning." Because I am old and have written and taught writing for decades, I "know" that the struggle of learning is the point, but students at elite universities especially have been working inside a system that privileges achievement and optimization. The hard part is that we can't force the struggle on students through simply being more punitive. They have to opt-in, but to do so they have to be helped to understand what the struggle entails, how it's meaningful, how to manage it, and perhaps most importantly, they have to be given experiences that they believe are worth doing - rather than outsourcing - and then we have to assess them in ways that genuinely value the struggle.
I've done something like 60 talks and workshops exploring this challenge over the last 2+ years at institutions all across the country and up and down the latter of selectivity/prestige, and I can report that there is great progress to be made, but also, the challenge appears to be hardest at places like Princeton because of the way prestige and achievement drive the culture.
One of the best skills you develop in the humanities is the ability to write. Researching and writing longer papers was extremely valuable in my later career as a lawyer. Moreover, reading quality writing from others teaches you how to write better yourself.
I understand the need to stamp out cheating, but would hate to see all term papers eliminated.
University administrators need to place severe consequences on those who cheat. A few expulsions might send the right message.
A very well-written piece, though something I've asked myself a lot in this discussion relates to how we grade. If oral exams are the only way to effectively assess students' knowledge of a subject, but the examination process is limited by the school's resources, then why not let the students jury each other?
I think we have a fear of peer-based grading in America because of how we grade: starting at 100 and taking points off, rather than starting at 0 (like in France) and working our way up to 20.
I believe that if the latter system were given to a group of students, along with a well-laid-out rubric, the quality of grading would be only slightly variant from that of a TA.
Along with that, an oral exam essentially nullifies the need for integrity on anything written alongside it, being that students must know, be able to articulate, and defend their position regardless of their use of AI.
I think a real possibility for examinations could look something like this:
Students are given a prompt for research
They disperse and write their papers independently outside of class
They come back together and are broken into groups for examination (let's say groups of five)
Members of each group take turns presenting to each other, taking notes, asking questions... etc.
The student's grades are sent to the professor along with their paper, giving five data points in total: four graded reports on their oral section, and the paper to be graded traditionally by the professor.
We don’t need to return to deciphering script in blue books. Lockdown browser do the job of preventing cheating in tests without forcing us to decipher handwriting. The bigger challenge is not exams but research papers. Is an in-class exam an appropriate final assignment for a research seminar? I don’t think so. But the research paper is an invitation for using AI. Does anyone have any advice on how to evaluate students’ performance in research seminars? What about graduate students? I find that graduate students are just as likely to use AI in their assignments as undergraduates . Should we police it? Or should we just ignore the AI, give them all A’s since grades really don’t matter in graduate school and avoid the unpleasant task of trying to figure out if and how our students are cheating.
One component that departments can consider is moving away from the idea that all courses are equal and should teach the same skills. What if, for upper-division courses, departments moved to some large lecture courses that focus on in-class exams -- courses centered on sets of content and context -- and some small research seminars focused on students producing writing and presenting their final papers orally. I teach our senior thesis course and work closely with a group of about 10 students. Rather than 2-3 class sessions a week, we have individual meetings throughout the semester to discuss their projects and how they are developing (and occasional class meetings where they critique and advise each others' projects). They have to produce a series of prospectuses that work to define the project, identify primary and secondary sources, and establish an outline. At the end of the semester, they have to do an oral presentation of their paper. While perhaps students use AI to assist them in some elements of this, based on the conversations we have in our meetings I am confident that the projects that they produce are theirs. What if students had to take 4 of these research-focused courses (and a variety of formats focused on writing might be offered) and 6-7 of the large lecture courses, with faculty offering on rotation fewer of the large lecture courses and more of the research seminars.
Good piece. It’s painful to see how university education has deteriorated, re evaluation of students and freedom to learn/ freedom to teach. The humanities disappearing. The managers (and politicians) setting the course. Students as customers. Etc. I’ve been retired since 2001 and am grateful that my problems with cheating students and open discourse were so much simpler then. They really were the good old days (in many respects).
As usual a very thoughtful and stimulating piece. I think we all need to go further, however, and realize that the structure of the humanities in particular is going to be radically transformed and perhaps needs to be. Not to mention the university as an institution. We can't just resist what is happening. We need to think proactively about what would be a good outcome of radical changes. AI will be part of it. As someone who taught for most of her career in large universities with big lecture classes and many TAs, it's obvious that this model cannot continue -- there will be fewer TAs, fewer jobs, and we need to figure out ways to teach content but also research and writing skills in this context. I think this means more teaching for everyone, some way of scaling up the liberal arts college model.
This and many complaints like it fail to imagine that education and intellectual work generally may be changed by technology. Education was transformed by writing (Plato, Phaedrus, D274), printing, and computing.
Mathematicians and engineers changed substantially what they taught after programmable computers became ubiquitous.
So educators will develop students’ capacity to do different and probably more sophisticated intellectual tasks with artificial intelligence.
Plato. Phaedrus Plato in twelve volumes, volume 9 translated by Harold N. Fowler. Cambridge, MA, Harvard University Press; London, William Heinemann Ltd. 1925.
Thank you for once again raising a crucial question that needs to be really thought through across the humanities and higher education. I think its quite important to identify the specific learning outcomes associated with teaching a history (or any humanities) course and of the assessments we use within those courses, because those should inform both the work we ask students to do and how we assess it. There is also the question of the professional responsibility of the individual faculty member within that.
Most of the teaching I do is quite different from what a Princeton history class instructor does in terms of learning outcomes and assessments. I am not sure what a "traditional term paper" is in my context, nor am I sure what an in-class written examination is intended to assess. For students in my classes, the majority of whom are taking 1 of 2 humanities courses as distributional requirements, expository written and oral communication competencies are especially crucial outcomes -- and in that sense, I do not think they are developing the competencies they need if they are NOT using AI (or more precisely aware of what they are doing when they use different forms of AI -- searching, note-taking, organizing or drafting/ revising). And this applies not only to written papers but to graphical presentations of information in slides or other presentantion formats.
So one thing I have tried to do, and wish I knew of more informed guidance on how to do this, is having students provide in their "critical apparatus" (notes or bibliography) and indication of the AI components of their work. Im hoping in the coming year to develop a format for this which I could use across assignments -- should it be a prefatory prose note, should it be a specific citation associated with parts of the work that may have been AI generated or AI-assisted. I also think there is a value in teaching students how to develop appropriate prompts for generative LLMs such as research queries, or for orgnazational requests, or for revision requests.
I think this is not really what my role should be as a classroom instructor. My university has a whole task force on AI and a bunch of resources for faculty but it seems to be largely produced by and for folks who work in areas that use quantitative data. I think really this sort of thinking should be done at the level of general education (which is being rapidly diminished as a share of undergraduate curriculum as pressure for a shorter time to degree and greater focus on vocational durable skills drives higher education policy esp over public institutions). It should be addressed in programs like "Writing Across the Curriculum" (which we do not have at my university) or in writing centers (which are generally staffed by graduate students in English or other language / culture disciplines but maybe need to have technical people involved in the future).
And finally and without a doubt, much of the real underlying question here is cost for the institution -- in particular the largest part of the cost of humanities education and of general education, which is staff compensation. The perception of AI as the latest in a series of technological innovations -- MOOCs being the most recent, I think -- which were viewed as a means to bring efficiencies to higher education is really the one that I think requires administrations to give the most thought and study. It would the sort of thing that a national body interested in higher education on behalf of the common good of the country, to balance cost and outcome, and to assess value, should take up.
As someone who has been working the problem of transactional attitudes towards school since before ChatGPT appeared I'm deeply sympathetic to these challenges. We know that students need to do the actual stuff or nothing is going to be learned.
But as you intuit, things like policing LLM use or oral exams or all in-person writing all run up against downsides that, really, are unacceptable. In my view, the only route towards a solution is to move away from the framework of "schooling" and toward a root-level examination of the experiences of "learning." Because I am old and have written and taught writing for decades, I "know" that the struggle of learning is the point, but students at elite universities especially have been working inside a system that privileges achievement and optimization. The hard part is that we can't force the struggle on students through simply being more punitive. They have to opt-in, but to do so they have to be helped to understand what the struggle entails, how it's meaningful, how to manage it, and perhaps most importantly, they have to be given experiences that they believe are worth doing - rather than outsourcing - and then we have to assess them in ways that genuinely value the struggle.
I've done something like 60 talks and workshops exploring this challenge over the last 2+ years at institutions all across the country and up and down the latter of selectivity/prestige, and I can report that there is great progress to be made, but also, the challenge appears to be hardest at places like Princeton because of the way prestige and achievement drive the culture.
Thank you!
I meant to say that if you ever want someone to talk through these things with. I’m easy to find, and happy to do so.
One of the best skills you develop in the humanities is the ability to write. Researching and writing longer papers was extremely valuable in my later career as a lawyer. Moreover, reading quality writing from others teaches you how to write better yourself.
I understand the need to stamp out cheating, but would hate to see all term papers eliminated.
University administrators need to place severe consequences on those who cheat. A few expulsions might send the right message.
A very well-written piece, though something I've asked myself a lot in this discussion relates to how we grade. If oral exams are the only way to effectively assess students' knowledge of a subject, but the examination process is limited by the school's resources, then why not let the students jury each other?
I think we have a fear of peer-based grading in America because of how we grade: starting at 100 and taking points off, rather than starting at 0 (like in France) and working our way up to 20.
I believe that if the latter system were given to a group of students, along with a well-laid-out rubric, the quality of grading would be only slightly variant from that of a TA.
Along with that, an oral exam essentially nullifies the need for integrity on anything written alongside it, being that students must know, be able to articulate, and defend their position regardless of their use of AI.
I think a real possibility for examinations could look something like this:
Students are given a prompt for research
They disperse and write their papers independently outside of class
They come back together and are broken into groups for examination (let's say groups of five)
Members of each group take turns presenting to each other, taking notes, asking questions... etc.
The student's grades are sent to the professor along with their paper, giving five data points in total: four graded reports on their oral section, and the paper to be graded traditionally by the professor.
Just a few thoughts,,,
Thank you! Very interesting.
We don’t need to return to deciphering script in blue books. Lockdown browser do the job of preventing cheating in tests without forcing us to decipher handwriting. The bigger challenge is not exams but research papers. Is an in-class exam an appropriate final assignment for a research seminar? I don’t think so. But the research paper is an invitation for using AI. Does anyone have any advice on how to evaluate students’ performance in research seminars? What about graduate students? I find that graduate students are just as likely to use AI in their assignments as undergraduates . Should we police it? Or should we just ignore the AI, give them all A’s since grades really don’t matter in graduate school and avoid the unpleasant task of trying to figure out if and how our students are cheating.
One component that departments can consider is moving away from the idea that all courses are equal and should teach the same skills. What if, for upper-division courses, departments moved to some large lecture courses that focus on in-class exams -- courses centered on sets of content and context -- and some small research seminars focused on students producing writing and presenting their final papers orally. I teach our senior thesis course and work closely with a group of about 10 students. Rather than 2-3 class sessions a week, we have individual meetings throughout the semester to discuss their projects and how they are developing (and occasional class meetings where they critique and advise each others' projects). They have to produce a series of prospectuses that work to define the project, identify primary and secondary sources, and establish an outline. At the end of the semester, they have to do an oral presentation of their paper. While perhaps students use AI to assist them in some elements of this, based on the conversations we have in our meetings I am confident that the projects that they produce are theirs. What if students had to take 4 of these research-focused courses (and a variety of formats focused on writing might be offered) and 6-7 of the large lecture courses, with faculty offering on rotation fewer of the large lecture courses and more of the research seminars.
Excellent. Thanks.
Good piece. It’s painful to see how university education has deteriorated, re evaluation of students and freedom to learn/ freedom to teach. The humanities disappearing. The managers (and politicians) setting the course. Students as customers. Etc. I’ve been retired since 2001 and am grateful that my problems with cheating students and open discourse were so much simpler then. They really were the good old days (in many respects).
As usual a very thoughtful and stimulating piece. I think we all need to go further, however, and realize that the structure of the humanities in particular is going to be radically transformed and perhaps needs to be. Not to mention the university as an institution. We can't just resist what is happening. We need to think proactively about what would be a good outcome of radical changes. AI will be part of it. As someone who taught for most of her career in large universities with big lecture classes and many TAs, it's obvious that this model cannot continue -- there will be fewer TAs, fewer jobs, and we need to figure out ways to teach content but also research and writing skills in this context. I think this means more teaching for everyone, some way of scaling up the liberal arts college model.
This and many complaints like it fail to imagine that education and intellectual work generally may be changed by technology. Education was transformed by writing (Plato, Phaedrus, D274), printing, and computing.
Mathematicians and engineers changed substantially what they taught after programmable computers became ubiquitous.
So educators will develop students’ capacity to do different and probably more sophisticated intellectual tasks with artificial intelligence.
Plato. Phaedrus Plato in twelve volumes, volume 9 translated by Harold N. Fowler. Cambridge, MA, Harvard University Press; London, William Heinemann Ltd. 1925.
http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.01.0174%3Atext%3DPhaedrus%3Apage%3D275
Thank you for once again raising a crucial question that needs to be really thought through across the humanities and higher education. I think its quite important to identify the specific learning outcomes associated with teaching a history (or any humanities) course and of the assessments we use within those courses, because those should inform both the work we ask students to do and how we assess it. There is also the question of the professional responsibility of the individual faculty member within that.
Most of the teaching I do is quite different from what a Princeton history class instructor does in terms of learning outcomes and assessments. I am not sure what a "traditional term paper" is in my context, nor am I sure what an in-class written examination is intended to assess. For students in my classes, the majority of whom are taking 1 of 2 humanities courses as distributional requirements, expository written and oral communication competencies are especially crucial outcomes -- and in that sense, I do not think they are developing the competencies they need if they are NOT using AI (or more precisely aware of what they are doing when they use different forms of AI -- searching, note-taking, organizing or drafting/ revising). And this applies not only to written papers but to graphical presentations of information in slides or other presentantion formats.
So one thing I have tried to do, and wish I knew of more informed guidance on how to do this, is having students provide in their "critical apparatus" (notes or bibliography) and indication of the AI components of their work. Im hoping in the coming year to develop a format for this which I could use across assignments -- should it be a prefatory prose note, should it be a specific citation associated with parts of the work that may have been AI generated or AI-assisted. I also think there is a value in teaching students how to develop appropriate prompts for generative LLMs such as research queries, or for orgnazational requests, or for revision requests.
I think this is not really what my role should be as a classroom instructor. My university has a whole task force on AI and a bunch of resources for faculty but it seems to be largely produced by and for folks who work in areas that use quantitative data. I think really this sort of thinking should be done at the level of general education (which is being rapidly diminished as a share of undergraduate curriculum as pressure for a shorter time to degree and greater focus on vocational durable skills drives higher education policy esp over public institutions). It should be addressed in programs like "Writing Across the Curriculum" (which we do not have at my university) or in writing centers (which are generally staffed by graduate students in English or other language / culture disciplines but maybe need to have technical people involved in the future).
And finally and without a doubt, much of the real underlying question here is cost for the institution -- in particular the largest part of the cost of humanities education and of general education, which is staff compensation. The perception of AI as the latest in a series of technological innovations -- MOOCs being the most recent, I think -- which were viewed as a means to bring efficiencies to higher education is really the one that I think requires administrations to give the most thought and study. It would the sort of thing that a national body interested in higher education on behalf of the common good of the country, to balance cost and outcome, and to assess value, should take up.
Thanks for this, Greg! Really interesting and important thoughts.
Good essay. But why aren’t long midterms and finals a solution. The British system is traditionally based on exams.
It’s partly a solution. But the British system also uses essays and tutorials.