Encouraging critical use of AI
by Marcus Birkenkrahe
Jim Monaghan asked what we can do to “ensure that students develop their critical thinking, analysis, and writing skills in an age of AI?” He also attached a link to a CHE article, “The New Nanny State”, which I could not access - but the subtitle perhaps says enough: “In their fight against AI, professors are monitoring their students in ways previously unimaginable.”
Leaving aside the question whether “fighting against AI” is a rational response to AI (I think not): I think that policing, or (more) nannying, is useless - though existing academic policies may force teachers to go down that route. What we need is more, not less independence, enabled by the increased power to build that AI gives us, and encouraged by open dialogue between AI-doomers and AI-boomers.
Here’s why: AI is separating the finished product from the thinking that used to be needed to make it. So an essay or a working program no longer necessarily shows anything.
Ways to get around that include:
- assessing the process (not just the product);
- bringing back the spoken word;
- using AI as something to critique, not just a tool;
- anchoring work in local, specific context;
- reducing reasons to cheat;
- showing the best uses of AI.
In my computer science (CS) courses at Augustana University (SD) I’m implementing most of these already, with some success (long-term effects remain to be seen):
- My CS students submit literate programs (notebooks) that must reflect their thinking and solution process when arriving at a solution.
- I encourage them to use AI as part of a human hypothesis -> AI response -> human reconciliation process.
- All assignments, quizzes etc. are specific to what we talked about in class that week (no reuse whatsoever).
- I allow AI, but I insist on specific, detailed AI-use statements.
- I create interactive companions to class content with AI for the students to use (or extend themselves) - here is an example of such a companion for a CS textbook we’re using: birkenkrahe.github.io/demos.
The graph above shows an early result of my “specify your AI use” policy for one course: by midterm, it has become a habit for most of my students to declare their AI use, and to open themselves to debate (which was the whole point). I’m very impressed: most of them are quite specific about their uses. They are:
- Understanding a task or a concept
- Fixing compile errors
- Checking an answer after a first attempt at solving the problem
- Line-by-line explanations or formatting
Only in 10% of the cases are students letting AI write all the code. This shows that AI use is common but not universal, that honest AI disclosure can become natural, and that specific disclosures (though lagging) are achievable.