Browse Comments — Relevant (AI ∩ value)

Dr. Dean Fido CPsychol AI review is a large amount of work, often outside personal expertise at least in politics that is heterogenous as a field, often operating from pilot processes (we have a new AI declaration the students sign that is the basis of the review this year) and if it goes rampant, I had 26 cases in a class last year, it is a month of work that is not properly workloaded. The advantage of our new AI declaration is that it allows for class level customisation. So in one of my classes I piloted a -30 points flat punishmrnt for hallucination they can challenge with an office hour discussion with me that would determine if it was a honest mistake instead. Got a few, nobody challenged them. I did not involve AI officers. But agentic AI can mechanically cross-check references, particularly if you load all the sources, so this way of catching AI misuse is disappearing. An agent writes the essay or corrects it adfing references, one downloads the pdfs, one does the cross-check, ranks probability of hallucinations, and then the student does a manual cross-check. Maybe a couple of hours of work, perfect referencing, zero reading and learning (apart for a bit of AI agent setup following youtube videos).

There can’t be anything more demoralising to a lecturer than marking work you can clearly see is AI-generated but it can’t be conclusively proved by available integrity software. How is it even possible to allocate a fair mark in those circumstances, especially when there is a clear disjoint between the standard of the student’s usual performance in class and the standard of the dissertation submitted. A possible solution is to call for an oral defence of the work.

Two years ago I proposed to go luddite in year 1 and 2 of university (pen and paper only, exams, orals), and then in the 3d year introduce classes that try to develop an AI augmented pedagogy. Some of us are starting to use more and more AI so we are actually getting a lot of ideas on how to teach it. But the political economics and climate impact of AI and the privacy issues around its systematic usage in the university is a crucial barrier. I think we need local models, specifically designed for university learning, that are free, and designed to promote critical thinking and less cognitive offloading.

Thank you for writing this so honestly, Paolo. The 'raised the floor' line is the one I keep sitting with. The layer I'd add. We have spent two years asking 'did AI write this?' The harder question is the one you are already moving toward. What can this student actually do that they could not do before? When the answer to that has to show up live, in a draft we watch grow, in a defense, in a problem we put in front of them, the detection question quietly retires. Looking forward to what your department settles on.

We have to find alternative ways to ensure the use of AI is limited to copiloting and not replacing the writing process. An approach, maybe, is to ask for regular review chapter by chapter and incorporating this in the assessment grading. Also a viva is a must and potentially has to carry a higher weighting where students unable to defend what they've written will give them out as potentially using AI irresponsibly. But it will take a lot of honest admission and "thinking out of box" and do away with some academic orthodoxy.

I think part of this discussion may be focusing too narrowly on assessment methods. In reality, we already have many alternative approaches available: oral exams, pen-and-paper assessments, practical activities, project-based work, and many others. We can certainly continue designing new assessment strategies adapted to the AI era. At the same time, teachers themselves can also benefit from AI to support assessment, feedback generation, and learning analytics. For this reason, I believe we indeed need to rethink teaching, learning, and assessment processes more deeply, but I also see many opportunities to achieve positive outcomes. My main concern, however, is not assessment itself. The real challenge is how we teach and support students so they can use AI effectively, critically, ethically and responsibly to achieve the best possible results in their work and learning processes.

I have developed a set of prompts that assist students with the workflow of a research paper, giving inspiration in the areas of topics and research questions, getting them to generate content with AI and then use AI to help critique and improve on the ideas and analysis. Students so far seem to think that it’s the best of both worlds. They do produce better work but they feel they’re in control and are learning by going through the process. It’s a work in progress but I would be happy to share the prompts with anyone interested

John Reeks I have mixed feeling on the impact on education. I have used LLMs to teach myself a lot of new things. It is like having an unreliable teacher that injects random errors in what they teach. And in some tasks it can be prompted in a way it simplifies human cross-checking. At the same time when an undergrad student confesses they have not read anything since chatgpt came out, or when it is obvious that some PhD students are not progressing due to it, the danger is obvious. We need a new pedagogy. Banning and monitoring in various forms will be part of it, but also dedicated technology developed in the university (those that have the money are already doing it), maybe hardware based solutions (like the monitoring laptop I described), and teaching how to use it properly. I have more problems with the ethics of using commercial models. I want the university to have ethical locally installed AI we can use without exploiting workers, damaging climate, and contributing to a political economics system that is abhorrent. I think we can figure out the pedagogy, if the ethics is fixed. But Covid did a number on the tech autonomy of our universities that are now all captured by microsoft or google. So unclear the UK can drive it.

Robert Studholme in my cloister/starship class I do exactly that after forcing them to read in class and argmapping in groups and restricting the domain to only a few papers they can use. And I agree that for some students that play along seem to work well and they quickly realise that good AI work is more time consuming than the old school approach. Because the helpful bit is adversarial mode and it basically asks the student to do an extra layer of work. But I have the nagging feeling that in my class they do it, in other classes they adapt it to become an anti-detection strategy. I feel a bit guilty at times I might be enabling significantly more advanced cheating strategies. It is also confusing the Academic Integrity process because a student can play dumb and say that the different confusing practices around AI, some people banning it, some people actively encouraging within guardrails like we experimented, has generated whatever cheating they are being accused off.

William Waites I am really currious to hear from the Academic Integrity officers that will work on this wave of assessments and dissertations because the agentic revoluction, claude cowork/code, and OpenAI codex have diffused very recently and they offer a new level of options that are starting to percolate down to students. It is probably a minority of students using them effectively, but I found a few in my narrow sample of dissertations, if you extrapolate from that it should be around 20-30%. And they generate a new level of complexity in academic integrity forensic, as I explained in other replies, they can be used to avoid mechanical fake references, but concept stretching and erronous content hallucination requires manual checking that might slip through, and might actually be harder for the student to spot even if they read the source because they are trusting the AI so much thet they might end up missunderstanding the source.