AI Battles – Spring 2023 Newsletter 2

Smeal Academy Spring 2023 Connections Newsletter 2 – Artificial Intelligence and Academy Integrity: The Battle of the AIs

The goal of the Smeal Academy is to equip the Smeal Community with best practices and strategies for teaching and learning with technology to support Smeal’s business education and research needs. This newsletter provides resources, tips, and pointers as we move through the semester.

(Meme created by an AI Meme Generator, Supermeme.ai)

In recent years, there has been a growing concern about the use of artificial intelligence tools, such as ChatGPT, in academic settings. These tools, which can generate large amounts of text that is difficult to distinguish from human-written text, can be used to cheat on assignments and exams. However, there are steps that educators can take to detect and prevent cheating, such as using plagiarism detection software and having a clear policy in place regarding the use of AI-assisted tools. It is important to use AI responsibly and to be aware of its limitations, particularly when it comes to generating content that will be distributed to a wide audience.

Did that introductory paragraph make sense? If so, we must confess that we cheated. We “wrote” it using ChatGPT. And if you have been paying attention to the headlines, you know that ChatGPT and other artificial intelligence tools make it much easier to cheat on written assignments. The eLDIG team has been experimenting with AI tools and evaluating their impact for Smeal. The subject is so important that we have scheduled a Smeal Academy session devoted to the topic on March 23rd at 12:00. But given headlines like ChatGPT passes MBA exam given by a Wharton professor, we wanted to provide quick tips to help you in the classroom right now.

The tools and their outputs are getting stronger and better in measurements of days and weeks (not months or years), so it is imperative that we wrap our human heads around their AI capabilities. (And rest assured, except for the first paragraph, all text in this newsletter was written by an actual human being.)

5 things to do now about AI Tools like ChatGPT

• Don’t panic. As with anything, results may vary. Our experimentation has shown some ChatGPT results to be laughable and easily spotted. Some results are decent at first glance but wouldn’t pass any basic grading process. The problem is that some results are very high quality and could fool many of us. If you are concerned that AI tools could jeopardize the academic integrity of your assessments, brainstorm with the Instructional Designers at eLDIG (eldig@smeal.psu.edu) to come up with alternatives that might be more AI-proof. Sometimes subtle changes make a big difference.
• Add a Syllabus Statement. Put a clear statement on your syllabus (or make an announcement to the class) about the use of AI tools and ChatGPT in your course. Formal language is in the works at both the Smeal and Penn State University levels and will be available soon. For now, make sure your students know if AI use constitutes a violation of academic integrity in your course. What are your policies and what are the possible outcomes if you suspect AI use? And if AI use is permitted in limited or creative ways, how should the resulting text be cited?
• Test it out. Get an account and play around with ChatGPT (or other tools) and your assignments. What do they do well in your subject area and assignments? What do they do poorly? If you prompt ChatGPT with one of your writing assignments, what does it typically return? You will be more able to spot possible AI use by students if you are familiar with the range of outcomes that ChatGPT provides. Also, one thing that we realized when testing ChatGPT was that it cannot provide current citations, however, it sometimes provides old (or fake?) ones. A quick Google search showed that most links or sources provided came up as 404 Not Found Errors. And ChatGPT did give the caveat that some sources might not be real, although they appeared very convincing. So, check student sources too!
• Talk with students. Have a frank discussion with your students about the tools. Are they using them? What do they think? Have they seen limitations? Students are often our best experts in new technologies and their opinions may surprise you. And don’t think writing is the only subject impacted. It is likely that your students are seeing (and using?) AI tools for image generation, code creation, and multiple other uses that impact their coursework.
• Do some research. Read up on how to spot student use of AI and emerging tools to help. Here are some good articles and places to start:
  • AI Writing Detection: A Losing Battle Worth Fighting
  • ChatGPT Advice Academics Can Use Now
  • GPTZero (AI Detector)
  • Sneak preview of Turnitin’s AI writing and ChatGPT detection capability

Quick Ways to Make AI Less Attractive to Students

• Focus on current events if possible. For now, ChatGPT uses data from 2021 and before. It does not tackle current events well or even at all.
• Flip the script and think about embracing ChatGPT. For example, you could give students a writing assignment and tell them to use ChatGPT. Their assignment is to then grade the writing product and pick out factual errors, biases, or problems with the final product. Ask them how they would improve upon ChatGPT’s output.
• Focus writing assignments on an iterative process rather than the end result. Make students show evidence of their work along the way including research notes, outlines, and rough drafts. At the end, ask them to write a personal reflection to sum up what they learned during the process. Or throw students curveballs halfway through the writing process. Ask them to integrate citations or references that aren’t well-represented in current data on that subject.
• Ditch the tech. If class size allows, you might ask students to perform writing tasks in person without technology. Blue Books, anyone? Alternatively, ask students to write an outline for a writing assignment during class (without notes or technology) and turn it in. After you have graded the outline, the student can write the essay using technology, but only using that outline as the basis. It is much more difficult to force ChatGPT to write to a specific script or outline.
• Oral Exams. Reserve the right to give a student an oral exam to evaluate their work if you suspect unapproved AI use. Ask students about specific terms used in their work and why they chose a certain citation or angle.
• Security through obscurity. Get bogged down in the details (in a good way). Ask students to focus their writing assignments for very specific audiences. Or use fictitious company names or your own original case studies as background for questions. The more obscure the details or background, the less chance the answers that ChatGPT provides will make sense.

It is a brave new world out there, and we all know that various technologies have disrupted the educational process before. Calculators, spell check, the computer, and even the blackboard were once thought to be the end of education as we know it. But as we have witnessed, the tools became integrated in the learning process, and we have all managed to keep teaching and learning with them. So, back to the first point: Don’t panic.

The eLDIG team is here for the Smeal community. If you want to discuss an idea or need for your course(s), please don’t hesitate to contact us to schedule a consultation at eldig@smeal.psu.edu.

Upcoming Live Sessions

eLDIG ID Tips

Contact Us