« Back to Results

Cheating in the Era of ChatGPT: Implications for Economics Instruction and Assessment

Paper Session

Friday, Jan. 5, 2024 8:00 AM - 10:00 AM (CST)

Grand Hyatt, Lone Star Ballroom Salon C
Hosted By: American Economic Association & Committee on Economic Education
  • Chair: Darshak Patel, University of Kentucky

Do Human Proctors Reduce Cheating in Open Book Online Exams: A Field Experiment?

Jose J. Vazquez
,
University of Illinois-Urbana-Champaign
Eric P. Chiang
,
Florida Atlantic University
Ignacio Martin Sarmiento Barbieri
,
University of the Andes

Abstract

As online course offerings become increasingly more common in institutions of higher learning, instructors are rapidly shifting from paper and pencil assessments to online assessments. Online assessments offer several key advantages, including reduced administrative costs, ability to use a wider variety of multimedia resources, and faster analysis of results.

However, online assessments create a challenge due to the potential for academic dishonesty, which may discourage instructors from implementing them. Although anecdotal evidence supports the notion that cheating is pervasive in online exams, the existing literature on cheating has focused mostly on exams conducted in a classroom than in an online setting.

This paper analyzes potential cheating behavior in online exams using experimental data. We conducted randomized controlled trials to estimate these effects in large enrollment Microeconomic Principles courses at a U.S. public university. Students took multiple open-book online exams during the course; however, only one (determined randomly) was proctored.

Our analysis investigates the effect of proctoring in online exams by the type of class (face-to-face vs. fully online course), by level of achievement, and by gender. We explore explanations for the differentials in performance and how the use of technological innovations in proctoring can mitigate academic dishonesty.

ChatGPT has Mastered the Principles of Economics: Now What?

Wayne Geerling
,
University of Texas-Austin
G. Dirk Mateer
,
University of Texas-Austin
Jadrian Wooten
,
Virginia Polytechnic Institute and State University
Nikhil Damodaran
,
OP Jindal Global University

Abstract

The Test of Understanding in College Economics (TUCE) is a standardized test of economics knowledge performed in the United States which primarily targets principles-level understanding. In February 2023, we asked ChatGPT-3 to complete the TUCE. ChatGPT ranked in the 91st percentile for Microeconomics and the 99th percentile for Macroeconomics when compared to students who take the TUCE exam at the end of their principles course. The results show that ChatGPT is capable of providing answers that exceed the mean responses of students across all institutions. One reason for ChatGPT’s success on the TUCE is that the questions are strictly multiple choice with textbook “answers.” The emergence of artificial intelligence presents a significant challenge to traditional assessment methods in higher education. It is not possible to evaluate students’ intellectual ability through artificial intelligence, as it only reflects their ability to access information. An important implication of this finding is that educators will likely need to redesign their curriculum in at least one of the following three ways: reintroduce proctored, in-person assessments; augment learning with chatbots; and/or increase the prevalence of experiential learning projects that artificial intelligence struggles to replicate well.

Worried About ChatGPT? How to Use Instructional Design to Decrease Student Cheating

Michael Enz
,
Virginia Polytechnic Institute and State University

Abstract

ChatGPT is the latest technology that has instructors worried about cheating in their courses. One approach to addressing this concern is to create new methods of detecting the use of artificial intelligence services. However, this approach is always one step behind the creation of new technology. Instead of chasing each new technology, instructors can use instructional design to create a course that highly discourages cheating. Amigud and Lancaster (2019) identify five themes explaining why students seek unacceptable help in completing their academic work. These five themes include academic aptitude, perseverance, personal issues, competing objectives, and self-discipline. This paper provides a description of how an instructor can use instructional design to address these five common themes and decrease the incidences of cheating in a course.

An Economist’s Guide to Teaching Students About, and With, ChatGPT and Other Large Language Models

Sean Flynn
,
Scripps College

Abstract

The optimal strategies for teaching with ChatGPT and other large language models depend upon a basic understanding of how artificial neural networks operate, which I contribute in the first section of this paper. I then discuss teaching with ChatGPT, including its usually robust capabilities with respect to: grading student work and providing student feedback; its use as a private economics tutor; and its ability to be “interviewed” on behalf of historical or living people. Examples are then shown of the difficulties it has solving certain math and algebra problems that are relevant to economics pedagogy. I conclude with strategies for integrating ChatGPT into student study flows.

Discussant(s)
Colin Cannonier
,
Belmont University
Diego Mendez-Carbajo
,
Federal Reserve Bank of St. Louis
Janine Lynn Flathmann Wilson
,
University of California-Davis
Laura Ahlstrom
,
Oklahoma State University
JEL Classifications
  • A2 - Economic Education and Teaching of Economics