How much does ChatGPT know about finance?

A topic of interest in many fields revolves around the presence of artificial intelligence (AI). With many people questioning the possibility of AI automating jobs, there is an ever-growing list of unknowns as this new technology evolves. Dan Greene, Bill Short Professor of Finance at the Wilbur O. and Ann Powers College of Business, noticed a rising concern in college students about the possibility of AI limiting the number of jobs available when they enter the workforce. In response to this concern, Greene combined his research with teaching strategies to create an innovative class assignment.

How much does ChatGPT know about finance?

His idea for his finance class originated from his co-authored study, “How Much Does ChatGPT Know About Finance?” In this study, forthcoming in Financial Analysts Journal, Greene and Douglas (DJ) Fairhurst from Washington State University explored large language models (LLMs), like ChatGPT, and their effectiveness in potentially replacing finance jobs.

“There’s a lot of talk in the industry about ChatGPT replacing jobs, but because the technology is so new, it’s hard to measure that. As we explored [ChatGPT], we found these tools aren’t quite as advanced as some make them out to be. In finance, there’s still a real need to build knowledge firsthand. Tools like this can support our work, but, as they are now, cannot replace the knowledge that comes from professional experience.”

Experiential learning in the classroom

Taking what he learned from his research, Greene decided to implement a similar assignment in one of his finance classes. The goal was to see if students found large language models (LLMs) to be a suitable replacement for a human in the role of a financial advisor.

He organized the unit into three sections. The first part was a skills test. For the skill test, students asked ChatGPT basic financial questions to rate their effectiveness on a three-point scale compared to that of a professional financial advisor. The second section asked students to test whether ChatGPT could fully replace a human by having the student assume the role of a client and ask ChatGPT to provide personalized financial advice based on a client profile. The third and final section asked students to act as financial advisors and use ChatGPT as a supplement rather than relying solely on it.

At the end of the unit, most students came to the conclusion that LLMS, like ChatGPT, are currently not sufficient enough to replace an experienced financial advisor. Nevertheless, the assignment provided students with exposure to an AI tool that many had not explored prior.

 “At the beginning of the unit, about half of my students had never used ChatGPT before. LLMs are going to be around in the future, and they are something students will encounter in their professional careers. Getting exposure to it now and seeing the ways in which it can be used as a tool will aid them as they enter the workforce,” explains Greene.

2024 Innovation in Teaching Award

For his finance lesson, Greene was recently awarded the 2024 Innovation in Teaching Award by the Financial Management Association (FMA) sponsored by the CFA Institute. The Innovation in Teaching Award recognizes educators who use innovative practices in the classroom to engage students and teach them relevant industry skills in a unique way.

“As business faculty, our role is to prepare students for the rapidly changing workplace, even as we ourselves are still discovering how to best utilize AI within our own expertise areas,” states Jennifer Siemens, Associate Dean for Faculty Excellence. “Dr. Greene’s research contributes to the understanding of AI’s impact on finance, while his innovative teaching empowers business graduates with confidence and readiness for success.”