Yunyi Jia‘s automotive electronics class at Clemson University was nearing its end when he slipped an Apple Vision Pro onto his head to illustrate the inner workings of electric motors in cars.
About 40 students watched Jia wave his hands in the air, as if he were drawing on an invisible chalkboard. The screens behind Jia showed him marking up 3D models, the same view he had inside the headset.
“A current comes in this direction,” he said, and through the magic of modern technology, a green arrow appeared on the screen as he traced it in the air.
Welcome to the future of education.
Classroom technology has gone from chalkboards and film projectors to whiteboards and video screens, and now Jia and his students are at the vanguard of what he said will be another leap.
“This will change the world of teaching we have today,” said Jia, the McQueen Quattlebaum Professor of Automotive Engineering. “This also changes students’ interest level, helps keep their attention and, most importantly, helps deepen their understanding and knowledge.”
The app that Jia used in class was developed by Rongyao Wang, a third-year Ph.D. student in automotive engineering under Jia’s supervision.
“I think in the future classroom, the class will be more interactive, rather than just one instructor showing things and writing on the board,” Wang said. “In the future, this kind of headset will be more commonly used in teaching courses, because not only can you see 3D objects overlaid in the real world, but you also potentially allow the students to interact with these objects.”
In a few years, it just might be common to walk into a classroom and see all the students and the instructor wearing mixed-reality headsets, Wang said.
For now, mixed reality in the classroom is still more experimental than everyday practice, with most universities only beginning to try the technology.
The sort of mixed-reality lesson that Jia brought to the Clemson University International Center for Automotive Engineering is so far most commonly used for training in industry, he said. Licensing for the apps can be very expensive, making it too pricey for many educators, Jia said.
Jia and Wang are working to make the app used in the Clemson class more sophisticated and widely available.
Shantanu More, a first-year master’s student in automotive engineering, followed along with Jia in class, quietly tracing arrows in the air from his seat near the front of the room. It wasn’t the first time Jia had used mixed reality in the classroom, More said in an interview later.
“Last time he brought a whole Formula One car and disassembled it in front of us and told us where the sensors were located,” More said. “It was really cool, because having such a complicated machine like a Formula One car in the classroom is a concept that is ahead of the curve.”
Jia said that in his courses he often has to teach concepts involving 3D models.
“I can use language, I can use pictures or videos or slides, but that relies on students to imagine those kinds of different models and different situations,” Jia said. “It’s not intuitive– and then it came to my mind, how can we make it intuitive?”
The mixed-reality approach does that, he said.
Lillian DeGroot, a second-year master’s student in automotive engineering, said she has never before seen mixed-reality used in the classroom quite like Jia does.
“It’s really helpful to see things in more of a 3D aspect,” she said. “I think it’s helpful especially for visual learners.”
Jia’s lab has been working on human-vehicle and human-robot collaborations using mixed reality for years.
For example, recent work that enables humans to ride in an actual vehicle in a real-world environment while immersively interacting with virtual scenarios has won best paper awards from IEEE and the Transportation Research Board, part of the National Academies of Sciences, Engineering, and Medicine. The technology has also been granted a U.S. patent.
“My approach in the classroom is really an extension of the research we have been doing in the lab,” Jia said.
Robert Prucka, interim chair of the Department of Automotive Engineering, said Jia’s mixed-reality work exemplifies how the department is pushing teaching forward.
“Our goal is to create the No. 1 student experience in automotive engineering,” he said. “Innovations like this make complex concepts more accessible and prepare our students to lead in a rapidly changing industry.”
