Robots and vehicles could get better vision in the wild with the help of Clemson University researchers

Clemson University’s Nianyi Li is leading a new project to create 3D vision systems that help robots and autonomous vehicles navigate forests, wetlands, sandhills and other off-road terrain more safely and effectively.

The research is funded with a CAREER award from the National Science Foundation, with Li serving as principal investigator.

Li and her team are focused on helping autonomous systems “see” and navigate through “unstructured environments.” They include areas that are natural or unpredictable settings, such as off-road terrain and disaster zones that lack the regular, straight-line features and consistent lighting of indoor or urban spaces.

The Li team’s work could help improve technologies involved in search-and-rescue missions, environmental monitoring and conserving natural resources.

Li aims to fill a critical gap in 3D vision research for unstructured environments, which are far less studied than indoor and urban settings.

To address this, the Li team will build advanced multi-camera and sensor systems combining RGB cameras, depth sensors, LiDAR and event cameras that excel in changing light.

The systems will collect data in Clemson’s Experimental Forest and other South Carolina landscapes. The Li team will then use the data to train algorithms to handle challenges such as irregular terrain and changes in lighting.

Her group is also developing deep-learning techniques for reconstructing 3D scenes and tracking moving objects, even when they get temporarily hidden or distorted.

“This project will significantly advance 3D vision technologies for autonomous systems operating in challenging, unstructured environments,” Li said. “Researchers and industries working on autonomous navigation, environmental monitoring, and search-and-rescue applications will be particularly interested in these outcomes. Ultimately, this work will enhance the adaptability and reliability of self-driving vehicles, enabling their safe and efficient operation in complex real-world conditions.”

The project also has an educational mission. Li will involve students from high school to graduate school, giving them hands-on experience in building and testing the technology.

High school students will take part through Clemson’s summer research programs, and undergraduates and graduate students will help develop algorithms, run field experiments and present results.

The work will be multidisciplinary, Li said.

“These projects combine hardware and software, so we can bring together students with different backgrounds to work collaboratively in this program,” she said.

Among those congratulating Li on the grant was Brian Dean, director of the C. Tycho Howle School of Computing.

“This well-deserved NSF CAREER award puts Dr. Li on the radar as one of the nation’s fast-rising leaders in her field,” he said. “Her innovative research and commitment to student engagement exemplify the kind of impact that define Clemson’s role in shaping the future of higher education and the workforce alike.”