Capturing the Imagination: Computer Science Professors Lead Advances in Robotics, Virtual Reality

Two visionary computer science professors are redefining the educational landscape in the College of Engineering as they spearhead cutting-edge research and innovative projects, transforming their fields and inspiring a new generation of tech pioneers.

Professor John Seng is leading students in robotics projects that blend artificial intelligence with mechanics, while his colleague Professor Jonathan Ventura is opening doors to virtual reality, enabling a wider audience to craft immersive digital worlds.

As faculty members of The Noyce School of Applied Computing, Seng and Ventura go beyond teaching technology; they are actively shaping the innovators of tomorrow.

John Seng: Inspiring students in STEM

Seng is overseeing the construction of two robots – Herbie and Pascal – who are captivating the campus community, sparking interest and enthusiasm wherever they roll.

Herbie, an autonomous robot with googly eyes and a big smile, continues to evolve with help from Robotics Club members. Herbie was designed to roam around Building 14 and serve as a goodwill ambassador, bringing joy to those he encounters.

“When I build a robot, the point is not what it can do but for the response it evokes,” Seng said. “People think that it’s cool, which leads them to think of other opportunities for invention.”  

After an unfortunate incident last year in which Herbie was toppled by an overeager preschooler, a group of students built a magic caboose for the 3-year-old robot. The trailer provides stability and features a mounted screen with a digital spinner illustrated with various animals. A handheld remote activates the spinner and a series of animal sounds.

The device engages preschoolers with STEM concepts – at a safe distance.

Pascal, Herbie’s robotic counterpart, is another of Seng’s passion projects – an autonomous droid being crafted to mirror the iconic “Star Wars” character, R2-D2.

A team of Robotics Club students has completed Pascal’s wooden frame, added three wheeled legs and a metal dome and designed lights and sound effects including whistles, beeps and songs that play through car stereo speakers.

A milestone moment came last spring when Pascal drove for the first time.

“It was definitely a breakthrough for the club,” said Seng, showing how the droid rolls and spins while pushing buttons and paddles on a large remote control.

Next, Pascal is set for a paint job and more controls, with the aim of programming the droid for autonomous operation.

“Our department should have robots rolling around,” Seng said. “It’s a visible sign of what we are up to.”

Jonathan Ventura: Imagining the future

Ventura is elevating virtual reality creation to new heights at Cal Poly as he works to make VR development accessible to a wider audience.

By engaging students in his endeavor, Ventura is not just teaching VR techniques, he’s nurturing a new generation of creators ready to refine the boundaries of immersive digital experiences.

“It’s exciting to feel like we’re creating the future, and it’s motivating to work on something so cutting-edge and impactful,” he said.  

 Ventura collaborates with students in a small lab in Building 52, developing virtual reality experiences using readily available equipment accessible to the public.

For the project he calls PanoSynthVR, Ventura and a team of students took 2D panoramic image from a public dataset, then fed the data into a model to produce a 3D rendering.

The panorama offers a circular perspective but doesn’t allow the viewer to see around the corner of a building, for example, so Ventura and his students employed machine learning to infer what would be around that corner, filling in the visual gaps.

Ventura published a paper on the project in 2022 and is working on the next iteration to improve the clarity of the scene. He’s also spearheading a project focused on 3D image reconstruction, which is critical for creating convincing and engaging immersive environments.

A grant from the National Science Foundation is funding Ventura’s efforts, enabling him to stay abreast of the rapidly evolving fields of virtual reality and artificial intelligence.

“We have to stay on the cutting edge at Cal Poly,” he said. “Virtual reality and augmented reality are on the cutting edge, and there’s a huge demand for students with those skills.”

To read more about Seng and Ventura’s ongoing projects, visit here.