Seeing Through Joyce’s Eyes
If you want to eat an elephant, conventional advice says to start with the first bite. To build a robot equipped with superhuman vision, ImmerVision says to start with the head.
Meet Joyce, a humanoid robot that has been under development for several months. Philosophically speaking, however, said Alessandro Gasparini, executive vice president of operations and chief commercial officer at ImmerVision, the project has been ongoing for 20 years.
A concept image of the Joyce robot. Courtesy of ImmerVision.
The project is unfinished, intentionally so, and was unveiled at the Embedded Vision Summit taking place virtually Sept. 15-25. Currently, the robot consists of a head equipped with 360° × 360° cameras, data-in-picture technology for synchronizing data, image processing algorithms, and open APIs.
The last detail is perhaps the most crucial to the project.
“The core of this is that basically the community can work together to achieve an industry goal — to bring the sense of perception to a robot,” said Alain Paquin, head of the Joyce project.
ImmerVision will be crowdsourcing technologies through community challenges to add enhanced capabilities and understandings to the robot. Joyce is the spiritual successor to a development kit that Gasparini had been working on for a couple of years to allow for greater collaboration within the computer vision community.
“There are a number of components on the market, there are a number of ways of integrating them, there are a number of algorithms, there are a number of everything. And we realize that ‘Okay, our expertise, it’s up to a certain level,’” Gasparini said.
Paquin used the anecdote of running a Roomba vacuum cleaner.
“The time that I can spend with a glass of wine in my hand watching [the vacuum cleaner] go around, it’s really impressive,” he said. “And at one point it frustrates me, because looking at it, I said, okay, they have all kinds of technology — they have this, they have that. Why is the robot bumping his head four times against the same spot in the wall?”
Paquin theorized that — noting the limits of a single company or a single group of people working on a project — a collaboration between various groups could yield a more intelligent robot, leading to the essential idea behind the Joyce project.
Gasparini said, “That’s the goal of Joyce. It’s really to open up a platform, a humanoid platform, a robot platform. That, instead of everything being developed by a closed system of engineers at ImmerVision, it’s to open it up to the whole community, to give a chance to smart people — whether they are a startup, a larger organization, a university — to bring some of their innovations in terms of technologies, sensing, to the system to enhance [Joyce's] abilities, to be able then to provide those features, like distance measurements, as a perception feature of the robot.”
In the near term, Joyce will be featured at industry events, where people will be able to log in with smart devices and see in 360° through the robot's eyes. The company has plans to bring Joyce to various locations, such as sporting events, concerts, road trips, the beach, helicopter tours, and even on skydiving trips. People will be able to log in for an immersive experience, to see and hear what Joyce experiences. Paquin said a virtual reality headset would make it especially immersive.
In the long term, as ImmerVision’s challenges arise, the robot will become more advanced. Features such as facial recognition, 3D perception, motion sensing, night vision, movement detection, and more will be added.
This innovative purpose is the inspiration for the community challenges. "Companies can then use the outputs of what they have built for Joyce to build [their own] devices,” Paquin said.
The level of integration required to build true robot perception will necessarily yield more advanced technologies. ImmerVision forecasts that the challenges will lead to improved technology for numerous applications, such as automotive, robotics, IoT, machine vision and learning, wearables, medical, drones, and more.
Eventually, the robot will be able to walk. But for now, the focus is on collaboration toward a singular goal — to bring computer vision beyond current possibilities.
“We would like, through this project, to push beyond the abilities that ImmerVision has or what our competencies are, but to be able to really ignite the ability to push the development of the device — a robot, a brain, a system that will be able to perceive like a human and beyond. And that, I believe, can only be done by a collectivity. It cannot be done by just one company,” Paquin said.
/Buyers_Guide/Immervision/c18759
Published: September 2020
Glossary
- machine vision
- Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data.
Key aspects of machine vision include:
Image acquisition: Machine vision systems use various...
- computer vision
- Computer vision enables computers to interpret and make decisions based on visual data, such as images and videos. It involves the development of algorithms, techniques, and systems that enable machines to gain an understanding of the visual world, similar to how humans perceive and interpret visual information.
Key aspects and tasks within computer vision include:
Image recognition: Identifying and categorizing objects, scenes, or patterns within images. This involves training algorithms...
- artificial intelligence
- The ability of a machine to perform certain complex functions normally associated with human intelligence, such as judgment, pattern recognition, understanding, learning, planning, and problem solving.
- machine learning
- Machine learning (ML) is a subset of artificial intelligence (AI) that focuses on the development of algorithms and statistical models that enable computers to improve their performance on a specific task through experience or training. Instead of being explicitly programmed to perform a task, a machine learning system learns from data and examples.
The primary goal of machine learning is to develop models that can generalize patterns from data and make predictions or decisions without being...
- embedded vision
- Embedded vision refers to the integration of computer vision technologies into various embedded systems, devices, or machines. Computer vision involves teaching machines to interpret and understand visual information from the world, much like human vision. Embedded vision takes this concept and applies it to systems where the processing occurs locally within the device, as opposed to relying on external servers or cloud-based services.
Key components of embedded vision systems include:
...
Businessmachine visionImmerVisionrobotcomputer visionperceptionsighthuman visionhuman vision perceptiontechnologyartificial intelligencemachine learningroboticscameraslensesEmbedded Vision