Neon’s super realistic digital people are real. Well, sort of.
The mysterious company, emerging from the Samsung Technology and Advanced Research Labs (aka STAR Labs), debuted late Monday at CES 2020 here in Las Vegas. It described its technology, also called Neon, as “a computationally created virtual being that looks and behaves like a real human, with the ability to show emotions and intelligence.”
Basically, Neon makes video chatbots that look and act like real people. Neons aren’t all-knowing smart assistants, androids, surrogates or copies of real humans, the company said in an FAQ shared with reporters. They can’t tell you the weather or how old Abraham Lincoln was when he died.
Instead, they’re designed to have conversations and behave like real humans. They form memories and learn new skills but don’t have a physical embodiment, at least not now. Neons can help with “goal-oriented tasks or can be personalized to assist in tasks that require human touch.” They can act as teachers, financial advisers, health care providers, concierges, actors, spokespeople or TV anchors.
While they can borrow traits of real people and have a similar look and voice, they can’t be exact copies of existing humans, Neon said. And each Neon is unique, with its own personality.
“There are millions of species on our planet, and we hope to add one more,” Pranav Mistry, Neon CEO and head of STAR Labs, said in a press release. “Neons will be our friends, collaborators and companions, continually learning, evolving and forming memories from their interactions.”
Before CES 2020 even started, Neon had generated some of the biggest buzz for the world’s biggest tech show. That came despite — or maybe because of — the fact that no one really knew what it was. Neon tweeted out several teasers after launching its social media accounts in mid-December, hinting at something it called “an ‘Artificial.'” The only concrete fact known about Neon was that it was run by Mistry, the longtime Samsung research exec who in October was named CEO of STAR Labs.
While Neon’s realistic digital humans could become companions for some people, they also bring up questions about whether they could be too realistic. Neon’s technology appears to be flirting with the “uncanny valley,” the sensation of disgust some people experience when they encounter something that looks very similar to, but isn’t exactly, a person.
An example of something couldn’t bridge the uncanny valley was the 2004 movie The Polar Express. The children’s film featured an all-animated cast of characters that looked somewhat lifelike — but not lifelike enough that they’d be confused for real people. Instead, that realism backfired and caused some viewers to feel uncomfortable.
Neon’s hype
On Saturday, Mistry tweeted out two photos of what appeared to be an avatar (or “artificial human”) that he called “CORE R3.” And unlisted videos, spotted on Reddit and compiled into a video by the Good Content tech page on YouTube, show various other humanlike avatars that look a lot like real people.
“It can now autonomously create new expressions, new movements, new dialog (even in Hindi), completely different from the original captured data,” Mistry tweeted.
The hints caused people to speculate on what Neon could actually be. Could it be a replacement for Samsung’s Bixby smart digital assistant? (Neon shot that theory down quickly.) Would it show up on Samsung devices? Does it mean we’d soon be living in a real-life version of HBO’s Westworld?
While Neon is funded by Samsung and is part of Samsung’s STAR Labs, its artificial humans are not a Samsung product. It also isn’t related to Bixby, Samsung’s digital assistant that first appeared in 2017’s Galaxy S8. STAR Labs describes itself as a “fully independent future factory with a mission to bring science fiction to reality.”
As companies like Google, Amazon and, yes, Samsung have discovered, the key to actually making smart devices useful is packing in artificial intelligence, typically in the form of voice assistants. Every tech heavyweight is investing in these assistants because they’re heralded as the future of how we’ll interact with our gadgets. The ultimate promise for the smart technology is to predict what you want before you even ask — or make you forget you’re not interacting with a real human.
Mistry said last month in an interview with Indian business news publication Mint that AI “has many years of development to go before science fiction becomes reality,” hinting that Neon’s technology wouldn’t be immediately available.
“While films may disrupt our sense of reality, ‘virtual humans’ or ‘digital humans’ will be reality,” he told Mint in late December. “A digital human could extend its role to become a part of our everyday lives: a virtual news anchor, virtual receptionist or even an AI-generated film star.”
The three R’s
The Neon artificial humans are powered by two pieces of Neon proprietary technology. The first, called Core R3, stands for “reality, realtime and responsive.” That’s what makes the Neons respond quickly and in a lifelike manner. The second is called Spectra, which is responsible for intelligence, learning, emotions and memory.
Neon describes Core R3 as an advance in the “domains of behavioral neural networks, evolutionary generative intelligence and computational reality.” The company said it’s “inspired by the rhythmic complexities of nature and extensively trained with how humans look, behave and interact.” The latency of Core R3, or how long it takes to respond to queries, is less than a few milliseconds, letting the Neons react and respond in real time.
“CORE R3 can computationally create lifelike reality that is beyond normal perception to distinguish,” Neon said in its FAQ.
Spectra, meanwhile, is still in the development stage. The company said it will preview the technology at something it called Neonworld 2020 later this year. It didn’t say when or where that would take place.
It’s also working on perfecting its Core R3 engine.
Privacy and security
The Neons’ lifelike appearance raises questions about whether the technology could be used for deepfakes, or videos manipulated to show real people doing or saying something they didn’t. Neon said that while an avatar may be based on the appearance of a real person, the technology behind the Neons is “fundamentally different from deepfake or other facial reanimation techniques.”
“CORE R3 does not manipulate an individual scene, videos or sequences,” the company said. “Instead, it creates unique behaviors and interactions of Neons in real time that never happened before. Thus, CORE R3 creates new realities.”
Neon also said it designed Core R3 with ethical considerations in mind related to privacy and trust. It secures data by using “state-of-the-art security protocols.” The only people with access to a person’s interaction with a Neon are that person and the Neon. Neons also never share private data without permission, the company said.
“By design, privacy is at a Neon’s core,” it said. “We truly believe the wonders of technology must not compromise our privacy.”
The company plans to beta-launch Neon later this year with certain partners around the world. It said that people will be able to license or subscribe to Neons for expert help — asking them for financial advice, for example. It expects them to one day appear in movies and on the news.
“We have always dreamed of such virtual beings in science fictions and movies,” Mistry said in a press release. “Neons will integrate with our world and serve as new links to a better future, a world where ‘humans are humans’ and ‘machines are humane.”
Neon is demoing its technology this week at CES. It said it plans to show Neons “from many walks of life,” including a yoga instructor, banker, K-pop star, news anchor and fashion model. Visitors to its booth will be able to see live demonstrations and watch the Neons respond in real time, as well as meet and interact with Neons.
