Meta’s latest VR headset prototypes could help it pass the ‘Visual Turing test’

Meta wants to make it clear it’s not giving up on high-end VR experiences yet. So, in a rare move, the company is spilling the beans on several VR headset prototypes at once. The goal, according to CEO Mark Zuckerberg, is to eventually craft something that could pass the “visual Turing Test,” or the point where virtual reality is practically indistinguishable from the real world. That’s the Holy Grail for VR enthusiasts, but for Meta’s critics, it’s another troubling sign that the company wants to own reality (even if Zuckerberg says he doesn’t want to completely own the metaverse).

As explained by Zuckerberg and Michael Abrash, Chief Scientist of Meta’s Reality Labs, creating the perfect VR headset involves perfecting four basic concepts. First, they need to reach a high resolution so you can have 20/20 VR vision (with no need for prescription glasses). Additionally, headsets need variable focal depth and eye tracking, so you can easily focus on nearby and far away objects; as well as fix optical distortions inherent in current lenses. (We’ve seen this tech in the Half Dome prototypes.) Finally, Meta needs to bring HDR, or high dynamic range, into headsets to deliver more realistic brightness, shadows and color depth. More so than resolution, HDR is a major reason why modern TVs and computer monitors look better than LCDs from a decade ago.

Meta Reality Labs VR headset prototypes


And of course, the company needs to wrap all of these concepts into a headset that’s light and easy to wear. In 2020, Facebook Reality Labs showed off a pair of concept VR glasses using holographic lenses , which looked like over-sized sunglasses. Building on that original concept, the company revealed Holocake 2 today (above), its thinnest VR headset yet. It looks more traditional than the original pair, but notably Zuckerberg says it’s a fully functional prototype that can play any VR game while tethered to a PC.

“Displays that match the full capacity of human vision are going to unlock some really important things,” Zuckerberg said in a media briefing. “The first is a realistic sense of presence, and that’s the feeling of being with someone or in some place as if you’re physically there. And given our focus on helping people connect, you can see why this is such a big deal.” He described testing photorealistic avatars in a mixed reality environment, where his VR companion looked like it was standing right beside him. While “presence” may seem like an esoteric term these days, it’s easier to understand once headsets can realistically connect you to remote friends, family and colleagues.

Meta’s upcoming Cambria headset appears to be a small step towards achieving true VR presence, the brief glimpses we’ve seen at its technology makes it seem like a small upgrade from the Oculus Quest 2. While admitting the perfect headset is far off, Zuckerberg showed off prototypes that demonstrated how much progress Meta’s Reality Labs has made so far.

Meta Reality Labs VR headset prototypes


There’s “Butterscotch” (above), which can display near retinal resolution, allowing you to read the bottom line of an eye test in VR. To achieve that, the Reality Labs engineers had to cut the Quest 2’s field of view in half, a compromise that definitely wouldn’t work in a finished product. The Starburst HDR prototype looks even wilder: It’s a bundle of wires, fans and other electronics that can produce up to 20,000 nits of brightness. That’s a huge leap from the Quest 2’s 100 nits, and it’s even leagues ahead of super-bright Mini-LED displays we’re seeing today. (My eyes are watering at the thought of putting that much light close to my face.) Starburst is too large and unwieldy to strap onto your head, so researchers have to peer into it like a pair of binoculars.

Meta Mirror Lake VR concept


While the Holocake 2 appears to be Meta’s most polished prototype yet, it doesn’t include all of the technology the company is currently testing. That’s the goal of the Mirror Lake concept (above), which will offer holographic lenses, HDR, mechanical varifocal lenses and eye tracking. There’s no working model yet, but it’s a decent glimpse at what Meta is aiming for several years down the road. It looks like a pair of high-tech ski goggles, and it’ll be powered by LCD displays with laser backlights. The company is also developing a way to show your eyes and facial expressions to outside observers with an external display on the front.

Repost: Original Source and Author Link


Sonantic uses AI to infuse emotion in automated speech for game prototypes

Join Transform 2021 for the most important themes in enterprise AI & Data. Learn more.

Sonantic has figured out how to use AI to turn written words into spoken dialogue in a script, and it can infuse those words with the proper emotion.

And it turns out this is a pretty good way to prototype the audio storytelling in triple-A video games. That’s why the Sonantic technology is finding use with 200 different video game companies for audio engineering.

The AI can provide true emotional depth to the words, conveying complex human emotions from fear and sadness to joy and surprise. The breakthrough advancement revolutionizes audio engineering capabilities for gaming and film studios, culminating in hyper-realistic, emotionally expressive and controllable artificial voices.

“Our first pilots were for triple-A companies, and then when we started building this,” said cofounder Zeena Qureshi in an interview with GamesBeat. “We went a lot more vertical and deeper into just working very closely with these types of partners. And what we found is the highest quality bar is for these studios. And so it’s really helped us bring our technology into a very great place.”

Building upon the existing framework of text-to-speech, London-based Sonantic’s approach is what differentiates a standard robotic voice from one that sounds genuinely human. Creating that “believability” factor is at the core of Sonantic’s voice platform, which captures the nuances of the human voice.

Obsidian Entertainment audio director Justin Bell said in a video that the tech will enable game companies such as his own to cut production timelines and costs with this new capability. Bell said that his team could send a script through Sonantic’s application programming interface (API) and then get something back that isn’t just robotic dialogue. It comes back as a real human conversation, and Bell said that could empower the team to tell a better story.

Above: Zeena Qureshi and John Flynn are the founders of Sonantic.

Image Credit: Sonantic

“It’s just really useful hearing something back very early in the process,” Qureshi said.

You could simply use these scripts and the voices generated to populate dialogue into the non-player characters of a game. But the point of this isn’t to put voice actors out of work, Qureshi said. Rather, it gives a readable, reviewable script to the creators much earlier in the creative process so that they can listen to the dialogue and change it much earlier in the process if it clearly doesn’t sound right, she said.

In order to demonstrate its voice-on-demand technology, Sonantic has released a demo video highlighting its partnership with Obsidian, maker of The Outer Worlds and a subsidiary of Microsoft’s Xbox Game Studios. Others using Sonantic include Splash Damage and Sumo Digital.

Sonantic partners with experienced actors to create voice models. Clients can choose from existing voice models or work with Sonantic to build custom voices for unique characters. Project scripts are then uploaded to Sonantic’s platform, where a client’s audio team can choose from a variety of high-fidelity speech synthesis options including pitch, pacing, projection, and an array of emotions.

Above: Sonantic’s tool helps audio engineers make better games and films.

Image Credit: Sonantic

Film and game studios are not the only beneficiaries of Sonantic’s platform. Actors can maximize both their time and talent by turning their voices into a scalable asset, as the Sonantic technology takes their voices and uses them to create different variations. Sonantic’s revenue share model empowers actors to generate passive income every time his or her voice model is used for a client’s project, spanning development, preproduction, production, and post-production.

“This technology isn’t made to replace actors,” Qureshi said. “What it actually helps with is at the very beginning of game development. Triple-A games can take up to 10 years to make. But they typically get in actors at the very early stages, because they’re constantly iterating. So they use text-to-speech that’s been an industry standard for the last few decades. But we’ve created a way that helps actors work virtually as well as in person. And it helps studios get voices into their game, highly realistic voices into their game from the very beginning to help them feel out the story arc, fill out the pacing, really understand what needs to change, so that their iteration cycles can continue to go really fast.”

Sonantic’s official launch follows last year’s beta release, which was captured in a video entitled Faith: The First AI That Can Cry.

The result is a streamlined production process. Teams won’t have to call back actors for reshoots or engage in re-edits of voices as much.

“Some of our studios have told us they save a week of time for their team every month,” Qureshi said.

An accelerator meeting

Qureshi met cofounder John Flynn in 2018. He had a great demo of the technology, and Qureshi had a background in speech and language therapy.

“When I heard his demo, I was like, ‘This is insane!’” Qureshi said. “It sounded better than any text-to-speech I’ve ever heard. And then he told me how he did it. And I thought, ‘This is exactly how I teach children.’”

Before that demo, all the speech-to-text algorithms Qureshi had heard flattened the delivery of the performance, so that it sounded robotic.

“The technology before didn’t captures the highs and lows of the voice,” Flynn said. “I changed it to make it work better by looking for those highs and lows and trying to like get the algorithm to focus on that more.”

Qureshi added, “The devil is in the details with communication. There are so many different ways to say something. So when I’m teaching a child, I have to teach them emotions. I have to teach them how to enunciate very clearly, how to project their voice, really use their voice as an instrument, and control it.”

Flynn said that most of the work of the past few years is to get models to do the same as what Qureshi could do with kids.

“Last year, we had the AI that could cry, with emotion and sadness,” Flynn said. “It’s really about the nuances in speech, that quiver of the voice for sadness, an exertion for anger. We try and model those really deeply. Once you add in those details and layer them on top, you start to get energy and it becomes really realistic.”

Besides games, Sonanctic works for films and TV production. The company has 12 employees, and it has raised $3.5 million to date from investors including AME Cloud Ventures, EQT Ventures, and Krafton Ventures.


GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Repost: Original Source and Author Link