SONY

Cooperative Innovations Ltd. enables Tate McRae to dance into a new dimension

Sony Group Corporation teams up with Sony Music artist, songwriter, and dancer Tate McRae, immersive technology studio Cooperative Innovations Ltd. and creative agency Ralph to create an interactive, 3D music video for the hit song “You Broke Me First” as part of its brand campaign, the Sony Collaboration Series.

  1. Connect with Tate McRae’s fans when a global pandemic has preempted live concerts
  2. Present Tate McRae’s amazing dance moves with unprecedented immediacy
  3. At a time when pandemic has kept us apart, dramatize isolation – and its transcendence
  1. Using the Spatial Reality Display, develop, prototype and deliver an interactive, 3D music video1,2
  2. Cooperative Innovations Ltd. and Ralph combines creative expertise with vast experience with Unreal Engine3 to advance the music video form

*1 Computer required with a recommended CPU of Intel Core i7-9700K @3.60GHz or faster; and a graphics card such as NVIDIA GeForce RTX 2070 SUPER or faster.

*2 See instruction manual accompanying the product for details on product use, incl. 3D feature.

*3 Recommend use of “high resolution, quality images” created using Unity or Unreal software.


  1. Demonstrates a new way of presenting and experiencing popular music
  2. Interactive content combining dance, music and 3D animation wizardry

Quotes:

“It feels like you’re looking into another world through some magic portal! You feel like you could reach out and touch it.”

- Simon Barratt, CEO and co-founder, Cooperative Innovations Ltd


“This is bonkers. I think music and technology go super hand-in-hand. But I’ve never really experienced it elevate to this level.”

- Tate McRae, Sony Music artist

At the height of the pandemic, Sony Music artist Tate McRae was social distancing. Since spending hours in her bedroom was not conducive to fan engagement, when Sony offered to create an interactive 3D music video, leveraging the company’s Spatial Reality Display, she seized the opportunity. Featuring the ballad “You Broke Me First,” the video starts with Tate alone in her bedroom, then transforms into an explosion of amazing dance, camera movement and atmospherics.

The task of making that vision a reality fell to Cooperative Innovations Ltd, a British studio specializing in VR/AR, 3D design, app and games development. According to Simon Barratt, CEO and co-founder, “We worked closely with Ralph and Tate to create the overall story and sequence of the music video. We started by prototyping visual effect ideas that would be appealing, feasible for rendering in real-time and also look good rendered out for the 2D video. To get the experience running in stereo 4K in real-time, we needed to do a lot of performance profiling and optimization work to ensure that the experience moved smoothly and it didn’t jar as users viewed the performance through the Spatial Reality Display.”

“While we have experience developing VR and AR content, this Spatial Reality Display is different,” says Barratt. “Combined with the head tracking, it feels like you’re looking into another world through some magic portal! It gives a sense of physicality to the content that makes it somehow feel like you could reach out and touch it more so than a current AR headset would. Using a headset 40 hours a week isn’t always possible, so a 3D display provides a great alternative.”

“As we were developing and iterating, we previewed elements on the Spatial Reality Display. The first element was our 3D mock-up of the bedroom. Looking into what we knew was going to be the start of the experience was really thrilling to see and got us all even more excited about the project.”


The music video plays in Unreal Engine 4 software3. Simon Barratt explains, “We’ve been using Unreal Engine for years. The Spatial Reality Display plug-in made it easy to get content up and running. We’re really impressed that it looks as good as it does and runs at high framerate with all the complex visual effects we have going on! We always talk about the ‘smoke and mirrors’ in projects like this – ironic as the video literally features smoke and mirrors!

“The main challenges were finding the right camera angles to make the best use of the 3D effect. While it’s awesome that the user can move their head around to really give their own unique perspective on the experience, it’s also a challenge when developing content. We checked it over and over for potential views that aren’t as compelling!”

Barratt envisions a bright future for Sony’s Spatial Reality Display. He says, “Previewing in 3D2 is vital for our clients in entertainment, automotive and architecture. While we love VR, it isn’t universal or comfortable for everyone. Sometimes it can be a pain to take a headset on and off. Having a 3D display sit on your desktop is really great. I’d absolutely recommend the use of the technology for viewing 3D assets not only during development but also for sharing with executives and creative directors.”