“With the streamlined workflow and world’s fastest VR and 360 renderer, artists can make high-quality VR quicker than ever before.”

Helsinki based electronic duo Phantom have a reputation as techies, even building their own instrument, an Ultrasonic Frequency Oscillator. Having experimented with Kinect cameras in past music videos, they wanted to explore VR for the release of Lost. With a brief to create an original, interactive VR video, Phantom reached out to 3D animator Zebracolor, otherwise known as Jani Isoranta Creative Director here at Notch.

Jani’s vision was to create a video in which you could feel fully immersed: “When you listen to music nowadays it’s often with many real-world distractions and song by song. You rarely get the chance to get ‘lost’ in the music in the way that people used to sit in their bedrooms listening to an album from start to finish. VR has a huge bonus in that it ‘locks you in’ – you can’t be distracted by your phone or TV while you have a headset on. It’s just you and the track.”

Lost was pitched from the start as a multi-format release. A traditional 2D music video; a VR/360 video – which is still relatively new but not unheard of nowadays; and something innovative: an interactive real-time experience for PC VR headsets. Jani knew that using Notch he would only have to create the content once and it could easily be retargeted to each format, “using Notch, we were able to create the Oculus experience and also render a YouTube 8k video at the flick of a switch. Without the need to create two separate iterations”.

“As it’s a music video we wanted to include the band themselves in the piece. Creating for VR adds another dimension so we felt traditional 2D video would be too flat in the 3D world.” Jani filmed the band volumetrically using a Kinect 2 sensor to get depth information. “Notch’s built-in Kinect recording and playback nodes made recording, editing and using the footage simple.”

Jani chose Oculus Rift as the VR platform, this decision heavily influenced his design process. With the Oculus Rift, the content would be experienced sat at a desk rather than standing and walking around. Oculus Touch controllers were incorporated into the design to add interactivity. Listeners would be able to conduct and animate visuals to the music. These interactive elements were created with a few simple nodes in Notch.

Despite being an experienced 3D designer, Jani found designing content for VR challenging to visualise, “Working on VR content in 2D has its limits – you need to see it in VR to do it properly. Stereoscopic effects are even harder, you have to see it and be able to edit it in the VR headset or you just can’t get it right, and you end up with an unconvincing, uncomfortable stereoscopic viewing experience. Notch just completely eliminated those problems”. With Notch, Jani was able to edit content in real-time in the VR headset and real-time render to stereoscopic 360 video. “We were able to actually live preview and live edit the final video result in the VR headset. We could tweak the stereoscopic effects live and see the result. We could edit the whole scene live and see how it would look. The render to video was phenomenally fast, the final render of the whole 5-minute video in 8k x 8k at 60fps and took about 1.5 hours”.

The result was a beautiful, interactive piece that truly immersed listeners in a musical space. The band loved the video in all its iterations. Lost is a showcase of what’s really possible with VR in music. Jani achieved his goal of creating an experience in which you could truly lose yourself in the music, in fact, we haven’t seen him since.