RED Digital Cinema, in collaboration with Media.Monks, present a multi-camera live broadcast workflow for sending live, 180-degree 3D Virtual Reality footage from live events to the cloud, and ultimately delivering the output to Meta Quest 2 and Quest 3 Virtual Reality headsets.

WORKFLOW

In this live broadcast workflow, multiple RED V-Raptor VV cameras were deployed at a live sporting event. The cameras were equipped with RED Connect modules, and Canon RF 5.2 mm f/2.8 Dual Fisheye Lenses. The lenses captured the stereoscopic image, which was streamed out of the RED V-Raptor in the visually lossless R3D REDCODE RAW format, and sent over a single-mode LC fiber network to a rack-mounted GPU-based camera control unit (CCU).

The CCU is configured with a NVIDIA Mellanox ConnectX-6 Network Adapter, fitted with Single Mode LC Fiber SFP+ adapters, which accepts the feed from the RED Connect module, and with the help of the RED Connect Software app, sends the data into the GPU for real-time processing.

The NVIDIA RTX 6000 Ada GPU decodes, debayers, and color corrects the video frames, and applies a RED Connect Custom Image Processor shader code. The RED Connect Custom Image Processor shader code de-warps the frames using a calibrated UV-map to produce a 3D VR image that is suitable for the Meta Quest 2 and Quest 3 headsets. RED Connect sends the frame to the Blackmagic Design Decklink 8K Pro SDI card, which sends the 8K video feed out as a Quad 10-Bit YUV 12G SDI signal.

RED Connect sends this feed to a Blackmagic Design ATEM Constellation 8K switcher to allow switching between the multiple V-Raptor feeds. The switcher outputs the feeds to an encoder server, which encodes the SDI input feed to an HEVC stream by using Open Broadcast Software (OBS).

The encoder sends the feed out through a 1 Gigabit network to a cloud-based origin server and content delivery network (CDN) in Amazon Web Services (AWS). The origin server wraps the HEVC stream in either HLS or DASH, which with the help of the CDN, is sent out for final delivery to the headsets.

The solution outlined here takes advantage of RED Connect’s ability to:

• Take in and decode live 8K stereoscopic footage in a visually lossless format
• Apply custom image processors to the live feed
• Output an image that is properly calibrated, de-warped, and ready for VR headsets