Live productions like broadcasts, concerts, theater shows, and conferences have historically required expensive specialized hardware and teams to composite footage, motion track cameras, and render photoreal CGI in real-time. Unreal Engine 5 provides an extensive toolkit accessible to more creators aimed at transforming these complex workflows.
Ingesting a Multitude of Video Feeds
To bring live video into Unreal, producers can leverage:
NDI – IP protocol widely supported for transmitting video over networks. Aligned in Unreal with sub-frame accuracy.
Capture Cards – PCIE cards or external boxes for ingesting professional video from HDMI, SDI, etc. Recognized as webcams.
Plugins – Tools for streaming from services like OBS vMIX, mobile cameras, conferencing systems.
Robust ingest empowers productions to leverage traditional broadcast sources, prosumer devices, internet streams, and more.
Precision Tracking of Live Camera Motions
Realistic compositing requires virtual cameras precisely matching physical camera motions. This is enabled by solutions like:
Optical Tracking Systems:
Vantage by Vicon - High-end camera tracking system using infrared optical markers. Provides extremely precise 6DOF data.
ARRI TRACKING by Arri - Similar optical tracking to Vicon but tailored for cinematography and virtual production.
Active Track by PhaseSpace - Cost-effective optical tracking using active LED markers.
Inertial Tracking:
ARRI Trinity - IMU-based camera tracking solution designed by Arri for filmmaking. Wireless and highly portable.
DJI Trinity - DJI's take on IMU tracking optimized for their Ronin camera gimbals.
Video Tracking:
Ncam - Uses video input and computer vision for real-time camera tracking, especially good for outdoor use.
Stype - Camera tracking dongles that attach to cameras to enable lens data and position tracking.
Hybrid Tracking:
FreeD - Combines video, inertial and optical tracking for robust indoor/outdoor use.
Teradek CTRL - IMU sensors combined with optical markers for versatile hybrid tracking.
Stype Camera Tracking Solution
The top end optical systems like Vicon and PhaseSpace provide the highest fidelity tracking data, but inertial and hybrid solutions offer more flexibility and mobility for active film sets. Ncam, Stype and FreeD are good choices for portable camera tracking.
With robust implementations, virtual UE5 cameras can be perfectly anchored to their physical counterparts for photorealism.
Assembling Stunning Live Composites with Sequencer
Unreal Engine 5’s Sequencer provides an intuitive multi-track timeline for compositing media sources into professional scenes:
Mix and edit physical/virtual cameras, pre-rendered VFX, clips, audio, animations, and more with transitions.
Dynamically switch, blend sources, overlay graphics/text, chroma key, matte operations.
Animate virtual camera motions and properties with keyframes for motion graphics.
Sequencer enables creating complex, broadcast-tier composites fully in real-time.
Harnessing Photoreal Real-Time Rendering
Unreal Engine 5’s renderer produces CGI rivaling offline pipelines thanks to innovations like:
Lumen: Fully dynamic lighting using ray tracing to match real-world scene lighting.
Nanite: Massive amounts of geometric detail streamed efficiently.
Temporal Super Resolution: Exceptionally crisp and stable imagery.
Movie Render Queue: Multi-channel EXR sequences ready for compositing.
Artists can build environments, effects, and assets rendered indistinguishably from traditional CG.
Democratizing Professional Production Capabilities
While Hollywood leverages custom solutions, Unreal Engine 5 makes these professional-grade live production tools broadly accessible. Integrating cheaper cameras, accessible mocap, and real-time rendering can achieve remarkable broadcasts and events. UE5 brings advanced capabilities within reach for more creators looking to redefine live programming.
Comments