Project Overview:
On siggraph in 2019, the virtual engine first demonstrated the in camera VFX of the new generation based on LED ¹） The technology demonstration of virtual photography technology has won the favor of many visual studio in Hollywood because of its freedom of creation which is not bound by time and space. Such as Mandalorians and Western world, the technology has been adopted in many major productions. By 2020, it will bring a glimmer of light to the film and television production industry under the haze of covid-19.
Looking at the European and American counterparts using new technology in actual combat, many domestic film and television companies have invested in the research and development of a new generation of virtual shooting solutions based on the virtual engine camera internal visual effect. One of them is a film and television startup called surreal, founded by a group of veterans in the virtual shooting industry. When they saw Epic Games, the new technology demo, they immediately realized that, compared with the previous generation of virtual production scheme based on green curtain, in camera VFX can solve the problems of light color surface green and anti green metal surface, simulate the real environment light, and enable actors to perform in a more close to real environment, Can say that the virtual engine once again subverted the entire virtual shooting industry! So, without surprise, first soil immediately put into the research and development of the new version of Sura, an independent virtual shooting solution.
As we all know, "practice is the only standard to test truth". This is why Epic Games' home "Castle night" is always the first "crab with new functions"; Oh no, it's "the person who eats crabs"; No, let's call it the first "crab eating" game. This is no exception to the new led stage based virtual capture feature of Sura.
Surreal, together with the famous craft creation team in the industry, recently completed a "practical exercise" which started from the actual needs of commercial film and TVC, and focused on exploring the application direction of LED stage and virtual production.
In this joint test, the team built a giant curved LED wall of 45m x 6m, which is composed of 1080 LED panels and a total of 38,776,320 pixels. But that's not all. The team also built a huge canopy of 20,643,840 pixels with a total of 20,643,840 pixels composed of 1280 LED panels of 21m x 15m.
It will be a huge challenge for any engine to render such a huge image in real-time photo-level! The new version of SURA uses Unreal Engine's nDisplay as the cornerstone of its LED intelligent virtual shooting system. The nDisplay in UE4.25 provides new functions such as RTT viewport, picp projection strategy and support for multiple screen synchronization strategies, which are indispensable functions for realizing in-camera VFX.
The test used a total of 5 workstations to form a cluster rendering, and hardware synchronization was carried out through the newly supported Nvidia Quadro Sync technology of nDisplay, of which 4 were responsible for rendering the LED wall and 1 was responsible for the rendering of the top LED.
In order to ensure the testing of the dual-camera solution, and to reduce the system delay as much as possible, the project must always run stably at 50FPS. In order to achieve this goal, Surreal has developed an intelligent and dynamic allocation of rendering resources for each nDisplay window according to the camera posture. Features.
At the same time, in order to leave more creative space for the later stage of film and television, SURA supports the display of the LED area where the camera’s frustum is located as a green screen. With the on-site lighting and scheduling, it basically solves the problem of LED green screen overflow, while retaining The advantages of LED simulation of ambient lighting.
It’s also worth mentioning that, in order to cope with the rapidly changing needs of the site, the test team used the built-in Multi-user Editing function of Unreal Engine to enable multiple people, multiple stations, and multiple jobs to edit simultaneously in the same project. No interference, greatly improving the on-site response speed and shooting efficiency.
How the project enhances the user experience:
With the help of Unreal Engine technology, we are able to create the world's leading visual virtual production tool, a standardized production process, and complete a complete virtual production process from pre-creation, real-time monitoring of the shooting site, and post-rendering synthesis and color correction. In the future, we will continue to develop and upgrade processes and tools, constantly liberate creative productivity, reduce production personnel and time costs, and help promote the digital transformation of the film and television advertising industry.
Does the project have potential market applications? Where is the application and market value?
The use of LED virtual shooting is the current development direction. It further liberates the middle and late stages of film and television, sharing part of the difficulty of the lighting engineer, and at the same time providing a more realistic reference for visual effects, and reducing part of the work of visual effects production. the amount.
Making good use of this system can help new directors better plan and experiment with their own ideas, and help directors who are accustomed to real shooting, allowing them to cross the green screen barrier, touch themes they dare not touch, and give them greater play to their abilities. The audience brings better and more thoughtful works. "
Team Introduction
Xianyang Film and Television Production (Shanghai) Co., Ltd. was established in Shanghai in 2018. With virtual production technology research and development as the core, it provides a series of innovative application services for the market, and creates a convenient and stable mixed reality visualization creation platform and tools to meet the needs of film and television, Personalized XR production requirements for TV broadcasting, network rapid content production, and other application fields. Since its establishment, Xianyang Film and Television has emerged as a new force, obtained numerous technical patents and software copyrights, and served many types of industry customers, accumulated rich project experience, and formed a self-developed software and hardware combination product.
