The VFX Pipeline: Your Ultimate Guide to the VFX Workflow

Ira Morris put it this way:

“All these specialists working in unison can be compared to a car manufacturing plant. As the vehicle goes through the pipeline different specialists will create different pieces in order to perfect the outcome. If every specialist understands and creates their specific piece to perfect specifications then the final car will all fit together for a beautiful, finished project. The VFX pipeline is the same – an idea is born, the job is completely understood (technical, visual, other artists workflows) by all specialists, each specialist completes their task, all pieces composited together for a beautiful, finished project.” 

Each project is different, and time constraints play a major role in how many specialists are needed. In some cases, one artist can handle the entire project, but they still have to follow the same pipeline - understanding ideas, completing parts of the project, and combining them. There are other projects that are much more complex and time-consuming. All tasks must be carried out by different specialists simultaneously during these projects to make the job come together faster and more precisely. It is impossible for one artist to be a perfect specialist in every aspect of VFX. In order to achieve high quality results, a team is required. 

A VFX supervisor should inspect every part as it is being performed to ensure that errors can be caught quickly and resolved. The larger the team, the more important it is to have a VFX supervisor. VFX supervisors monitor and inspect every stage of production (including before the shooting is performed) to ensure the desired result is achieved. 

VFX Workflow During the Different Stages of Production 

The VFX workflow can only succeed by understanding the sum of its parts. With workflows broken down into manageable segments, all tasks will be completed on time and within budget. There are three phases to the work: pre-production, production, and post-production.

It is important to understand that the visual effect is an essential part of the image just like set building, makeup, costumes, lighting and so on. Therefore, the work required to create the effect begins in pre-production, long before the first shot is captured.

VFX Pre-Production

The VFX pre-production phase very much mirrors the development and pre-production phase of the producer’s process.  It all starts with planning, developing and researching where the technical approach to the film's effects are decided.

Research and Development

During this stage, software preferences and techniques for the movie are discussed in great detail. Concepts and ideas are pitched, preliminary teams are assembled, and early visuals are created. If the film requires significant VFX components, artists, programmers, even scientists and mathematicians, are brought in to discuss what special effects programs should be used to bring the concept of the film to life. It could be mass produced software such as Houdini, which is a node based procedural workflow, or something custom might have to be built for the production's unique needs.  For example, Buf worked from 2D photographic reference images to develop photo realistic CGI sequences in David Fincher’s Fight Club using a workflow and software developed in-house.

Ira Morris explained the importance of collaboration during this phase of the project:

 “Work on an image should always begin with the discussion of the idea with the creative team, VFX supervisor, director, cameraman, producer and everyone who should take part in creating the effect. For example, most of the time the work on the effect is carried out in collaboration with make-up artists, costume designers or stuntmen. The shot must be prepped so that after the image is captured everything is ready for the work to continue. In order to have an accurate idea of how the effect should look it is necessary to make concepts of the effect to discuss what is possible and how it will need to be captured.”

Pre-Visualization

Visual representations of the film's actions will be created by a team of artists at this stage in the form of storyboards and animatics, which is a moving form of the storyboard. The animatic is created in its most simple form, portraying the sequence timing of the project through 2D storyboard drawings. In order to determine how to execute every shot, everything from character motions to the storyline will be analyzed. Then these 2D animatics are converted in the pre-visualization stage into a 3D animated rough draft of each shot.

Ira Morris explains how previs is critical to the VFX workflow:

“The shot must be prepped so that after the image is captured everything is ready for the work to continue. In order to have an accurate idea of how the effect should look it is necessary to make concepts of the effect to discuss what is possible and how it will need to be captured. A VFX team can also start making the 3D elements or characters, draw backgrounds, do test work on various FX tasks so that when the filming begins everything is planned out and understood. The beginning of the filming is the “point of no return”. Everything must be planned correctly before the shooting begins. Mistakes made while filming are sometimes very difficult to correct, and this leads to loss of quality and time-consuming fixes.” 

Production

Although many elements of the production phase of VFX overlap with pre-production, there are three main steps that take center stage during this cycle: 3D modeling, matte painting and principal and reference photography.

3D Modeling

3D modeling turns concept art into a digital subject. A wide range of digital applications and animation techniques are used in this labor-intensive process. There is not a down time in the 3D modeling workflow as it requires skilled planning and execution throughout all phases of production.

A premier example of stellar motion capture work during production is Industrial Light and Magic’s transformation of actor Bill Nighy as Davy Jones in Pirates of the Caribbean. 15 years later, the visual effects created with motion capture (or “mocap”) are still a gold standard. From a single two dimensional image, they were able to extract the actors full 3D performance and turn it into an animation, put it into the creature and right back into that same shot. Using this technology, they capture the performance "in the plate". Gore Verbinski (the director) can direct the actors and move the camera like he would in real life action sequences. 

Matte Painting

Matte paintings are used to create the illusion of a set that doesn't exist. The technique originated with hand-painted techniques using matte paint since it does not reflect light.

Although 3D models make up most of an environment, matte painting is still used for many backgrounds.

For digital, matte painting is replaced by “compositing” or “chroma keying”.  Compositing involves combining visual elements from different sources to appear as though they are in the same place. During production, a green screen or blue screen is used to film the visual effect, which is later replaced in post-production using a compositing software program. 

Principal and Reference Photography

A production member from the VFX team known as a Lidar supervisor will be on set during filming to survey, photograph and scan sets. This role is primarily concerned with taking 3D volumetric scans of sets and locations, capturing photographic textures of those locations, and giving the data back to the visual effects studio. The film scans are then used for various post-production tasks, such as camera matchmoving or creating digital set extensions.

There are a lot of challenges involved. The time to capture scans and textures is usually very limited. However, it is an essential one for both VFX-heavy and non-VFX films, since that kind of data can help in a number of ways with shot production, or even facilitate 'manufacturing' shots that cannot be achieved during principal photography.

VFX Post Production

All the elements of a film are put together during the VFX Post-production phase - video footage, special effects, computer-generated effects, music, sound, editing. A project's timeline is dominated by the post-production stage since the majority of VFX work is completed there. It can be a lengthy process, taking months to even years to complete. 

Although many elements of VFX overlap such as modeling, in post-production the final processes include rigging, matchmoving, animation / FX animation, texturing, lighting and rendering, rotoscoping and compositing.

Rigging 

The rigging process is assembling or putting the bones in the mesh of the 3D model. Riggers create realistic structures with joints that correspond to the way objects should move or react within certain environments.

It then becomes a back and forth process with the animators who try to 'break' the rig in order to create a revised one. The rigging team uses a variety of tools to calculate skin weights, and the animators use the results to bring a character, prop, or object to life. In most cases, motion capture cameras are used to collect data, which animators then use to create the final effect.

Match Moving or Motion Tracking

Match-moving or motion tracking is a system that allows you to insert computer graphics into live-action footage with the correct position, scale, orientation, and motion in relation to the objects in the shot. Anytime computer-generated imagery (CGI) needs to be placed into a live-action sequence, or vice versa, a matchmove is required.  

Animation and FX

Once the rigging and motion tracking happen, it’s time to bring the rigged elements to life. The next step is amending any kind of animations that require simulation. Then, the FX team is tasked with adding simulated elements to film, which are usually broken into three subcategories:

• Particles 

• Fluids 

• Flame and rigid body dynamics 

   

The FX team often work alongside the animators to ensure realism and natural movements. Polishing is also applied over existing scenes to enhance the visuals.

Texturing

This process involves the addition of surface color and texture to the 3D models, making the models realistic and nearly complete. For example, scales added to a crocodile’s skin, reflections on car doors, basically any quality that mimics the attributes you would see in real-life. All these detailed characteristics on people and objects are in aid of hyper-realistic results.

Lighting and Rendering

Once the texture setting is correct, lighting artists will begin rendering the final CGI. Each shot is finalized and enhanced using shader settings and lighting maps. To ensure realism, lighting artists use industry software such as Maya to create, adjust, and place spot, area and directional lights.

As soon as this is done, the scene will be sent to a compositor for finalization. 

Rotoscoping

By rotoscoping, artists manually draw around and cut out objects from movie frames in order to use the required parts of the image. The parts of an image that are wanted after cutting out are known as mattes. Roto artists work on the areas of live action frames where computer-generated (CG) images or other live-action images will overlap or interact with the live image.

To attack a small town in Alfred Hitchcock's The Birds, for instance, scores of birds were needed. Because the shots were originally taken against cliffs, beaches, or the sea, it was impossible to control the birds or even key them against a background. The only option was to trace the birds. It took two artists 3 months to rotoscope the 500 frames of one attack shot. 

Compositing

The final step in post-production is compositing. The live footage, the scene preparation data, the matte paintings, and the various VFX renders are given to a compositor, who takes all the elements of the film and then layers them on top of one another. At this point, color correction, masking, etc. are applied and refined to achieve the final results. It is the compositor’s job to make sure that real-life objects and characters are engaging with computer-generated effects and make everything look seamless and realistic.

Ira Morris had this to add about the final post-production process:

“After pre-production and production are complete the editing process starts post-production. The editor will finalize the edit and prepare shots to be delivered to the VFX team. We prefer to work before the color grade to maximize detail and information within the image. Once receiving the material from the editor, the technical elements can be removed – tracking markers, rigs on actors, mats on the floor, and any other elements that help specialists with each task. Then the CGI elements, that have been created before and during film, can be added into the shot and manipulated to fit within the scene. At this stage there is a close cooperation with director, creative team, cameraman and colorist in order to make corrections or adhere to wishes and requests. A complex visual effect is made step-by-step starting in pre-production and carrying through until the very end.”

VFX Pipeline Best Practices 

A VFX pipeline is essential for making sure all the 3D elements and complex effects are handled professionally. Beyond the technical chain of processes from planning to production to post-production, here are some best practices to consider for a successful workflow.

Always Be a Student

Modern requirements for visual effects are constantly growing and the effects are becoming more and more complex. In an industry that is constantly evolving, it is important to adopt the mindset of always being a student. New technology is creating solutions that help speed up workflows and simplify work. What was difficult to do a few years ago is now considered an easy task. 

Communication

The main step that will help create a seamless workflow is being able to communicate efficiently and effectively with the clients in pre-production. A properly planned and executed shot will allow the VFX artist to complete the best possible effect and finalize the shot faster. Having excellent communication skills will give your clients confidence in your work and always keep them coming back. Communicating how the shot should be prepared will save the client money, ease their stress about the outcome and will allow the artist to work toward perfecting the effect instead of fixing the mistake. 

These two best practice tips were offered by Ira Morris of Filebytes:

Back In Black

“One hack that an artist can add to their tool belt is to make sure the client never uses green on replacement screens. We see this a lot with cell phones. Most of them are just fully green (I believe there is an app to add green). There aren’t even any markers to track. Best practice is to keep the screen black and add little white markers in the corners. This will save a lot of time with not having to track manually and removing the green screen spill. The artist can also restore the reflections easier to keep the natural look. Any time an artist adds an image within another image it should always be black with little white markers to track and place within the scene.” 

Save Color for After VFX

“Another hack to consider and one that clients always ask us about is to work off the flat, unaltered image. This will allow the artist to work with the most detail possible for tracking and the colorist can color the already finished file. A lot of times the colorist is working on the color at the same time as the effect is being completed. The colorist will replace the shot and be able to use mattes, that we supply, to separate and adjust the effect however needed for the client’s approval. If the color is done first, then the file will be harder to work with and will probably need to go back to the colorist to color grade the effect. We flatten our effects in order to match the file so that the colorist only uses the matte for minor adjustments. This helps with eliminating the halo effect that can occur if the mattes are relied on too heavily.”

Wrapping It Up

VFX workflows are complicated. As technologies evolve, so will the VFX pipeline.  It’s important to always be learning and keeping up with the exciting innovations. Balancing the technical demands of the workflow with the creative tasks of the project can also be a challenge. Communication is key, as is understanding each person's role on the team and the time needed to produce fantastic results.