VFX - Digital Visual Effects

Movie Magic...
Almost any Hollywood movie that you see today uses
visual effects -- from the ships in "Star Wars" to the
monsters in "Godzilla," visual effects make the
imaginary look completely real!

Visual effects help enhance the look of a movie or

create scenery and situations that cannot (or do not)
exist in real life.
The visual effects are obvious in any science fiction film
-- of course the scenes of spaceships battling one
another in far-off galaxies are not real.

The only way to create these scenes is through an

amazing set of tools and technologies that let imaginary
places look totally real.
What you may find surprising is that almost every
Hollywood movie produced today contains visual
effects of some sort.

VFX is also known as Digital Visual Effects or DFX - the

word digital in the phrase digital visual effects means
that VFX houses primarily uses computer hardware and
software to create their effects.
The general term used in the industry for what VFX
houses do is CG, short for computer-generated.

For example, when talking to the artists you will hear

them say things like, "That entire scene is CG," or
"Those are all CG soldiers," or "The actors are real, but
everything else is CG."
Computer-generated effects make imaginary
characters like Godzilla possible, and they also create
almost every effect that used to be done using models.

The advantages of CG effects are their realism,

flexibility and relatively low cost (compared to the
alternatives).
A VFX team works with the movie's director both
during filming (production) and then extensively after
filming (post-production) to create the effects.

Involvement during production helps the director make

creative decisions so that the effects can be integrated
into the film more easily, and also allows a VFX team to
add different markers and other features to each scene
to make post-production work easier.
VFX Markers on a
greenscreen
For example, in certain scenes the camera might be
fitted with encoders that will allow for easier integration
of the effects that the VFX team creates.

During post-production, the director works extensively

with the VFX team to make sure that the effects in each
shot have exactly the right look for the film.
A typical movie might have 1,000 to 1,500 shots. A
shot might be one or two seconds long, or 30 to 60
seconds long.

A given scene in a movie might be filmed with a

number of different cameras so that there are wide
vistas, close-ups, changes of perspective and so on. In
the final movie, these different viewpoints are mixed
together to create the scene.
Therefore a single scene might contain dozens of
individual shots. By splicing all of the shots together in
the correct order you create the complete movie.
The VFX Process
Step 1: Scanning

Hollywood movies are typically shot on 35mm film at

24 frames per second. The first step of the visual
effects process is deciding which of the shots need to
have visual effects applied to them.
The rolls of film for these shots are then scanned, or
digitised, in a hermetically sealed scanner room.

Most reels are scanned at ‘2K’ resolution (2,048 x

1,556 dots per frame), an average VFX film may have
around 150 shots, the ‘intermediate files’ for these 150
shots would consume about 1.5 terabytes of disk
space.
Once it is digitised, a shot may go through a number of
3D & 2D artists.

A big part of realistic computer generated effects is the

creation of 3D models and characters and their
integration into final composites by 2D artists.
3D Processes
3D Tracking

The tracking department uses markers added to the

scene to create a 3-D model of the scene and then a
3-D camera.
The goal is for the 3-D camera to exactly mimic the
motion of the real camera so that the 3-D elements
added to the scene look right and move correctly as
the camera moves in the actual scene.
3D Modelling

A 3-D model is a collection of shapes that form the

outside of the object being modeled.
Most 3-D or CG creatures are formed from a
combination of spheres, cylinders and other shapes
that are molded on the computer screen into exactly
the right configuration to look like the creature they are
representing.
3D Setup

Setup is the process of adding a "skeleton" of bones

and joints to a 3-D model so that the different shapes in
the model move correctly with respect to one another.
In some cases the bones and joints are created by
hand. In other cases they come from motion capture
data.
Motion Capture
To gather motion capture data, an actor is fitted with a
suit that has reflective markers or lights at every joint.
The actor moves on a special stage and 3-D cameras
watch the actor from a number of different angles.
Computer software is then able to track all of the
markers and, with the help of a technician, bind them
together into a stick figure that accurately duplicates
the motion of the actor.

The stick figure is the bones and joints that then control
and animate a 3-D model.
Mo-Cap Transformation
3D Animation

In the animation process, an artist choreographs the

movement of a 3-D character.
Objects (models) are built on a computer (modeled)
and 3D figures are rigged with a virtual skeleton.
Then the limbs, eyes, mouth, clothes, etc. of the figure
are moved by the animator on key frames.
2D Processes
2D Painting

Painting is used
extensively by any
visual effects team.
e.g.
Create matte
paintings for
backdrops
Paint out wires, harnesses, brackets and other safety
equipment

Paint over the "holes" sometimes created by

rotoscoping

Touch up things like grass that has lawn mower tracks

in it
2D Compositing
Compositing is the process of adding all of the different
layers (these layers are known as elements, or plates)
of a shot together to create the final shot.
The compositing artist incorporates all of the elements
in the right order so they overlay each other properly to
create the final shot.

This is one of the most difficult aspects of the VFX

process, as artistic training, technological proficiency,
an understanding of cinematic conventions and an eye
for detail are all required.
A visual effects team is responsible for all of the effects
shots in a single film.

At PHOTON, a team consists of a producer, several

supervisors (for example a 3-D supervisor, a 2-D
supervisor, etc.) and a number of artists.

One of the first tasks for the team is the research and
development process.
The team then uses what it creates during the R&D
process to manipulate the shots it is responsible for.

With the help of the supervisors and artists on the team

as well as the film's director, the producer's job is to
look at all of the shots, understand what the director
wants to change in each one and then estimate the
amount of time that all of the tasks will take.
A bubble plate from
Ghosthip
The Technology
Digital visual effects at the Hollywood level require an
incredible technology infrastructure that includes both
hardware and software.

The machine room pictures that follow shows just how

extensive the hardware investment is. The team needs
massive amounts of hardware for four reasons:
The scanned film and the
different layers that the
team creates require
gigantic amounts of disk
space.
A single frame of a film,
once scanned and stored
on a disk, consumes on
the order of 10 megabytes
of disk space. All of the
shots of "Ghostship"
together consumed 1.6
terabytes of disk space.
Individual artists need
high-end desktop
machines to work on
and render their
individual models and
layers.
Pictured at right are the
Silicon Graphics
workstations used to
power Flint, Flame &
Inferno.
Rendering requires
massive CPU
resources. To render
any animated 3-D
figure or any effect like
water or smoke, the
CPU must generate
millions of polygons,
lines, points, etc and
then light them
correctly. And it must
do this over and over
again for each frame of
the shot!
Compositing -
Compositing combines
dozens of layers into a
single shot. Because of
the resolution involved
-- millions of pixels and
tens of millions of bytes
per frame -- and the
layering, both the CPU
workload and the
storage requirements
are immense.
A thousand words...
The easiest way to understand what a visual effects
team can do is to look at an example. The first example
that we will use involves a surprising number of
techniques to completely change the landscape from a
wooded field to a seaside town! This is a shot from the
movie "The Patriot."

To get a feeling for the total transformation, take a look

at these two short videos. The first shows the original
shot as filmed. The second shows the scene as it
appears in the movie:
 Before & After
This section of video is 20 seconds long, or about 480
frames.
Step by Step
First, a variety of things might be done to the original
shot to clean it up, correct the color and so on.

Since the camera pans across the scene in this shot,

the next step is to build a 3-D model of the camera so
that all of the visual effects the team will create will
mesh with the original scene exactly
One of the first steps taken to add visual effects to the
scene is the rotoscoping.

An artist sits at a computer and, frame by frame,

outlines the portion of the original shot that will be used
in the final version.

Rotoscopers are known within the industry as ‘Roto

Dogs’, due to their relatively low place in the VFX food
chain!
The breastworks, a portion of the field and several of
the running soldiers will all be used, but the explosion
seen on the left and everything else will be removed.

The artist will outline these elements and create a

‘matte’ that can be used to create an empty space in a
background scene. This allows a new object to be
placed in the scene.

Another camera crew has created an ocean shot. Once

the ocean shot is cleaned up and color-corrected, it
and the rotoscoped scene can be integrated:
Rotoscope Integration
Another artist in the paint department has been
working on a matte painting of the town. This is a high-
resolution digital image created using a painting/
illustration package.

Once the town is added in, the scene looks like this:
Town Integration
In this shot there are a number of boats encircling the
town and firing on it. The boats are all computer
generated. Each one is modeled and then added to the
shot:
Boat Integration
The cannon fire for the boats is its own stand-alone
effect that is created separately and then added to the
shot.

A variety of other effects are added to the shot,

including things like smoke over the town, people in the
town, a large explosion on the right, etc.
Cannon Fire
The final step in the process is compositing all of the
different components of the shot (the rotoscoped
foreground, the water, the town, people in the town,
the boats, the cannon fire, the smoke, the explosion,
birds flying overhead and so on) layer by layer to create
the final shot as it will appear in the movie.

Once digitally composited, the shot is written back to

film so that it can be spliced into the movie.
Final Composited Scene
This brings us to the end of our topics for this
semester.

Next week we will be doing a lightning fast revision of

the topics covered over the last 12 weeks.

Make sure you make it next week as we will also have

a general question time where you can clear up
anything you are not quite sure about.