TechNote (Archived)

Flash for videographers and 3D animators

The development of video support in Flash

Beginning around the release of Flash 4, developers found multiple ways to "fake" video playback in Flash. This technique involved rotoscoping an existing digital video into a sequence of bitmap files, one for each frame. The technique was essentially the same as a flipbook of pencil drawings that looks like animation when flipped through rapidly. The bitmap sequence would then be imported into Flash. The result, when exported as a SWF file and played back through Flash Player, "looked" like video. But having each frame contain a bitmap was very costly in terms of file size, and the process was awkward.

Flash MX was the first version of Flash to add support for the FLV format, playing through Flash Player 6. This allowed import of any standard video file supported by QuickTime or Windows Media Player, including MPEG, DV (Digital Video), MOV (QuickTime), and AVI. This process involved the addition of the Sorensen Spark video codec to Flash authoring and to Flash Player. Flash MX installation included the Spark codec. A video could be imported and embedded on the Flash timeline in a manner similar to the earlier "flipbook" technique. To play back these videos, the Sorensen Spark codec was built into Flash Player. The technique proved very popular, though file size was still an issue.

Flash MX Professional 2004 expanded on the FLV concept. It was still possible to embed video on the timeline using the earlier technique. But this version of Flash installed a standalone Flash Video Encoder. This allowed developers to convert existing video into FLV format. These videos then played back via internal implementation of the Sorensen Spark video codec in Flash Player.

The conversion to external FLV allowed two important improvements. First, developers had the ability to play FLV inside Flash as a linked file that played progressively. This allowed file swapping and delivery of different files for different bandwidth, and created a file size advantage with the compression of the FLV format. It also allowed true streaming of the FLV format (when coupled with the new Flash Communication Server product, now renamed Flash Media Server). These two abilities contributed to a rapid explosion of video use in Flash that began with Flash MX Professional 2004 and Flash Player 7.

Adobe Flash Professional 8 added a second codec, the On2 VP6 for further compression options, and added support for alpha channels inside FLV, allowing chromakeying in Flash content. Additionally, Flash Professional 8 included the first FLVPlayback component, making the addition of video to Flash content very simple.

SWF files compared to FLV and other video file formats

Vector sprites: The SWF file format uses vector sprite-based animation. Each piece of art is described once, and then can be reused indefinitely. A sprite-based format describes each element and then provides compact instructions on how this sprite should be manipulated.

Video bitmaps: In contrast, a video codec describes the bitmap content of each video frame. The content of each frame is essentially a bitmap, calculated pixel-by-pixel. This is true of FLV and all other digital video such as QuickTime, WMV, and AVI. The Sorenson Spark codec and the On2 VP6 codec used in Flash Player 6 and later describe the contents of each keyframe in the video, and then describe the items that move between key frames. This is how a codec achieves smaller file size.

In a SWF file, media elements and instructions are interleaved; at the head of the file is the artwork needed for frame one, followed by the positioning instructions for frame one. If frame two needs new art then this comes next in the file, followed by the instructions for frame two. The entire file is built up in this way: first with the necessary media descriptions for a frame, and then the instructions on how to display that frame.

Each picture, each sound, each frame of video, and each element in the animation need to come down the wire before they can be manipulated. In Flash, the file size is determined by the number and size of all separate media elements. In video, the file size is determined by picture area, frame rate, total duration, and compression settings.

Note: The full SWF file format documentation is available for download, and a short article on optimizing SWF files for streaming is available as How can I make SWFs start quickly? (TechNote tn_14300).

Where does FLV come from?

FLV is a digital video format. It can be created in a number of ways:

Converted from existing QuickTime, AVI, DV, or other digital video format using Flash Video Encoder (included with Flash MX 2004 and later)
Exported from Adobe After Effects (version 7 and later)
Captured live from webcams by Flash Media Server
Converted from existing digital video using Sorensen Squeeze or On2 Flix Pro
Flash vector sprite animation can be converted to FLV by exporting to QuickTime or AVI and converting to FLV using Flash Video Encoder.

Some things are better in SWF than FLV

You get screen size for free. A large canvas costs no more than a small canvas when the content is vector sprites drawn in Flash or imported from Illustrator or another vector application. Small vector elements can animate on a large canvas as easily as they animate on a small canvas, and the total picture can also scale easily. But in an FLV, increased video size means increased file size. The effect of this can be alleviated by smart development using a Flash Media Server hosted FLV and SMIL file support. SMIL allows you to deliver varying sized FLV streams based on the end user's bandwidth. For further information on using SMIL in Flash, see "Using a SMIL file" in Flash LiveDocs.

Reusable elements for free. If you have a short cycling vector sprite animation, then this animation clip can be run indefinitely without significantly affecting file size. The significant cost in a SWF file is the initial description of a media element. Once the media is downloaded, then it can be animated and reused at very low cost. This is also true of FLV when it is played progressively (and stored in the browser cache). However, this is not true of FLV streamed from Flash Media Server, which is not cached.

Tinting and coloring for free. Flash color effects are operations that are applied to elements drawn in Flash. These instructions are relatively low in cost. Filters can be used to apply similar effects with a slightly higher file size cost.

2D animation at very low cost. Once a media element has been downloaded, it can be moved left and right, up and down, scaled, and rotated with only a few bytes of instructions. Some of these options are available with FLV video as well. For example, it can be moved around the screen if necessary. It can be scaled if necessary, though scaling up is not advised.

Longer animations at low cost. In Flash, additional animation time may not require additional media and therefore may be very economical. In FLV and digital video, each additional frame of animation adds incrementally to total file size.

Shape-changing animations at low cost. Flash shape-tweening lets you apply instructions to a piece of vector art to change its shape and color over time. In FLV and digital video, a shape-changing animation requires that each frame be downloaded separately.

Panning and zooming at low cost. In Flash, the main cost of a camera move is in the labor to construct it. The various objects in a scene can move, rotate and scale without significant effect on file size. Camera moves can typically be very expensive in a digital video file because so many pixels change each frame, increasing file size significantly.

Some things are easier in FLV than SWF

Less processing power. Digital video presentations tend to have a narrower range of processor loads than the range of various SWF presentations. In Flash, if many elements are animating at once, if partial transparency is applied to all of them, if they are changing shape, if they cover a large area of the screen, and if MP3 audio is decompressing in the background while a download is simultaneously occurring, then slower processors show the effects of these requests.

More predictable frame rate. Video codecs are time-based animation streams, and most movies drop frames if necessary in order to maintain the desired frame rate. In event-based animation systems such as Flash, typically each frame is guaranteed to display and time is dropped in order to ensure that each frame is seen.

Audio synch is simpler. Because audio is also time-based, it is easier to synchronize in a time-based video player than in an event-based interactive player. More information on this topic is available in What are some ways to synch audio in Flash? (TechNote tn_14108).

Greater detail at lower cost. Captured imagery (video cameras, scanners) or synthetic imagery (3D renderings, photo-processing) often is more economically expressed in a pixel-based video codec. Some codecs display a single bitmap more economically than JPEG, and many codecs can use interframe compression to make a bitmap stream more compressed than it would be as individual images. If you're dealing with much captured imagery, then a video codec typically produces smaller files.

It's easier to scale bitmaps and apply effects in video. In a digital video file, all resampling is done at design time, and so more expensive algorithms can be used than in realtime scaling. Although Flash Player can resize bitmaps, to do so in realtime usually results in some quality degradation and a definite performance hit. (Because Flash files often scale in themselves, it's very common to have bitmap resizing. Scaling up FLV video in Flash Player is possible, but this degrades quality. It is preferable to make multiple FLV files and deliver the appropriate size.)

Some differences between SWF and FLV

Flash Player renders curves to pixels in realtime. Flash realtime rendering is closer to a VRML or other realtime 3D engine than to a video player. A video player receives descriptions of pixels and decompresses these descriptions to screen. Flash receives descriptions of curves and converts them to pixels on screen.

Transitions are handled differently. Assuming that the art is already downloaded, then a jump cut in Flash is essentially free. However, in FLV, a transition such as a cut requires all pixels to change and be recalculated. Wipes, pushes, and slides can be accomplished in Flash without a significant impact on file size. Even straight cross-fades can be done by animating opacity in Flash (although it's rare to change the alpha of many objects simultaneously and can be processor intensive).

Pixel-based transitions, such as pixel dissolves, are generally difficult or impossible in Flash because of the vector nature of Flash sprites. However, these are very commonly pre-rendered in an FLV.

Timelines may differ among applications. The Flash timeline is similar to that used in cel animation. In traditional cel animation, any number of objects can be painted onto one celluloid layer. A single cel layer can also cycle through various pieces of art, as for a walk cycle. Motion is a property of the cel layer itself: if the walk cycle moves across the screen, then pegs are used to move the whole layer.

The Flash timeline is more similar to a traditional animation system than to a video editor or a 3D animation tool. You can use as many layers as you wish and put many things on each layer, and motion is applied to a layer rather than to an object on that layer.