The next thing we do is to create a
CSequence. This object
provides us the ability to compress frames. We have to call this with each
frame to compress, in order, and there's an interesting reason for this. If we
were using a compression scheme meant for single images, such as JPEG, we could
do the images in any order, since each frame would have all of the information it
needed to be decompressed and rendered. This is generally not true of
video compression schemes, which often use "temporal compression":
techniques to compress data by eliminating redundant information
between frames, such as an unchanging background. Because of this approach,
decoding a given frame might depend on information from one or more previous
frames, which is why we have to do our compression through an object that
understands that we're working with a series of images.
CSequence constructor looks like this:
CSequence seq = new CSequence (gw, gRect, gw.getPixMap().getPixelSize(), CODEC_TYPE, CodecComponent.bestFidelityCodec, StdQTConstants.codecNormalQuality, StdQTConstants.codecNormalQuality, KEY_FRAME_RATE, null, 0);
These arguments are, in order:
QDGraphics src: the
QDGraphicsfrom which to get image data. In our case, the offscreen
GWorldinto which we draw.
QDRect srcRect: the portion of the
srcto use. In our case, the whole thing.
int colorDepth: an
intindicating the likely depth (4-bit color, 32-bit color, etc.) at which the frames are likely to be viewed. Pass
0to let the Image Compression Manager choose for you. More info lives in the docs for the native function.
int cType: the codec type, as described above.
CodecComponent codec: often used to request a specific behavior of the given codec, such as the
int spatialQuality: a quality setting for the images, from
codecMinQuality, through low, normal, and high, up to
codecMaxQualityand, in for codecs that allow it,
int temporalQuality: the quality setting for inter-frame compression, with values as above.
int keyFrameRate: the maximum number of frames allowed between "key frames," which are the frames that have all of the information they need, and that may be needed for multiple subsequent frames to decompress.
ColorTable clut: a custom color lookup table, often set to
nullto let QuickTime use the table from the source image.
int flags: one or more behavior flags, logically
OR'd together. One interesting option is
codecFlagWasCompressed, which hints that the source image was previously compressed and gives the codec a chance to compensate for the artifacting and other image degradation that occurs when an image has been compressed with a lossy codec (like JPEG).
Once we've created the
CSequence, we get an
ImageDescription object, which we'll need later when adding
samples to the
Now we can start the loop to draw, compress, and add frames. We calculate a
fromRect, inside of the original image. This will be the
source of this frame. Next, we create a
Matrix that maps and
scales from its original location and size to the offscreen buffer's location
and size; in other words, a rectangle at (0,0) with dimensions
GraphicsImporter.draw() performs the scaled drawing of the region
into the offscreen
Matrix drawMatrix = new Matrix(); drawMatrix.rect (fromRect, gRect); importer.setMatrix (drawMatrix); importer.draw();
Next, we compress the image that was drawn into the offscreen
CompressedFrameInfo cfInfo = seq.compressFrame (gw, gRect, StdQTConstants.codecFlagUpdatePrevious, compressedImage);
The arguments to this call are:
QDGraphics src: the source image to compress.
QDRect rect: what portion of that image to use.
int flags: behavior flags. Among the most useful is
codecFlagUpdatePrevious, which is used for codecs that use temporal compression. Another interesting option not needed here is
codecFlagLiveGrab, which you'd use if you were generating images from a live source, possibly image capture, and needed to compress frames as quickly as possible. In the typical QuickTime style, the desired flags are mathematically
RawEncodedImageinto which the compressed frame will be written. This is the object we made sure was big enough with that
compressFrame call returns a
CompressedImageInfo object, which has an important method called
getSimilarity(). This value represents how similar the compressed
image is to the one compressed just before it. A value of 255 means the images
are identical. 0 means the compressed frame is to be a "key frame,"
meaning it has all the image data it needs, it does not depend on other frames,
and other frames may depend on it. Other values simply represent image
difference, where low values mean low similarity.
With the frame now compressed into the
RawEncodedImage, we can
add a sample to the
VideoMedia, with the
method inherited from the
videoMedia.addSample (imageHandle, 0, cfInfo.getDataSize(), 20, imgDesc, 1, (syncSample ? 0: StdQTConstants.mediaSampleNotSync) );
The arguments to this method are:
QTHandleRef data: a reference to the sample data; in this case, to the
int dataOffset: an offset into the
data. This is
0in our case, since we're using all of the
RawEncodedImagethat was populated by
int dataSize: the number of bytes of
data, starting at
dataOffset, to use. Again, we're using the whole
int durationPerSample: how long this sample lasts, expressed in units of the media's timescale. Since our timescale is 600, a duration of 20 equals 1/30th of a second.
SampleDescription sampleDesc: an object that tells the media what to do with the sample data being passed in. This is why we got an
int numberOfSamples: the number of samples provided by this call. For video, this is typically one frame. For other kinds of media, there are some performance considerations described in the native docs.
int sampleFlags: behavior flags. The interesting value here is whether or not this is a "key frame," also known in QuickTime as a "sync sample." We set the
mediaSampleNotSyncflag if our earlier call to
CompressedFrameInfo.getSimilarity()returned non-zero. Note that failing to set this flag correctly is a popular cause of movies that "blur" when scrubbed or played from any point other than the first frame, as explained in an Apple tech note.
Once the loop finishes, we do the same clean-up tasks as with the text-track samples in Part 1 -- declare that we're done editing and insert the media into the video track:
videoMedia.endEdits(); videoTrack.insertMedia (0, // trackStart 0, // mediaTime videoMedia.getDuration(), // mediaDuration 1); // mediaRate
Finally, we save the movie to disk, exactly as before.
Here, for those with QuickTime 5 or 6, is a videotrack.mov movie produced by the sample code. If you recompile and re-run the code with different codecs and different sizes, you'll see some fairly dramatic differences in file size and image quality. I've used 160x120 to keep the file size small, in order to avoid abusing O'Reilly's bandwidth, and the compression artifacts here are more visible than in the 320x240 version.
Also remember that while we just copied a scaled section of an image into
the offscreen buffer, you can do any kind of imaging with this buffer before
compressing it into a frame. For example, you could do the drawing commands in
QDGraphics class, or use the
QTImageDrawer to use
Java 2D Graphics methods to draw into the QuickTime world. With some
bit-munging, you might even find a way to render 3D graphics from JOGL into QuickTime ... anyone up for rendering Finding Nemo directly into a QuickTime movie?
This completes our tour of QuickTime media structures, in which we've gone from the high-level view of what makes up a movie to the low-level mucking around with individual samples. This is a little "closer to the metal" than QTJ usually requires, but if you believe in keeping simple tasks easy and complex tasks possible, this has been an example of the latter.
Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.
Return to ONJava.com.