The Return of the Blue Q

by Chris Adamson
10/29/2003

When last we saw QuickTime for Java (QTJ), the media API had been unceremoniously and unapologetically broken by Apple's Java 1.4.1 implementation for Mac OS X. As I wrote in a weblog entry at the time, the workaround was to force apps to run with Java 1.3.1, which all Mac OS X users could still use. However, this was an intolerable situation to Mac- and Windows-based Java developers who needed a high-performance media API that supported modern formats and features -- who wants to depend on an API with no apparent future?

With the release of QuickTime 6.4 for Windows and Mac OS X, QTJ has a future again. However, the API has changed greatly in 2003, so much so that nearly any existing QTJ application will remain broken unless it is rewritten for the new QTJ 6.1.

This article will describe the new QTJ by relating the history of why it was broken in the first place, how it was fixed, how to use the new version, and what we might expect to see from QTJ going forward.

From Biscotti to Broken

QuickTime for Java is one of the oldest third-party Java APIs. Originally developed under the codename "Biscotti," and formally known as QuickTime for Java, QTJ was announced in 1998. Starting with QuickTime 3, QTJ development continued alongside the rest of the QuickTime API. As the underlying native library gained features, QTJ would grow more powerful. Features that didn't require new method calls, such as support for a new media format or codec, were available immediately to QTJ. In other cases, a feature might wait one version -- for example, QuickTime 5's API for broadcasting became available to Java developers in QTJ 6.

Related Reading Mac OS X for Java Geeks By Will Iverson

It helps to understand the nature of QTJ's architecture. QTJ's goal is to present a native, straight-C media API to Java developers in a way that makes sense in Java (because it is mostly composed of calls to a native library, it can be thought of as "QuickTime from Java"). It creates an object-oriented view of QuickTime by crafting together C structs and the functions that use them into sensible objects. For example, the Movie struct and several dozen functions from Movies.h are represented as a class called quicktime.std.movies.Movie. This collection of classes is then farmed out into two sets of packages:

1. A wrapper to the underlying QuickTime API, directly translating Java calls into native equivalents. This includes all packages in the quicktime.std hierarchy, as well as a few others.
2. A bridge into Java, providing points of contact between QuickTime and Java, most obviously in providing a means of rendering QuickTime content into AWT or Swing components. These classes are in the quicktime.app hierarchy.

Apple's J2SE 1.4.1 implementation featured a rewrite of the GUI layer in Cocoa, whose object-orientation and threading models were a better fit for Java. QTJ continued to work with Java 1.4 on Windows, but the Mac version of QTJ was still Carbon-based, and calls from Cocoa's world of pre-emptive threading to Carbon's cooperative threading were a dicey proposition. There was also a too-tight integration with the older QuickDraw graphics API. For whatever reason, Apple chose not to address these problems before the launch of their 1.4.1, instead letting QTJ apps just fail with a NoClassDefFoundError when they attempted to access Carbon. Mac developers were up in arms that Apple's QTJ worked on Windows but not on their own platform.

After their Java 1.4.1 release, Apple posted information about the incompatibility on QTJ's home page, saying that the future of QTJ was under consideration, and soliciting feedback as to what features of QTJ were actually being used by developers and users. The quicktime.app was very generous in offering a large collection of display classes: effects and rendering conveniences like an off-screen Compositor, 2D "sprites," AWT and Swing components with a wide range of resizing and display options, etc. Fixing all of that code would require a lot of rewriting. With limited resources, Apple was asking the community to help identify the pieces that would be fixed.

Downsizing QTJ

The new release of QTJ, version 6.1, solves this problem by radically reducing the size and scope of the Java bridge. This new version is delivered as part of QuickTime 6.4 for Mac OS X 10.3 ("Panther") and Windows. Note that on Windows, you need to do a custom install or upgrade to get QTJ, as shown in Figure 1, since QTJ is not part of the default installation.

Figure 1. Custom QuickTime install on Windows

On Mac OS X 10.2 ("Jaguar"), the situation is a little confusing as of this writing. Updating to QuickTime 6.4 wipes out QuickTime for Java, but a separate update for QTJ 6.1 is available via the Software Update utility. Upgrading to QuickTime 6.4 but failing to get the QuickTime Java update will totally break all QTJ apps, so it's important to update both. It's not unreasonable to expect that some future version of the QT 6.4 updater would install QTJ 6.1 automatically, so this warning may not be necessary in the future.

Downloading an SDK to write QTJ apps is not strictly necessary, since all you need is to put the QTJava.zip file in your classpath. The installer puts this file in /System/Library/Java/Extensions on Mac OS X, or the lib\ext directories of any JRE folders it finds on a Windows machine. However, the QTJ SDK has traditionally included a generous collection of sample code, as well as Javadocs for the QTJ API. A Windows version of this SDK is available from Apple's SDK page. The Mac version currently exists as separate pieces, with the Javadocs in one place and the demos available in another. Mac developers will want to consult the QTJ section of What's New in QuickTime 6.4, which indicates which of these demos do and don't work with QTJ 6.1 and Java 1.4.1.

Looking at the new QTJ, the solution to the complex legacies of the old Carbon QTJ code is apparently to start over with a radically simplified quicktime.app hierarchy, one that only provides a basic ability to get QuickTime content, such as movies and images, into the Java display space. Presumably, this smaller set of functionality represents what developers said they needed, and omits the extra goodies.

Specifically, all of these packages from earlier versions of QTJ are now off-limits when developing for Java 1.4.1 or higher on Mac OS X:

• quicktime.app.actions
• quicktime.app.anim
• quicktime.app.audio
• quicktime.app.display
• quicktime.app.event
• quicktime.app.image
• quicktime.app.players
• quicktime.app.sg
• quicktime.app.spaces
• quicktime.app.ui

These packages, however, are still available, and represent the core of QuickTime functionality:

• quicktime
• quicktime.io
• quicktime.jdirect
• quicktime.qd
• quicktime.qd.text
• quicktime.sound
• quicktime.std
• quicktime.std.anim
• quicktime.std.clocks
• quicktime.std.comp
• quicktime.std.image
• quicktime.std.movies
• quicktime.std.movies.media
• quicktime.std.music
• quicktime.std.qtcomponents
• quicktime.std.sg
• quicktime.streaming
• quicktime.util
• quicktime.vr

Note: several qd3d packages have been omitted from this list because while they still exist, they do nothing on Mac OS X, which does not support QuickDraw 3D.

The only surviving part of the bridge is the quicktime.app.time package, which provides time-based callback functions, such as the TaskAllMovies class. It can be used to give movies time to redraw themselves, though this is rarely used by application developers, since adding a movie to a GUI sets up tasking callbacks for you.

Of course, if you look at the Javadocs, or peer into the .zip file with a jar tf QTJava.zip, you'll see all of the old packages and classes are still present. This is apparently to give developers the opportunity to keep using those classes with Java 1.3.1 on Mac OS X, or to use them with Windows. However, the affected classes are clearly marked as deprecated, and attempting to use one may bring up a runtime exception. For example, if you attempt to run an old QTJ app in Java 1.4.1, trying to set up the GUI's QTCanvas will kick up this exception:

quicktime.QTRuntimeException[QTJava:6.1.0g1]
Unsupported on Mac OS X and Java 1.4 and higher.,QT.vers:6408000

Replacing the old QTJ GUI classes is a small new package called quicktime.app.view, not to be confused with the deprecated quicktime.app.display. This class defines interfaces for components that can be used in AWT and Swing: QTComponent and QTJComponent. In the latter case, the "J" should be understood in the sense of a Swing JComponent, not the last letter of "QTJ."

Using the New Classes

The means of getting and using these components has changed radically. In the old QTJ, you would wire up a Movie to a Java GUI like this:

1. Create a QTCanvas or JQTCanvas component.
2. Add it to your GUI. Optionally, set resizing behavior for the component.
3. Get a Drawable, such as MoviePlayer or QTPlayer, from a Movie, GraphicImporter, or other visual QuickTime source.
4. Call QTCanvas.setClient with the Drawable to wire up the component to the movie, by way of the Drawable.

In the new QTJ, it works like this:

1. Call QTFactory.makeQTComponent() or QTFactory.makeQTJComponent() with your Movie to get a QTComponent or QTJComponent, respectively. Another signature of makeQTComponent() takes a MovieController argument, which causes the returned widget to include a typical QuickTime movie control bar.
2. Call QTComponent.asComponent() or QTJComponent.asJComponent() to get an AWT or Swing component that can be added to your GUI.

One thing to notice about this is that while the classes returned by QTFactory may call themselves "components," suggesting membership in the AWT Component hierarchy, these interfaces really only provide methods to get and set their source movies or images. For compile-time type safety, you need to make the extra asComponent() call to ensure that you have a real AWT Component.

You might also have noticed that the new workflow says nothing about resizing behavior. While the old QTCanvas described specific behaviors for resizing, such as maintaining a movie's aspect ratio or only resizing to even multiples of its original size, the new components have no such controls. Instead, they make the fairly natural default choice of reporting their content's size in getPreferredSize(), while scaling into any size that might be imposed by the AWT layout. Practically speaking, this means if you add a QTComponent as the only member of a java.awt.Frame and then pack() it, the resulting window will be just the right size for the movie. On the other hand, if you want to force a movie to always be, say, 360 by 240, you could force this with something like a GridBagLayout.

Other Changes

A few other changes are worth noting. The QTFactory class that provides these components used to be in quicktime.app, and provided methods for getting Drawables from media sources like files or URLs. The new QTFactory is in quicktime.app.view. Since both versions exist inside of QTJava.zip, it's theoretically possible for the names to collide, but only if you import quicktime.app.*, whose classes have all been deprecated.

One important feature that is now missing is capture. The underlying classes for creating a SequenceGrabber exist in quicktime.std.sg, but there's no clear way to get the captured images onscreen. It may be possible to capture to an offscreen GWorld and then devise a way to get those bits into a component, but I'm not aware that anyone has gotten this working yet. We may just have to wait for the next QTJ.

Rewriting the Examples

I've rewritten all of the example code from previous ONJava.com articles for QTJ 6.1 and included it with this article. For instance, Figure 2 shows the "Simple QTJ Editor" from a few articles back running on Mac OS X 10.3 in Java 1.4.1 with QTJ 6.1.

Figure 2. Simple Editor in QTJ 6.1, running on Mac OS X 10.3

Here's a summary of the changes for each of the examples:

• "A Gentle Re-Introduction to QuickTime Java, Part 1": The simple player demonstrated in this article simply replaces the QTCanvas with a QTComponent.

• "A Gentle Re-Introduction to QuickTime for Java, Part 2": The editor in this article is a little tricky, since I wanted to force the sizes of the components to be 320 by 240 for the target movie, and 160 by 120 for the source movie. I forced this by putting the QTComponent into the center of a BorderLayout, and then using Swing's Box to create horizontal and vertical struts of exactly the desired width or height, putting those struts in the south and east of the layout, thereby forcing a suitable size for the center.

• "Making Media From Scratch": This two-part article stays entirely within the quicktime.std packages, making rewriting unnecessary. There are a couple of quicktime.app imports, but these appear to have been left over from debugging and aren't used at runtime.

I haven't included an update to "Java Media Development With QuickTime for Java," since I haven't been able to work out some problems with its code. That article offers a bridge from the Java Media Framework to QTJ. More accurately, it uses QTJ as a Player for JMF. One of JMF's basic assumptions is that a movie's "visual component" and its "control panel component" are two separate widgets. In QuickTime, it's more typical to include the default control as part of the visual presentation of the movie. In the old QTJ, it was possible to create a "detached controller," which allowed you to create two QTCanvases or JQTCanvases, one for showing the movie and the other for the control bar. This technique, as described in an Apple tutorial, crashes in QTJ 6.1. An alternative would be to create a controller-less QTComponent with QTFactory.makeQTComponent(Movie), and then make a custom AWT component, for which you'd have to handle mouse-clicks and make appropriate calls to Movie.start(), Movie.setTime(), etc.

Also crashing is the MovieExporter code used in the article "Parsing and Writing QuickTime Files with Java." This is somewhat more troublesome, since the MovieExporters are entirely within the quicktime.std hierarchy, which seemingly didn't change for QTJ 6.1.

I've filed bug reports for these, and Apple developers have repeatedly stressed on mailing lists that they want bug reports and feature requests to be properly filed, which you can do by going to Apple's Bug Reporter. One trait of QuickTime for Java development is that new functions in the native API often don't show up in QTJ until someone asks for them. I complained in an earlier article about the lack of QTJ constants for MPEG-4 and Sorenson 3 video codecs; filing a bug on these took five minutes, and kMPEG4VisualCodecType and kSorenson3CodecType appeared in QTJ 6.1 a short time later.

The Future of QTJ

Is the shrunken quicktime.app going to hold back QTJ in the future? I don't think so. It's likely that many developers use a media API just to play media, meaning a simplified bridge could actually be better, since it is now easier to use -- it was arguably confusing to have to get a Displayable from a Movie and then use setClient() to connect it to a QTCanvas, all just to show some video. Also, QTJ still picks up new features every time the underlying library gains support for new media types or codecs. That brought MPEG-4 playback to Java developers with QuickTime 6, and means that the professional-quality "Pixlet" codec introduced in Mac OS X 10.3 should work in QTJ with no code changes required.

Moreover, the core QuickTime functions in quicktime.std provide APIs for movie editing, transcoding between formats, metadata, low-level sample access, and other tasks. This is the heart and soul of QuickTime, and it's not going anywhere. For creating and working with media, the QuickTime API remains unmatched.

Example Code

• qtj61-add-timecode-track.tar.gz
• qtj61-editor.tar.gz
• qtj61-make-csequence.tar.gz
• qtj61-make-text-track.tar.gz
• qtj61-player.tar.gz

Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.

---

Return to ONJava.com.