Re-Introducing QuickTime for Java, Part 2

by Chris Adamson
06/04/2003

In the first part of this re-introduction, we looked at the history of QuickTime for Java (QTJ), the issues involved with writing, compiling, and running QTJ apps, and presented a very basic movie player. In this second part, we'll explore the organization of QTJ and use its editing API to write a small video editor.

Getting Object-Oriented

In the first article, I noted that a newcomer might be surprised at the conventions of QTJ programming and the remarkable number of packages that even a simple app needs to import. Both of these traits have their beginnings in the fact that QTJ did not come about the way that many Java APIs do, with the design of the API and then a reference implementation. In QTJ's case, the implementation already existed, in the form of the native QuickTime libraries on Mac and Windows. The Java API had to come second.

`task()`: The Most Import Method You'll Never Call

One curious Movie method you might notice in the JavaDocs is task(), whose description reads: "the moviesTask method services this movie. You should call moviesTask as often as possible." First of all, "moviesTask" is the name of the C function. As you might guess, the Java equivalent is task() in the Movie class. I guess they copy-and-pasted this from the C docs without "Java-izing" it.

More interestingly, you could look through all 50+ sample apps in the SDK and not once see a call to task(). What's the deal?

It is true that task() must be called frequently for your Movie to do anything, but as it turns out, QTJ is very generous in providing these calls for you. In our simple player example in the previous article, the MoviePlayer registers with the TaskAllMovies class to provide regular calls to task(). Some of the other Drawables, like QTPlayer (which we'll use in the next section), implement the Tasking interface, which gives them access to a "tasker" thread, which periodically calls back to their task() methods.

So you're largely off the hook for having to call task() periodically. The rare case where you have to care is when you have a Movie that has not yet been added to a GUI, yet is performing some kind of activity. In a previous article about extending the Java Media Framework with QTJ, we wanted to load a Movie from a URL and while we didn't want to wait for the entire file, we needed to wait until there was some data so we'd know the dimensions of the Movie. Here's an edited version of the code:

qtMovie = Movie.fromDataRef (urlRef,
                             StdQTConstants4.newMovieAsyncOK |
                             StdQTConstants.newMovieActive);
System.out.println ("Got Movie " + qtMovie);
qtMovie.prePreroll (0, 1.0f);
qtMovie.preroll (0, 1.0f);
while (qtMovie.maxLoadedTimeInMovie() == 0) {
   qtMovie.task (100);
}

This code tells QuickTime to get the movie from the URL (wrapped in a DataRef object), and to prePreroll and preroll it (which let QuickTime allocate resources for playing back the movie). Since the Movie hasn't been added to a MoviePlayer, QTPlayer, or anything else that will provide timed calls to task(), we have to provide the task() calls, at least until we add it to a MoviePlayer, QTPlayer, or similar task()-caller.

One other interesting thing to note in this example is how we call the fromDataRef with two behavior flags, StdQTConstants4.newMovieAsyncOK and StdQTConstants.newMovieActive, mathematically OR'ed together with the | operator. This is a very common practice throughout the QTJ API. Flag constants are defined in the various StdQTConstants classes, and are typically ints whose values are powers of two, meaning they have exactly one bit turned on. Here's an example from the native Movies.h file:

enum {
  newMovieActive                     = 1 << 0,
  newMovieDontResolveDataRefs        = 1 << 1,
  newMovieDontAskUnresolvedDataRefs  = 1 << 2,
  newMovieDontAutoAlternates         = 1 << 3,
  newMovieDontUpdateForeBackPointers = 1 << 4,
  newMovieDontAutoUpdateClock        = 1 << 5,
  newMovieAsyncOK                    = 1 << 8,
  newMovieIdleImportOK               = 1 << 10,
  newMovieDontInteractWithUser       = 1 << 11
};

By OR'ing them together, you can pack the flags into one int. The method receiving the call will mask off bits to determine with which flags it was called. Unfortunately, the appropriate flags are often not described in the JavaDocs — for Movie.fromDataRef(), the JavaDocs advocate using newMovieAsyncOK, but other appropriate flags like newMovieActive are only described in the appropriate section of the C documentation. Fortunately, methods in the JavaDocs generally link to the appropriate C call's documentation, but sometimes you still have to think about how the C call translates to Java.

Creating Movies

Like I said earlier, what sets QuickTime apart is its focus on being a media creation API. Our closing example will show off some of those features.

While we don't have the space here to rewrite iMovie completely in QTJ, we can certainly replicate the core of its functionality — combining video clips from multiple sources and saving them into a new movie file.

To keep things simple, our "cuts-only" editor will simply let the user load a source movie, select some or all of it, and copy that to the system clipboard. Paste will put that video (or whatever video is in the system clipboard) into a target movie, either appending to the target or replacing the target's selection. If you don't want to touch the clipboard, there's a "low-level" editing API, specifically the method Movie.insertSegment(), that could be used instead. Post-paste, the user can then make another selection from the source movie, or open a different source movie. When done, the user can save the target movie to disk.

In general, our needs can be broken up pretty simply:

At launch time, create an empty target movie.
When the user clicks a "Load Source" button or menu item, show a FileDialog and load a new movie from the selected file.
When the user does a "copy," put the source movie's selection on the system clipboard.
When the user does a "paste," put whatever is on the system clipboard (provided QuickTime can handle it) into the target movie.
When the user does a "save," we save the target movie to disk.

How much does QTJ help us to do this? The sample SimpleQTEditor class is about 360 lines, and 100 of that is in GUI layout. By way of comparison, it takes the Java Media Framework over 1,000 lines just to concatenate two media files together on the command line.

Here's a sneak peek of the editor, scaled down to fit on the page:

Setting Project Builder to use Java 1.3.1 for QTJ on Mac
Figure 1. Running SimpleQTEditor

QTJ makes handling the above tasks remarkably straightforward:

Creating the empty target movie. For this, we construct a new Movie and get a MovieController, which is needed for the QTPlayer to show an onscreen control bar. We enable editing and add it to the GUI.
```
protected void initTargetMovie( ) throws QTException {
    targetMovie = new Movie ();
    targetMovieController =
        new MovieController (targetMovie);
    targetMovieController.enableEditing (true);
    targetPlayer = new QTPlayer (targetMovieController);
    targetMovieCanvas.setClient (targetPlayer, false);
    targetMovieCanvas.clientChanged (320, 256);
    targetMovieCanvas.invalidate();
    validate();
}
```
Notice, by the way, that using enableEditing changes the slider (AKA the "play head") on the control bar (AKA the "scrubber") from its usual ball shape to this:

The pointy shape is probably meant to make it easier to see where you're clicking on the scrubber, but this looks a lot different than the custom controllers in the QuickTime Player or iMovie. Frankly, it looks kind of weird. Of course, you could always create your own controller widget in Java (by subclassing Canvas or JComponent), track the mouse and keyboard actions, and call methods on the Movie or a MovieController to play, stop, move around, etc.

Loading the source movie. Here we bring up a FileDialog and try to load it with Movie.fromFile(). If that works, we replace any movie currently in the source QTCanvas with this one. We also switch the states of the open and close buttons.

protected void openSourceMovie() {
    FileDialog fd = new FileDialog (this,
                                    "Select source movie",
                                    FileDialog.LOAD);
    fd.show();
    if (fd.getFile() == null)
        return;
    try {
        File f            = new File (fd.getDirectory(), fd.getFile());
        QTFile qtf        = new QTFile (f);
        OpenMovieFile omf = OpenMovieFile.asRead (qtf);
        Movie openedMovie = Movie.fromFile (omf);

        // if we get to here, we can remove any existing
        // source movie and reset the canvas' client
        sourceMovieCanvas.removeClient();
        sourceMovie = openedMovie;
        sourceMovieController =
            new MovieController (sourceMovie);
        sourceMovieController.enableEditing(true);
        sourcePlayer = new QTPlayer (sourceMovieController);
        sourceMovieCanvas.setClient (sourcePlayer, false);
        closeButton.setEnabled(true);
        openButton.setEnabled(false);
    } catch (QTException qte) {
        showOKDialog ("Error", qte.toString());
    }
}

Copying the selection from the source movie. This is trivial.
```
protected void copyFromSourceMovie() {
    try {
        if (sourceMovie != null) {
            Movie clipMovie = sourceMovie.copySelection();
            clipMovie.putOnScrap (0);
        }
    } catch (QTException qte) {
        showOKDialog ("Error", qte.toString());
    }
}
```
The copy is fast and the paste will be too, even if you're working with many megabytes of media. That's because QuickTime is all about working with references to media, so in this case, we're basically copying pointers to the media in their original locations (in the source movie files), not copying over all the media itself.
Pasting to the target movie. The paste would be trivial if not for a few UI niceties I've included. First, we create the empty target movie if it doesn't already exist. Movie.fromScrap() is a simple enough call, though we need to double-check that it actually did return a Movie and not null. The first nicety here is that the paste is made to the end of the movie if there is no selection — it would otherwise go at the beginning, and that's counter-intuitive when you're selecting sources and putting them into the target one after another. Post-paste, we clear the selection (so the user doesn't inadvertently wipe out some or all of this movie with the next paste), and jump to the end of the target movie to show that we did something (it might be nicer still to jump to the end of the pasted segment).
```
protected void pasteToTargetMovie() {
    try {
        if (targetMovie == null)
            initTargetMovie();
        Movie clipMovie = Movie.fromScrap(0);
        if (clipMovie == null)
            showOKDialog ("Whoa.", "No movie on scrap");
        else {            
            // check selection, paste to end if none
            TimeInfo selection =
                targetMovie.getSelection();
            if  ((selection.time == 0) && (selection.duration == 0)) {
                targetMovie.setSelection (targetMovie.getDuration(), 0);
            }
                
            targetMovie.pasteSelection (clipMovie);
            targetMovie.setSelection (
                   targetMovie.getDuration(), 0);

            targetMovieCanvas.removeClient();
            targetMovieCanvas.setClient (targetPlayer, false);

            // move to end of target movie, to show edit
            // had an effect
            targetMovie.setTimeValue (targetMovie.getDuration());
        }
    } catch (QTException qte) {
        showOKDialog ("Error", qte.toString());
    }
}
```
The use of removeClient and setClient is kind of overkill, but the preferred method of sizing a QTCanvas — setting a resize behavior and informing it of changes to sizes in the underlying movie — doesn't apply in this case. There is no target movie attached to the QTCanvas when the GUI is created; that's done by lazy instantiation in the first call to pasteToTargetMovie().

Saving the target movie to a file. There are several ways to save a QuickTime movie to disk. In this case, we use the very handy Movie.flatten(), which takes all the references to media in other locations (files, URLs, etc.) and "flattens" the references into a self-contained movie file, using whatever encoding and compression formats are present in the originals.

protected void saveTargetMovie() {
    FileDialog fd = new FileDialog (this,
                                    "Save target movie",
                                    FileDialog.SAVE);
    fd.show();
    if (fd.getFile() == null)
        return;
    try {
        // flatten this movie to specified file
        File f     = new File (fd.getDirectory(), fd.getFile());
        QTFile qtf = new QTFile (f);

        targetMovie.flatten (
                     0,
                     qtf,
                     StdQTConstants.kMoviePlayer,
                     IOConstants.smSystemScript,
                     StdQTConstants.createMovieFileDeleteCurFile,
                     StdQTConstants.movieInDataForkResID,
                     qtf.getName());
    } catch (QTException qte) {
        showOKDialog ("Error", qte.toString());
    }
}

QuickTime will let you paste in video of different sizes and will try to scale them appropriately in the target QTCanvas. It's generally pretty intelligent, though you can do some silly things by mixing movies with different aspect ratios, like 4x3 home movies with 16x9 movie trailers. Speaking of scaling, it's worth noting a difference between this example, where we created the GUI first, and the SimpleQTPlayer, where the movie was connected to the QTCanvas before bringing up the GUI. The movie supplies the QTCanvas with a preferred size, which was honored in the original example. In our editor, we set a size for the source and target QTCanvases, forcing QuickTime to scale the movies displayed in those widgets. In this case, we're depending on the default QTCanvas behavior of allowing resizing to any dimensions — there are several other modes described in the JavaDocs, such as resizing to even multiples of the original movie's size, for performance reasons.

Onwards and Backwards

That's it for our post-facto introduction to QuickTime for Java. You should now have a basic understanding of how to write and build simple apps with this API. Future articles will head into more of the API, but if you want to try out a few more things, check out the previous articles in the series. In Java Media Development with QuickTime for Java, which is about writing a limited JMF-to-QTJ bridge, we noted Movie.setRate() for playing a movie faster, slower, or even backwards, and included a recipe for getting the current QuickTime frame as an AWT Image. In Parsing and Writing the QuickTime File Format, we looked at the data structure that makes up a QuickTime movie, dumped its raw bytes to disk to create a playable all-reference movie, and iterated through each of the ways to save a movie to disk, from creating simple shortcut files to using MovieExporters to convert a movie to any QuickTime-supported format.

Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.

Return to ONJava.com

Comments on this article

1 to 2 of 2