ONJava.com -- The Independent Source for Enterprise Java
oreilly.comSafari Books Online.Conferences.

advertisement

AddThis Social Bookmark Button

Java Media Development with QuickTime for Java
Pages: 1, 2

Loading a Movie

To create a Movie in QuickTime, you can call the new Movie() constructor to create a new Movie in memory, one that you'd then add tracks to, but for our purposes, we want to quickly get a Movie loaded with the media from some file or URL. The most general way to do this is to use the static Movie.fromDataRef() method, which takes a DataRef argument. The DataRef class is a generic reference to a media source — a file, a URL, even a location in memory in the form of a QTHandleRef (and you didn't think Java had pointers!).



The second argument to the Movie.fromDataRef() is an int called flags. The use of behavior-modifying flags, typically combined with logical ORs when you're using more than one, is seen throughout QuickTime, and it can be very aggravating to the Java programmer, as appropriate values are almost never detailed in the javadocs. In the case of Movie.fromDataRef(), the method's javadoc documentation does recommend the newMovieAsyncOK flag (defined in StdQTConstants4, one of several massive classes that define pseudo-consts), but doesn't mention the useful newMovieActive flag that is commonly set for this method. For the real story about appropriate flags, you have to see the method's native API documentation, and I suppose we're lucky to have the convenient hyperlink from the javadocs, as the full QuickTime API reference currently weighs in at a table-cracking 3,304 pages.

But there's one more surprise when loading a movie this way. Using the newMovieAsyncOK method means that QuickTime won't block on loading media, and while that's good for user-experience reasons (we don't want to block interminably, especially if we're in Java's AWT event-dispatch thread), it has a nasty side-effect for "quick start" movies, which are non-streaming movies that QuickTime can nevertheless start playing before it has all the bytes, so long as it thinks the download will stay ahead of the playback. The problem is that when our call returns immediately, our movie has no width and no height, and won't until some amount of media is downloaded. That messes up the creation of our GUI widget (below) by defaulting it to a zero-by-zero size.

The native API has a method called GetMovieLoadState that we could use to tell if we have at least a little bit of the movie downloaded — a return value of kMovieLoadStatePlayable means we have enough to start playing — but the method seems not to have a QuickTime for Java equivalent (although its return values are defined in StdQTConstants5). So instead, the included code handles the problem by blocking until the movie has more than zero seconds of media loaded. This is determined by a call to maxLoadedTimeInMovie(), and while it's zero, we call task() to give QuickTime more time to load.

The Movie-loading code is below (with comments and printlns removed), and merits two more small comments. The first is that Java on Mac OS X seems to like URLs of the form file:/, while QuickTime needs file:///, so we need a workaround for that. Secondly, we call a pair of methods, prePreroll and preroll, to coax QuickTime into getting ready to play the movie sooner.

public void setLocator (MediaLocator ml) {
    super.setLocator (ml);
    String urlString = ml.toString();
    try {
        java.net.URL url = ml.getURL();
        if (url.getProtocol().equals("file"))
            urlString = fixFileURL(url);
        else 
            urlString = url.toExternalForm();
        DataRef urlRef = new DataRef (urlString);
        qtMovie = Movie.fromDataRef (urlRef,
                        StdQTConstants4.newMovieAsyncOK |
                        StdQTConstants.newMovieActive);
        qtMovie.prePreroll (0, 1.0f);
        qtMovie.preroll (0, 1.0f);
        while (qtMovie.maxLoadedTimeInMovie() == 0) {
            qtMovie.task (100);
        }
    } catch (QTException qte) {
        System.out.println ("Couldn't get a QT Movie from " +
            urlString);
        qte.printStackTrace();
    } catch (java.net.MalformedURLException murle) {
        System.out.println ("Bad URL: " + urlString);
        murle.printStackTrace();
    }
}

Getting a GUI

In JMF, you get the media into the GUI by calling getVisualComponent on the Player. If the media has images or video, this returns an AWT Component. To get a Swing-friendly component, you first call Manager.setHint(Manager.Manager.LIGHTWEIGHT_RENDERER). Getting a control widget, such as the typical time-slider with popup volume control, is done with getControlPanelComponent().

In QuickTime, the visual representation does not come from the Movie. Rather, you create a GUI component, create a Drawable from the Movie, and add the Drawable to the component with setClient(). QuickTime offers a heavyweight QTCanvas for AWT use and a lightweight JQTCanvas for Swing use, though performance in Swing is so poor that you might be better off using QTCanvas and working through any AWT-Swing collisions (one good hint from the quicktime-java mailing list: if your only issue is with JMenus disappearing under the QTCanvas, call setLightweightPopupEnabled(false) on the menus).

The Drawable that's passed to setClient comes in several different flavors for different purposes. Here are a few popular ones:

  • QTPlayer — shows media with the standard QuickTime controller.
  • MoviePlayer — same as above without the controller.
  • MoviePresenter — draws movie to an offscreen buffer (so you can apply effects) and then draws buffer to the screen.
  • SGDrawer — shows the stream from a capture device such as a webcam.

In our case, we use the MoviePlayer, using two separate QTCanvases or JQTCanvases, one for showing the movie and another for the controller, which allows us to return the movie and its controller as separate components with the JMF Player's getVisualComponent() and getControlPanelComponent().

Here's our implementation from Handler (which is what JMF requires us to call our Player implementation), which instantiates an instance variable called movieCanvas:

public java.awt.Component getVisualComponent() {
    System.out.println ("getVisualComponent()");
    Movie movie = getMovieFromDataSource();
    if (movie == null)
        return null;
    if (movieCanvas != null)
        return movieCanvas;
    try {
        MoviePlayer moviePlayer =
            new MoviePlayer (movie); 
        Object swingHint =
            Manager.getHint(Manager.LIGHTWEIGHT_RENDERER);
        if ((swingHint != null) &&
            (swingHint instanceof Boolean) &&
            ((Boolean)swingHint).booleanValue()) {
            JQTCanvas jqtc = new JQTCanvas();
            jqtc.setClient (moviePlayer, true); 
            movieCanvas = jqtc;
        } else {
            QTCanvas qtc = new QTCanvas();
            qtc.setClient (moviePlayer, true);
            movieCanvas = qtc;       
        }
    } catch (QTException qte) {
        qte.printStackTrace(); 
    }
    return movieCanvas;
}

Small JMF-QTJ differences

A few notes on areas where JMF and QTJ have similar ideas but different details:

setRate() — both APIs have a similar concept of representing the playback rate as a float value, where 1.0 is playing forward at regular speed, 2.0 is forward double-speed, -1.0 is backwards at regular speed, etc. In JMF, the rate can only be set when a Player is stopped, in effect saying "this is the speed to play at when started." In QTJ, starting and stopping is implicit in setting the rate, meaning that setRate(1.0) is equivalent to "start playing at rate 1.0, regardless of whether or not the movie is currently playing." Similarly, setRate(0.0) in QTJ is equivalent to stop(), though actually calling stop() is recommended. Another point to note is that JMF's setRate() is not guaranteed to work for any arbitrary value other than 1.0, and in practice, negative rates never seem to work with the default JMF components.

Volume / Gain — a JMF GainControl has a concept of a "mute" that is independent of gain. Like the "mute" button on a television, calling setMute(false) brings up the volume to the level it was at before you hit "mute." QuickTime's volume has no corresponding concept, so we remember the "old" volume on a setMute(true) and restore it on setMute(false).

QT time scale — JMF Time objects work with seconds or nanoseconds. QuickTime movies report duration and current position in integers, in a scale that is unique to the Movie. For example, if getTime() returns 1800, and getTimeScale() returns 600, then you're 3.0 seconds into the movie. In other words, timeInSeconds = qtTimeValue / qtTimeScale.

Grabbing an Image from a Movie

Related Reading

Java 2D Graphics
By Jonathan Knudsen

At the 2002 O'Reilly Mac OS X Conference, an attendee asked me how you grab an image from a QuickTime movie. I said I didn't remember the details right then, but that it involved drawing the bytes to an offscreen buffer and then getting the image from there. Well, half-right, and half-wrong.

Half-wrong because if all you're interested in is getting the image written to a file, it's easier than it looks. Movie has a getPict method that gets the image at any time in the movie and returns it as a Pict, a wrapper around the classic Macintosh PICT image format. Pict, in turn, has a writeToFile().

But to support JMF's FrameGrabbingControl, we have to return the grabbed frame as an AWT Image, wrapped inside a JMF Buffer, and that's where things get tricky. We need a QuickTime RawEncodedImage for transferring the bytes, so we draw the Pict to an offscreen QDGraphics, which in many ways resembles an AWT Graphics object (although the name is a trap, since Mac and QuickTime developers commonly refer to this object as a GWorld).

To get the bytes, we're fortunate that QuickTime uses a 32-bit ARGB scheme, meaning 8 bits each of alpha-channel (i.e., translucency), red, green, and blue, all packed into a 32-bit integer. We can use the DirectColorModel to represent this arrangement, and from there create an Image from the bytes in memory. (Jonathan Knudsen's Java 2D Graphics offers a full explanation of ColorModels and other low-level Image details.)

Once we have the AWT Image, we just have to use JMF's ImageToBuffer class, represented here as an instance variable called itb, to get the Image wrapped by the specified Buffer return type. Here's the complete method for grabbing the current frame:

public Buffer grabFrame() {
    Buffer convertedBuffer = null;
    try {
        // Grab a Pict from the Movie and draw it to a QDGraphics
        // note: could get a RawEncodedImage from the Pict, but
        // apparently no way to get a PixMap from the REI.
        QDRect box = movie.getBox();
        Pict pict = movie.getPict (movie.getTime());
        QDGraphics g = new QDGraphics(box);
        pict.draw(g, box);
        // get data from the QDGraphics
        PixMap pixMap = g.getPixMap();
        RawEncodedImage rei = pixMap.getPixelData();
        // copy bytes to an array
        int intsPerRow = pixMap.getRowBytes()/4;
        int[] pixels = new int [intsPerRow * box.getHeight()];
        rei.copyToArray (0, pixels, 0, pixels.length);
        // now coax into image, ignoring alpha for speed
        DirectColorModel mod =
            new DirectColorModel (32, // bits/sample
                                  0x00ff0000, // R
                                  0x0000ff00, // G
                                  0x000000ff, // B
                                  0x00000000); // ignore alpha
        Image image =
            Toolkit.getDefaultToolkit().createImage (
                new MemoryImageSource (box.getWidth(),  // width
                                       box.getHeight(), // height
                                       mod, // color model
                                       pixels, // data
                                       0, // offset
                                       intsPerRow));
        // convert Image to a Buffer (frame rate == 0 is meaningless
        // because we are not generating live video)
        convertedBuffer = itb.createBuffer (image, 0);
    } catch (QTException qte) {
        qte.printStackTrace();
    }
    return convertedBuffer;
}

Thanks to Andrew Millin and Vickie Jaffee from the quicktime-java list for the format-conversion recipe used in this method.

Conclusions

So there you have it ... a small amount of code provides a dramatic improvement in JMF's supported media formats and performance on the platforms QuickTime supports. With a similar plug-in approach, we could support other media formats; for example, by writing a JNI bridge to the recently open-sourced Helix Community code to add support for Real-format media.

On the other hand, this approach obscures the details and thus the power of QuickTime for Java, and if we're really just interested in working with QT-supported media, we should dig deeper into that API. In coming articles, that's what we hope to do.

Editor's Note -- due to reader feedback about installation issues, Chris has graciously updated the installation paragraph. We apologize for any inconvenience caused by previous versions of this article.

Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.


Return to ONJava.com.