ONJava.com -- The Independent Source for Enterprise Java
oreilly.comSafari Books Online.Conferences.

advertisement

AddThis Social Bookmark Button

Streaming QuickTime with Java Streaming QuickTime with Java

by Chris Adamson, author of QuickTime for Java: A Developer's Notebook
01/12/2005

This isn't an excerpt from my soon-to-be-released QuickTime for Java book, though now I wish it was.

You see, the thing is, a lot of us in the QTJ world had long assumed that the streaming APIs in QTJ were broken, and I didn't want to commit to covering something that was broken. OK, I went ahead and did so by covering capture, which various people got working through different techniques, but I didn't want to do it twice. Besides, the case against streaming seemed particularly bad. No one could get its demo code working--this post to the quicktime-java list is typical of many users' desperation with getting Apple's AudioBroadcaster and DrawableBroadcaster demos working. To make things worse, one of the demos depended on a GUI preview component class that was taken out of QTJ in version 6.1 as QTJ radically scaled back its GUI offerings, offering components only for Movies, MovieControllers and GraphicsImporters, not streaming Presentations, video capture, or certain graphic niceties like "composited" content created from multiple sources. So, given official demos that didn't seem to work in the first place and that now had key classes deprecated (and throwing RuntimeExceptions if they were actually used in Java 1.4), the prognosis for actually streaming content with QTJ 6.1 seemed very bad.

Which is why I'm delighted, and more than a little surprised, to report that streaming actually does work in QTJ 6.1. In this article, I'll introduce the basics of simple webcasting with QTJ.

Requirements

QuickTime's streaming API, represented in Java by the package quicktime.streaming works only on Mac OS (Classic and OS X). The classes are present in QTJ for Windows, but they won't do anything. However, you can use QuickTime for Windows as a client for a stream, and if running in Java isn't critical, you can get the Darwin Streaming Server, an open source project that's runnable on Windows 2000 Server and 2003 Server, as well as Solaris 9 and Red Hat Linux 9.

Related Reading

QuickTime for Java: A Developer's Notebook
By Chris Adamson

The simplest thing to stream with QuickTime is live content, which is what I cover here, so you'll at least need an audio input device, such as a built-in mic or a headset. Having a QuickTime-supported camera, like an iSight, is far more impressive, of course.

What Streaming Is, What It Isn't

Given the overloading of the term "stream," it's not easy to nail down what the term "streaming" even means. For example, QuickTime has long supported a "fast-start" feature--a movie could start playing if QuickTime determined that it had enough data that it could start playing and would not run out of data at the current download rate--that some people mistook for a form of streaming. And to be sure, that has its advantages: it's easier to set up and it ensures that all packets get to the client. But real streaming, that is to say, streaming that corresponds to Internet Engineering Task Force (IETF) standards, is a different matter entirely, not supported by QuickTime until QuickTime 5, and not exposed to Java until QTJ 6.

This form of streaming allows the server to control the transmission, and is heavily optimized to keep things running in real-time. Clients don't have to download potentially huge files, and this approach is particularly well-suited to live broadcasts. In fact, QuickTime's streaming uses two types of "real-time" streaming protocols: Real-time Transport Protocol (RTP) to send packets of media data, and Real-Time Streaming Protocol (RTSP) for control information. RTP uses the potentially lossy UDP, so all parties are designed to tolerate the dropping of packets during a transmission. This means that clients have to gracefully handle not getting all of the data for a video frame or an audio sample. This is preferable to a TCP/IP-based approach, which could make an indeterminate of retries (and thus take an indeterminate amount of time) to get a missed packet.

Presentations and SDP Files

In QuickTime, streaming is based around the idea of a Presentation, represented in QTJ as the class quicktime.streaming.Presentation. In many ways, it is the streaming equivalent of a Movie--where a movie might have audio and video tracks, and a collection of metadata to tie it all together, the presentation will combine some metadata with multiple audio and video streams. It's also worth noting that audio and video are the kinds of media you're most like to stream, since certain other media types supported by QuickTime (sprites, Flash content) do not handle lost packets well, and are thus not well-suited to streaming.

You might have guessed that to set up a stream, you'll need to create a Presentation and kick it off. But how? The most common way to do so is to create a Session Description Protocol (SDP) file, and toss it over to the static factory method Presentation.fromFile(). The SDP file is in a reasonably straightforward text format, defined by RFC 2327 and several updates. I find this a lot easier in theory than in practice, but let's worry about implementation details later. Here's an SDP used by some of Apple's streaming examples, and in Tim Monroe's QuickTime Toolkit Volume Two:


v=0
c=IN IP4 224.2.1.2/20/1
m=audio 1000 RTP/AVP 12
m=video 2000 RTP/AVP 101
a=rtpmap:101 H263-1998

To explain this file, line by line:

  • v=0: This is the version number of SDP. The version here is 0 and there is no minor number used in SDP.
  • c=IN IP4 224.2.1.2/20/1: This provides information about the connection used by the presentation. IN IP4 means an IPv4 Internet address. 224.2.1.2 is the address (note that this is a multicast address, so multiple clients can connect to the broadcast), 20 is the time-to-live, and 1 is the number of contiguous multicast addresses to use.
  • m=audio 1000 RTP/AVP 12: The m= lines define the streams used by the broadcast. This one is obviously audio, sent via RTP to port 1000. The 12 defines the payload type, in this simple QCELP audio. Some of these are defined in RFC 3551.
  • m=video 2000 RTP/AVP 101: This media line defines a video stream, to be delivered by RTP to port 2000. Payload types are only defined up to 95, so the use of 101 means that the video format wasn't given a payload type in the original RFC, and instead it will be mapped to a well-known constant later in the SDP.
  • a=rtpmap:101 H263-1998: This completes the dynamic payload typing indicated on the previous line. To use one of these types, you use a value between 96 and 127 (101 in this case) and then a string to name the payload type (H263-1998).

This is all well and good, but when I used it with my sample program, I got a video stream but no sound. So I used a very different SDP, one that was originally provided with the QTJ DrawableBroadcaster demo. Yes, the one they deprecated:


m=audio 2656 RTP/AVP 96
c=IN IP4 239.60.60.60
a=rtpmap:96 x-qt
m=video 2700 RTP/AVP 96
a=rtpmap:96 x-qt

The big difference here is that both audio and video use the same dynamic payload mapping, and it's not to a real codec, but the genericized x-qt. The win here is that you can pick any of QuickTime's audio and video codecs at runtime, instead of forcing the issue in the SDP file. The downside is that this may not be parsable by non-QuickTime clients, whereas using sufficiently standardized and/or old codecs and specifying them in the SDP makes it more likely that other clients (Real, JMF, etc.) will be able to handle your stream.

So there's your SDP file. Now to make a Presentation out of it.

Creating the Presentation

My streaming server program is called LittleBroadcast, and it's really not a lot of code, just over 140 lines. I'll step through it in this article, commenting general sections, but providing its full listing. It's also available in a .tar.gz file, along with the SDP file and an Ant build file, in the Resources section below.


package com.mac.invalidname.qtjstreaming;

import quicktime.*;
import quicktime.std.*;
import quicktime.util.*;
import quicktime.qd.*;
import quicktime.io.*;
import quicktime.streaming.*;
import quicktime.app.time.*;

import java.io.*;
import java.awt.*;
import java.awt.event.*;

public class LittleBroadcast extends Tasking implements ActionListener {

This is the typically long list of QuickTime imports, including qd in order to use its QDGraphics to provide an offscreen drawing surface for the camera, io to read the SDP file, streaming for the streaming API, and time to pick up some tasking utilities to give the Presentation time to run. On that last point, notice that the class extends Tasking--that provides a task() that's called back periodically. For this application, it's used to continually call the Presentation's idle() method and give it some cycles to work with. You'll learn in the book is that the same thing is needed for Movies, but the tasking is almost always handled for your automatically. No such luck with Presentations. (Or capture, for that matter, but I digress.)


    boolean broadcasting = false;

    public static final int BROADCAST_WIDTH = 176;
    public static final int BROADCAST_HEIGHT = 144;

    Button startStopButton;
    Button configButton;

    Presentation pres;
    int presenterTimeScale = 600;

These are the server's instance variables. broadcasting is a flag used to determine what to do when the start/stop button is pressed. Next is a pair of constants for the size of the broadcast video, followed by the buttons for the server GUI. Finally, there's a Presentation object, and its timescale. (The time-keeping system for the media, where a timescale of 600 means there are 600 units in a second; 600 is a common default in QuickTime.)


    public static void main (String[] args) {
        System.out.println ("main");
        try {
            QTSession.open();
            new LittleBroadcast();
        } catch (QTException qte) {
            qte.printStackTrace();
        }
    }

There's nothing particularly special about this main. I put all of the smarts in the constructor in case I needed to create an inner class, for which I'd need a instance. Maybe you'll find that useful if you extend this code.


    public LittleBroadcast() throws QTException {
        System.out.println ("LittleBroadcast constructor");
        QTFile file = new QTFile (new File ("little.sdp"));
		try {
			MediaParams mediaParams = new MediaParams();
            mediaParams.setWidth (BROADCAST_WIDTH);
            mediaParams.setHeight (BROADCAST_HEIGHT);

            QDGraphics myGWorld =
                new QDGraphics (new QDRect (
                    BROADCAST_WIDTH, BROADCAST_HEIGHT));
            mediaParams.setGWorld (myGWorld);

			PresParams presParams =
                new PresParams( presenterTimeScale,
                            QTSConstants.kQTSSendMediaFlag | 
                            QTSConstants.kQTSAutoModeFlag |
                            QTSConstants.kQTSDontShowStatusFlag,
                            mediaParams );
			pres = Presentation.fromFile(file, presParams );

The first thing the constructor does is to load an SDP file called little.sdp. But this isn't all it needs to create the Presentation--a number of parameters need to be set by the server application for use in the Presentation.fromFile() call. First, you create a MediaParams object, on which you can set the video height and width. Another important thing you have to do is to provide the camera a drawing surface, which you do by creating a QDGraphics and setting it on the MediaParams. Yes, the names are weird, because the QTJ designers wanted to stress the similarity to an AWT Graphics object, but every method that gets or sets with such an object uses its native-API name, GWorld. Finally, you create PresParams to set parameters for the whole Presentation. This takes a somewhat arbitrary timescale, some behavior flags mathematically OR'ed together, and the MediaParams. The possible behavior flags, all defined in QTSConstants, include:

  • kQTSAutoModeFlag: Use defaults for everything. Most importantly, this uses the default Sourcer, the source of the Presentation, which is the SequenceGrabber that performs capture from the various input devices. It's also possible to broadcast a QuickTime file on disk or arbitrary content; I'll revisit these topics in the future.
  • kQTDontShowStatusFlag: Don't create a streaming status handler, which would cause the connection and status information to always be shown on the client side.
  • kQTSSendMediaFlag: Send, don't receive data.
  • kQTSReceiveMediaFlag: Receive, don't send data.

With the SDP file reference, parameters, and GWorld set up, you create the Presentation with Presentation.fromFile().


            // find audio stream
            Stream audioStream = null;
            for (int i=1; i<=pres.getNumStreams(); i++) {
                System.out.println ("stream: " + i + ": " + 
                                    pres.getIndStream(i));
                Stream aStream = pres.getIndStream (i);
                if (pres.hasCharacteristic(aStream, 
                    StdQTConstants.audioMediaCharacteristic)) {
                    audioStream = aStream;
                    break;
                }
            }
            System.out.println ("audioStream = " + audioStream);
            pres.setVolumes (audioStream, 100, 100);
            System.out.println ("created presentation, gworld == " +
                                pres.getGWorld() + ", size == "+
                                mediaParams.getWidth() + "x" + 
                                mediaParams.getHeight() + ", streams == " +
                                pres.getNumStreams());  

This isn't genuinely necessary, but it shows you how to tour through the Presentation to pick out individual streams. Presentation.getIndStream will return a Stream by index (note that QuickTime indexes are, as always, 1-based). It iterates over these asking for the audioMediaCharacteristic to find the audio stream (for video, you'd ask for the visualMediaCharacteristic). This example sets the volume on the audioStream, for both the left and right channels, to its maximum value of 100.

Finally, the println dumps a bunch of interesting metadata from the Presentation and the MediaParams.

Configuring the Presentation


            SettingsDialog sd = new SettingsDialog (pres);
            System.out.println ("Did settings");
            pres.preroll();
            broadcasting = false;

These are the final steps to set up the presentation. The SettingsDialog presents the user with a choice of input devices for audio and video (the two streams the SDP file specified as being in the Presentation). Each stream can also be customized with a compressor (MPEG-4, Sorenson Video 3, H.263, etc.) and a packetizer (which is sometimes dictated by the compressor; watch to see if it changes automatically when you change compressors). An example of this GUI is shown in Figure 1.


Figure 1. SettingsDialog for a Presentation

In the pictured case, the audio has defaulted to the computer's line in. To change it to the iSight, you'd click the Source button, which brings up a device selection list, as shown in Figure 2.


Figure 2. Source selection dialog

The last thing to do is to call Presentation.preroll(), which, like Movie.preroll(), gives the Presentation an opportunity to pre-allocate resources and get ready to begin streaming the Presentation.

Pages: 1, 2

Next Pagearrow