Wireless DevCenter    
 Published on Wireless DevCenter (http://www.oreillynet.com/wireless/)
 See this if you're having trouble printing code examples


Home Monitor on a Cell Phone

by David Fox
07/23/2001

David Fox won the Motorola-Nextel Developer Challenge at JavaOne by whipping up a program that lets you control lights and appliances at home via cell phone, and stream webcam images to its tiny screen. Oh, and, he did it in six hours, and drove away in a BMW Z3 for his efforts. Here's how he did it.

Probably there's something wrong with me, but I'm telling you the truth: I was in it for the fun, not the BMW.

I live in New York, and have never even owned a car. Somehow, though, I was lucky enough to come home from JavaOne with a toylike, two-seater convertible. This is how it happened.

Before the Contest: Planning the App

The contest was simple enough -- write an application, any application, for the new Motorola i85s cell phone. Oh, and do it, from design to prototype to build to deploy, in six hours or less.

The challenge was part of this year's JavaOne, sponsored by Motorola, the phone maker, and Nextel, the exclusive carrier service. Twenty-five computers were set up, with twenty-five cell phones attached. At the back of the room were a platter full of sandwiches and brownies, a silver bowl full of soda and beer, and a bunch of Java techies to help answer questions. There was also a chair set up for deep-tissue massages. First prize was a new BMW Z3 convertible and two i85s with a year of free service.

So as challenges go, this was no triathlon or Everest climb. Some people, I'm sure, were just there for the free massage and food. But having competed in a similar competition the previous year, I knew that endurance was still an issue. It was far too easy to fall into the trap of coding monstrously for two or three hours and then getting frazzled.

Like many programmers I know, once I get in the zone of coding, time has no meaning. I can go for hours, and in some cases days, without distractions like food, water, sunlight, hygiene, or human contact.

But I knew that I had to be careful of burning out too quickly. I had to pace myself… I told myself that I'd spend 55 minutes of every hour in the trenches, and be sure to take 5 minutes off to refocus my eyes, crack my knuckles, pound down a few brownies, and pace around the room.

Now, the i85s is something of a trailblazer -- the first United-States available phone to support Java 2 Micro Edition, or, as it is called by its friends, J2ME. Luckily, I'd played around with J2ME before, in my role as a multiplayer game developer. I knew a little about its far reaches, and its limitations. I've tried in the past to create big, entertaining experiences on those tiny screens. But it's very difficult to port, say, Tomb Raider or Quake onto a cell phone. I did believe, however, that there was a lot of potential for text-based games and games that use the strengths of an always-on, always-with-the-player wireless connection.

Figure 1. A screen shot from the mobile phone emulator application shows an image of the home map.

For example, the request might look something like this:

http://www.myhouse.com/x10controlservet?lighta=off&heaterb=on

When I found out about the challenge, I began asking myself what I could do that would actually be fun, sexy, useful, and possible to pull off in six hours. Naturally, my first thought was to do a game. But it just so happened that I was tinkering around with a home automation system that day. Now, I don't mean to make my apartment sound like something out of The Jetsons, but over the years I've set up a few X10 <http://www.x10.com> Firecracker modules into my lamps and appliances, allowing me to turn things on or off from any room in the house. I've also played around with wireless webcams from X10 and other companies -- cameras that grab frames of video and stream them to the PC. It's fun stuff, though it's often more for the purposes of impressing friends than anything else.

For the past few months, a side project of mine has been to play with servers such as JShape X10, a program that interfaces with X10 Firecracker modules. JShape's server also comes with a WAP front end, which allows you to turn lights on and off from a cell phone. But controlling so much information on a small, slow phone with a limited language like WAP just isn't too appealing.

Then the idea hit. With the advent of the i85s and J2ME, not only could the light-and-appliance interface look snazzier and be much more responsive, but I wondered if it would be possible to go a step further and use J2ME's graphical capabilities to actually stream the frames from a webcam across the Internet and directly onto the phone. Not only would you be able to flick on your A/C on your way home from the office, not only could you turn off all your downstairs lights from upstairs and save on energy bills, but you could spy on the nanny, or keep an eye on the teenager during your European vacation, or catch burglars in the act, or (and I don't by any means recommend this) keep tabs on a wayward spouse. I would call it Home Monitor.

Writing From Scratch is Noble, but . . .

JavaOne 2001 was the usual circus. I ducked away from the hoards of Java developers, into the room where the challenge was being held. It was hard to take six hours out of my frantic conference schedule, but I was ready and willing.

It began without much fanfare. I and 24 other programmers wandered around, like dogs sniffing for their sacred spots. The feng shui felt good in one corner, and there I sat.

The first half hour or so was spent getting familiar with the territory, tools, and techniques. The machines were set up with the Java Micro Edition of Metrowerks Code Warrior, which seems by all accounts to be a good development tool. But I was used to WordPad along with a quick and simple J2ME Wireless Toolkit from Sun -- nothing more than a console that let you compile, pre-verify, and package your MIDlet files and then test them out on various emulators. The files are called MIDlets instead of Applets, by the way, because the phones run a special form of J2ME called the MIDP (Mobile Information Device Profile).

Although writing from scratch is noble, time was short. It made much more sense to take a good, working piece of code and use that as the framework. I scanned the various sample applications available for us to look at and stole one. I tore it apart until it was nothing more than a glorified "Hello World" program. But I knew, at least, that I had something workable.

Before going too far, I also wanted to play with the process of getting the MIDlet off the computer and onto the phone. Each phone was attached to a PC using a simple serial cable. Sun's Wireless Toolkit takes all your MIDlet classes, graphics, and other resources and combines them into one JAR file and a JAD, a short descriptor file that summarizes what's inside the JAR. Using Motorola's software, it was crazily easy to connect to the phone and upload the JAR and JAD.

The interface of the phone itself is pretty intuitive. You just hit the Menu button, and the second option down is Java Apps. You scroll down to the application you want, hit the Select button, and off you go.

So by now I'd wasted about an hour. It was time to rock.

Figure 2. Menus guide a user through choices of lights and appliances to control.

Building the Controller

The key to writing a user interface in J2ME is to stick to the basics as much as possible. MIDP makes it really, really easy to assemble forms using stuff like radio buttons, check boxes, and text fields. But if you want to get a little trickier and create your own components from scratch, you're going to have lots of debugging time ahead of you.

It didn't take long to set up a hierarchical menu system that lets you choose which room you want to control, and then lets you choose which light or appliance you want to mess with. For the sake of this challenge, everything was hard-coded to suit my apartment, my lights, my air conditioning, and my Mr. Coffee. A real application, of course, would be much smarter. It would fetch an XML properties file from a server and craft the interface dynamically, so that you could hit one server and control a huge office building with complicated machinery, and hit another server to control the porch light at a log cabin.

Each appliance was set up as a check box. If the box was checked, the appliance was already on. You could check or uncheck any box you wanted, and then select the "Do It" command. The program would figure out what changes you made and then send them via HTTP as parameters to a simple Servlet.

It took a few hours to get all the kinks out, and to handle some refresh bugs. But it was all working by hour four. You could turn on or off any appliance, and your changes would be reflected on the server. Of course, the server wasn't actually attached to anything. It was just a proof of concept. It would have been really impressive to use the cell phone to turn off all the lights throughout the Moscone Convention Center, but I probably would have gotten arrested.

More or less happy with the results, I grabbed a few brownies, drank a few more Cokes, and felt the caffeine and sugar surge like a little army through my veins. I hopped around the room, peeking at the competition. Most people had their noses inches away from the screen, their fingers fluttering across the keyboard, utterly rapt with what they were doing. One guy had his head between his knees, groaning. In other words, it looked like a typical development house.

The Extra Mile: Adding the Web Cam

So I had less than two hours left, and I wanted to attempt the really fun part: the streaming webcam. A typical webcam will create a new image -- say, myimage.gif -- every second or so. It will then copy that image to a public Web server.

Since MIDlets can only display graphics using the PNG file format, I figured that all I had to do was convert myimage.gif, on the fly, to myimage.png. At the same time, I could strip out the color information and reduce the image so that it fit on the tiny 110x85 screen. By making the image as small as possible, it would be possible to retrieve each frame in minimal time.

Of course, I didn't have time to do any of this. So I just converted and resized a few images by hand and put them up on a server. The images were a series of a young girl, looking into her webcam, smiling and talking.

I'd figured that this part would be pretty easy. After all, I could just use the createImage method and point it to the URL in question. But it didn't work like that. The createImage method allows you to load up a local PNG, for example:

createImage(String filename);

And it allows you to create an image from a byte array:

createImage(byte[] imageData, int imageOffset, int imageLength);

But it doesn't allow you to input a URL as a parameter.

Figure 3. Figuring out how to serve the webcam images to the mobile phone was frosting on the cake. Somehow, it worked on the first try.

There are other methods in the MIDP Connector class that let you retrieve a URL. But then what? You can't exactly download the image, store it on the hard drive, and then load it up again. After all, there is no hard drive on a cell phone. And Java's security sandbox forbids storing foreign files directly in memory.

I was about to give up. After all, there was only about an hour and a half left and I thought I had a reasonably cool Home Control app. But I couldn't see any way of getting the image off the server and squeezed into the phone. So I went and got a massage.

The masseur was a hell of a nice guy, very talkative, into yoga, art, raves, and other nifty San Franciscan things. He also did wonders to my every joint and muscle, not missing a spot.

As I wobbled away from the chair, I had a stupid, crazy idea. I realized that while it wasn't easy to download an image directly, I could download the binary contents of the image file. I could then feed that byte array directly into the createImage method. It took about a half hour, but I came up with something like this (it definitely wasn't commented or formatted as nicely!):

I tried it out and somehow, I'm not sure how, it worked. First try!

Now I had to tie it all together. I quickly hacked together an inner class called VideoTimer, which called a nextImage() method every few milliseconds:

class VideoTimer extends TimerTask {

    VideoTimer()
    {
    }
    
    public void run() {
	try {
	    nextImage();
	} catch (Exception ex) {
	    System.out.println("Exception " + ex);
	}
    }
}

The timer was then triggered like this:

timer = new Timer();
vt = new VideoTimer();
timer.schedule(vt,(long)10,10);

And voila! I now had a webcam streaming down to my phone!

Conclusion

The contest was over. I went home and tried to sleep, but my brain was still in heavy programming mode, with Java code leaking behind my eyeballs like sheets of rain on a window. I thought of all the cool features I couldda, shouldda, wouldda added: multiple camera configurations, smart device detection, image conversion on-the-fly, and a more robust and dynamic menu system.

The next day I received a call: I was one of the top six finalists, and would I kindly present my app to a panel of judges? The presentation happened quickly, and I remember it in a blur. The audience and judges seemed to appreciate the home automation part, but they really liked the streaming webcam. I think seeing a live image flicker on the tiny screen may have helped open up many people's imaginations, helping them realize that all this talk about Java and embedded systems isn't hype -- not all of it, at least.

When they announced me as the winner, I was in awe. I was happy for the prize. But more so, it was exciting for me to affirm that by focusing, preparing, and persisting you can use today's technology to make very cool things happen in very short amounts of time.

Copyright © 2009 O'Reilly Media, Inc.