OpenP2P.com    
 Published on OpenP2P.com (http://www.openp2p.com/)
 See this if you're having trouble printing code examples


Feed Your Head at Etech 2003

by Daniel H. Steinberg
05/01/2003

Editor's Note -- As the Emerging Technology Conference came to a conclusion on Friday, April 25, 2003, many of the participants were still just as enthusiastic as they had been on day one, and the sessions were as full as ever. Daniel Steinberg was there from the beginning, and he's pulled together some of his favorite observations for this article.

A Feast for the Mind

Friday morning, attendees at ETech have that same look they wear when dessert is announced after a huge Thanksgiving dinner. They've already consumed so much. They just want to sit quietly with a cup of coffee and digest everything while talking to friends about this shared feast. On the other hand, just like the pumpkin and apple pies, the remaining sessions and events do look good. Perhaps just a taste. Which one? A little of both, please. Whipped cream? Oh, sure.


Lunch went beyond merely satisfying physical hunger; it was also a time to feed the mind. [Photos by Derrick Story.]

And like Thanksgiving dinner, the ETech attendees are busy packing up the leftovers. They are exchanging cards and email addresses, downloading presentations, and beginning conversations. From hardware hackers, to sociologists, to developers, to journalists, to--you name it, this has been a gathering designed to promote synergies. Tim O'Reilly and Rael Dornfest followed the voice that whispered in their ear to build it and trusted they will come. This isn't heaven, it's Santa Clara. Here are some of the leftovers I've packed.

Sociologist Speaks to Geeks

Marc Smith is a sociologist trying to understand the social aspects of online communities. Smith works in Microsoft Research's Community Technologies Group studying social cyberspaces. He's trying to answer questions about places where groups meet for collective action that spontaneously emerge. In particular he's interested in these collections that are more than the sum of their parts. That links back to the Tuesday morning keynotes of Bonabeau and Rheingold. As these ideas recur and reinforce each other themes emerge at this conference that is more than the sum of its parts.

O'Reilly Open Source Convention.
O'Reilly Open Source Convention.

Smith does not have an easy task. You can walk into a room and make a quick and fairly accurate assessment of the current state of the room. Sure, you don't know what happened before you got there, but others know that about you. When you enter a location on the Web such as a newsgroup or a chat room, you don't have many tools for determining what's going on. Smith is working to change that experience.

If you try to get information on a newsgroup you can get a list of previous posts that include the subject, the sender's login, the date and time the message was posted and its size. Smith notes that this developer created summary of a newsgroup "is an interface to a social space defined by the most antisocial people on earth."

He asks the audience how they might determine which of the Buffy the vampire slayer groups would be the best one to join? As a first pass, he presents an interface that allows you to enter a fragment of a newsgroup name. You will then see a list of newsgroups together with statistics that include the numbers of posts, posters, returnees, lines per post, replies, repliers, and unreplied to posts. This information can be useful. For example if four hundred of the fifteen hundred posters were active the previous month, you get a feel for the size of the core group. Smith says that generally the core group is about 2 percent of the size of the group and they generate more than 60 percent of the content.

Similar statistics for individuals tell you something about them. For example, some authors can be classified as "answer people". Look at the percent of posts that are replies. A high ratio of reply to posts may indicate an answer person. How many different threads is the author posting to. A high thread to post ratio again helps identify an answer person. An audience member raises the question that these might also be characteristics of spammers. Smith agrees but says that other metrics help sniff out spammers. For example, someone posting to more than one hundred newsgroups is unlikely to be a real person. He acknowledges that there will be false positives in identifying answer people but wants to take advantage of the mostly correct information that can be divined from newsgroup usage.

Other metrics are being used to visualize the activity of newsgroups using a tree map. The tree provides insights into the structure of message threads. You can see how many days a particular person has been active. Smith says that 67 percent of the people post only once. Again a small minority of 2 percent post more than three days a month. You can display information for inter-newsgroup or intra-newsgroup.

"The key to a healthy community," Smith says, "is the ability to get other people to come back, be engaged, and interact." Microsoft has identified metrics for measuring the health of a community. To measure the retention of leaders they look at the average number of days that the top forty authors are active. Interaction is healthy when the percent of posts that are replies is high and when the percent of posts that have no reply is low. Size and growth can be measured by the number of posts and the percent change, but Smith asks whether continued growth is always desirable. You can also measure the topical focus by looking at the percent crossposted. The speed of the time to reply is also crucial and can be measured by the number of questions closed and the average time to closure.

You can check out the early stages of this work at http://netscan.research.microsoft.com. You won't see some of the more visual tools yet. Smith shows the audience visualizations of newsgroups. Recency is indicated by color, the number of posts is indicated by posts. The vertical position in the window corresponds to the number of days active and the horizontal axis corresponds to the number of posts per thread. The result of this visual mapping is striking and immediate. You can get a feel for a community and determine whether or not you will feel at home there. Other visual tools include one that indicates on a per author basis the number of threads the user posted to that were or were not initiated by the author. The Netscan data can be used by people to evaluate themselves, others, and their newsgroups.

Dynamically Connecting Non-Geeks

It's just short of a year since Apple announced Rendezvous, a robust dynamic service discovery mechanism. As Rendezvous architect Stuart Cheshire describes it, the technology is quite straight forward. The big idea is that the industry has settled on IP to connect devices over wide area networks. For more local connections, however, we have a sea of protocols. The two questions Cheshire asks the audience are, "Why not settle on a single protocol?" and "Why not settle on the same protocol we use in wide area networks?"

Accepting that premise, there are a few issues that need to be resolved. First, when you use IP you either have a dedicated IP address that has been assigned to your device or you use DHCP. On a local network you would like to remove the requirement of DHCP. The strategy is self-assigned, link-local addressing. Choose any IP address in a range set aside for this purpose. Ask if anyone else on your local network is using that address. If not, then that is your address. If someone else is already using it, choose another address and try again.

The second issue is naming. You don't want end users to have to use IP addresses to locate services. A more user friendly solution is human readable names. There is no DNS server to resolve names to the IP address that was self-assigned in the last step. The solution is similar. Pick a desired name in the .local subdomain. Issue a query to see if someone else is already using it. If so, pick another. If not, the name is yours.


Stuart Cheshire's Rendezvous talk was right in step with the laptop toting, wireless networking audience.

If names are self-assigned and IP addresses are self-assigned, then the next step is to support browsing so that you can locate the services you are looking for. Devices already need to implement mDNS, so Rendezvous uses standard DNS capabilities to identify services. Cheshire offers an example of searching for a printer by issuing a DNS Query to find a device that supports ipp (the Internet Printer Protocol): _ipp._tcp.local.PTR ? . You will get back a list of records and the first part of each name you get back is the user friendly name for that service.

Cheshire presents three demos. He connects a netcam. Because he happens to know the name of the netcam he is able to type its .local name into his browser and with no configuration the camera's output is displayed in the browser. As a second demo he connects a Brother printer with the Rendezvous services browser open and as soon as the printer comes to life it announces itself to the network and is visible in the browser under available printers. Cheshire remarks that Rendezvous has many of the ease of use features of AppleTalk while resolving some of the limitations and while depending on widely adopted standards.

As a third demo, Cheshire started the Hydra application and opened a document. Within moments, half a dozen audience members had started up their versions of Hydra and connected to his document. While he returned to his presentation, those of us with Hydra continued to create and edit a document in parallel. One person uploaded an ASCII image of a face. Another person added a pupil to one eye and a different person added a matching pupil to the other eye. Emerging networks. Collaborating communities. The whole is much greater than the sum of the parts.

Kay Issues a Challenge

Tim O'Reilly introduces Alan Kay to the morning keynote audience with a Kay quote that "it is easier to invent the future than to predict it." Kay cautions that his talk begins with a bit of complaining, but says that the last 20 years have been pretty boring in the world of computing. Given the glimpses into the future that we're getting, that doesn't sit well with the audience. Kay says that the problem is that after personal computers got commercialized, a lot of vendors targeted businesses and businesses aren't that creative about using any tools. Businesses are prime examples of learning a system and sticking with it.

Kay's first principle is sticking with children. He asserts that most people are instrumental reasoners. Their judgment of any tool or idea is seen in terms of the current goal structure. Kay says "adults basically have too much context. Many things work pretty well and pretty well is the enemy of qualitative improvement and vice verse."


"It is easier to invent the future than to predict it," Alan Kay

For those who are pretty satisfied with the current state of software, Kay showed movies of applications from forty years ago. He showed a demo of Sketchpad from 1963. Ivan Sutherland looked at an awful display and asked the critical question, "What else can it do?" He ended up creating an object oriented system that could be used easily for prototyping and along the way invented computer graphics. Kay continues his history lesson with example after example of applications that ran well over thirty years ago on constrained computing devices and displays and were responsive and innovative.

Kay next demonstrates some of the work he is doing with children and Squeak. He shows how easy it is to design a car and give it behavior. He then quickly sketches a steering wheel and connects the car to the steering wheel so that he can use the steering wheel to control the car. Kay walks through an example of children exploring the acceleration of an object due to gravity. Students film an object falling and pull out every fifth frame. They use objects to measure the changes in velocity as the object falls and then create a model of a falling object that they run side by side with the original video to verify their model.

Kay and David Smith demonstrate Croquet, an immersive three dimensional world. Kay describes this as a broadband conference call. Using p2p everything one does is shared and replicated. As Kay navigates around the space his character comes in and out of Smith's view. They interact with each other and with the space. When one of them creates a fish that he introduces into the world, the other sees the same fish and both can follow its movement. This is the modern day incarnation of Kay's Vivarium project. Imagine a Rendezvous enabled Croquet. At next year's ETech, attendees could be navigating a common virtual space in addition to instant messaging each other.

Just Hanging Out

It's no secret that when groups of people come together to hear interesting presentations, the greatest value is usually derived from unscheduled conversations. A group that forms to share a power strip is soon talking and sharing ideas and finding common interests. Two or more people find themselves grabbing a cup of coffee or sitting at the bar comparing notes from the outstanding talks of the day. Speakers come to these conferences expecting to interact with other attendees. Everyone is generous with their time and ideas. From the skeleton of the scheduled conference, the full conference emerges.

Three people heading off to dinner soon came a dozen (Bunnie meet Possum, Possum this is Bunnie). Conversation starts with Bunnie Huang's cool tutorial on Hardware Hacks where the dinner party began to form. Soon the conversation turns to the DMCA and what sort of hacks are legal. Tim O'Reilly and Cory Doctorow argue the implications of DRM in the light of the DMCA with those seated in between looking from one to the other like the audience at a tennis match.


Greg Olin (left) chats with Brent Simmons and James Duncan Davidson at the Mac OS X Innovators reception sponsored by Apple Developer Connection.

At the reception for the Mac OS X Innovator Contest awards, Dan Wood, creator of Watson, stops over to talk to Brent Simmons who has just won the award for his NetNewsWire application. James Duncan Davidson and Greg Olin banter ideas back and forth about applications they are working on. Greg organizes an impromptu strategy session with James and Brent to further the discussion. Robb Beal, the creator of Spring, who placed second in the contest joins the conversation. Spring, Watson, NetNewswire each are individually important applications created by single developers. Taken together (and they can be purchased as a bundle) they change your relationship to the Internet from being just browser driven.

There were plenty of leftovers that could have been packed, but they would have just sat in some note on a laptop--the metaphorical refrigerator. By the time they would have been remembered they would be showing signs of age and other more recent leftovers would have been stored away as well. You just have to trust that collectively, nothing will go to waste. Someone else will take home the yams and the stuffing. Just a little turkey and perhaps some cranberry bread.

Daniel H. Steinberg is the editor for the new series of Mac Developer titles for the Pragmatic Programmers. He writes feature articles for Apple's ADC web site and is a regular contributor to Mac Devcenter. He has presented at Apple's Worldwide Developer Conference, MacWorld, MacHack and other Mac developer conferences.


Return to OpenP2P.com.

Copyright © 2009 O'Reilly Media, Inc.