Gnutella and the Transient Web03/22/2001
The Web was born in a research environment of always-on, permanently connected computers that could just as easily support Web publishing as they could facilitate Web browsing. During the subsequent popularization of the Web, consumer PCs and other devices that are transiently connected to the Internet became prevalent. These transient devices, which came and went unpredictably and held dynamically assigned Internet addresses, broke the Web's symmetry, because they were only effective at browsing.
Clay Shirky aptly characterized them as the "dark matter of the Internet" - apparent only because of their tug on servers and never luminous as servers themselves. To the lament of Tim Berners-Lee and other Web pioneers, the Web became more of a one-to-many medium than the many-to-many communication system originally envisioned.
The Gnutella protocol restores the Web's original symmetry, enabling even transient computers to effectively participate as servers. It's far from a complete solution, and alternative systems may eclipse it. Nonetheless, this simple and idiosyncratic protocol is currently in the vanguard of the emergence of the transient Web. The transient Web has the potential to be every bit as disruptive as the conventional "permanent" Web, and possibly more so.
In this article, on the occasion of the first anniversary of Gnutella's release, we'll take a fresh look at this peer-to-peer network as the harbinger of something bigger than itself: a Web that includes transient devices as servers. We'll conclude by reviewing some recent notable developments on the transient Web, including the exposure of personal data, the Mandragore P2P virus and the BearShare Defender.
Gnutella's Relation to the Web
What do Gnutella and the Web have to do with each other? Isn't Gnutella just one of many P2P file-sharing systems?
Yes, Gnutella enables P2P file sharing, but take a closer look. With Gnutella, file transfer is accomplished via HTTP, the same protocol Web browsers and servers use to transfer Web pages and other data. Under the hood, each Gnutella application contains a no-frills Web-server component for serving files and a primitive browser-like element for retrieving them.
The relation between Gnutella and the Web is therefore quite simple: Gnutella hosts are Web sites, albeit transient ones, and downloading a file from a Gnutella host is technically equivalent to fetching a file from a Web site. Most Gnutella applications combine server and client functionality into a package known as a "servent." Even users who share no files, and who instead simply use Gnutella to search and download, are running (empty) Web sites while they run servents. This would be equivalent to running a Web server each time you started your browser.
|Do you think Gnutella is the "transient Web?" Or is it too unreliable? Does the Gnutella virus indicate a fatal flaw with the system?|
|Post here to discuss the issues with Kelly, or start your own thread.|
The transient nature of the sites is, of course, terribly important. As a protocol, Gnutella's specification of HTTP for file transfer is rather mundane. After all, users have always been able to run Web server programs on their PCs, but the problem is that these sites and their contents were impossible to discover by other users due to the transience of the PCs. What's novel is that the Gnutella protocol addresses the problems of how to discover and search transient Web sites.
Why these problems deserve addressing is an entirely different story. For purposes of this article, the key point is that Gnutella can be viewed as augmenting HTTP with an additional layer of intersite plumbing that enables transient Web sites to be found and searched. This plumbing supports:
- the broadcast transmission of queries across transient sites and the routing back of responses;
- the broadcast transmission of "is anybody out there?" pings and the routing back of "I'm here" responses. For more detail on how Gnutella works, see Andy Oram's early article.
The Gnutella network is itself similar to the Web in many ways. Open and decentralized, there is no single responsible company, no central server and no single point of failure. Gnutella is a protocol for which many developers have created compatible code, and a Gnutella network exists only to the extent these programs are running and communicating with one another. There is a general public network, and private networks can co-exist in isolation or attached to the public one. Because of this, Gnutella application developers feel themselves akin to makers of Web browser, server, proxy and other applications. They are builders of interoperable software for a network larger than the sum of its parts. In fact, they are makers of transient-Web applications. The Gnutella network is simply one form of the transient Web.