oreilly.comSafari Books Online.Conferences.
Articles Radar Books  

Remaking the Peer-to-Peer Meme
Pages: 1, 2, 3, 4, 5

Peer-to-Peer and Devices

We've all heard popular descriptions of technologies such as BlueTooth and Jini. I walk into a room with my wireless laptop, and it queries other devices: "Hey, are there any printers here that can print a postscript file?" If this isn't peer-to-peer, what is? As we have billions of computing devices, some fixed, some mobile, some embedded in a variety of appliances (even our clothing), we'll need technologies that allow the formation of ad-hoc peer groups between devices.

As you look at these technologies, you see a great deal of overlap between what's needed for peer-to-peer on devices and peer-to-peer in areas from Web services to file sharing. Key technologies include resource discovery, reliability through redundancy, synchronization and replication, and so on.

Strategic Positioning and Core Competencies

This whirlwind tour of a new list of canonical projects allows us to tell a very different story about peer-to-peer. Not only is it fundamental to the architecture of the existing Internet, but it is showing us important directions in the future evolution of the Net. In some ways, you can argue that the Net is reaching a kind of critical mass in which the network itself is the platform, more important than the operating system running on the individual nodes.

Sun first articulated this vision many years ago with the slogan "The Network is the Computer," but that slogan is only now coming true. And if the network is the computer, then the projects under the peer-to-peer umbrella are collectively involved in defining the operating system for that emergent global computer.

That positioning guides technology developers. But there is a story for users, too: You and your computer are more powerful than you think. In the peer-to-peer vision of the global network, a PC and its users aren't just passive consumers of data created at other central sites.

Perhaps more important still is a vision of the core competencies that peer-to-peer projects will need to bring to the table. High on the list is metadata management. Whether you're dealing with networked devices, file sharing, distributed computation, or Web services, users need to find one another and what they offer. While we don't have a clear winner in the resource discovery area, XML has emerged as an important component in the puzzle.

What do we mean by metadata? In the case of Napster, metadata means the combination of artist and song names that users search for. It also includes additional data managed by the central Napster server, such as the names and Internet addresses of users, the size of the music files, and the reported amount of bandwidth of the user's Internet link. (You can refer to this as the Napster "namespace," a privately-managed metadata directory that gives Napster the ability to link users and their files with each other.)

In considering Napster, it's worth noting that the "namespace" of popular music is simple, and well-known. The Napster model breaks down in cases where more complex metadata is required to find a given piece of data. For example, in the case of classical music, an artist/song combination is often insufficient, since the same piece may be performed by various combinations of artists.

A related observation, which Darren New of Invisible Worlds made at the summit, is that Napster depends on the music industry itself to "market its namespace." Without pre-existing knowledge of song titles and artists, there is nothing for the Napster user to search for. This will lead to additional centralization layers as unknown artists try to provide additional information to help users find their work. This is much the same thing that happened on the Web, as a class of portals such as Yahoo! grew up to categorize and market information about the peer-to-peer world of hyperlinked Web pages.

It's easy to see, then, how understanding and managing namespaces and other forms of metadata becomes central even to peer-to-peer applications. What's more, it is also the key to peer-to-peer business models. Controlling namespaces and resource discovery has turned out to be one of the key battlegrounds of the Web. From Network Solutions, which largely controls DNS domain name registration, to Yahoo and search engines, identifying and capitalizing on the ways that centralization impacts even radically decentralized systems has turned out to be one key to financial success.

Instant messaging turns out to tell a similar story. The namespace of an instant-messaging system and the mapping of identity onto user addresses is the key to those systems. You have only to witness the efforts of AOL to keep other instant-messaging vendors from reaching its customers to understand just how important this is. (Note, however, that in the end, an open namespace with multiple providers will create a more powerful network than a closed one, just as the open Web trumped closed information services like AOL and MSN. AOL now succeeds for its customers as a "first among equals" rather than as a completely closed system.)

In the case of a distributed computation application, metadata might mean some identifier that allows the distributed data elements to be reassembled, and the address of the user who is working on a particular segment. SETI@Home also tracks user identity as a way of providing a game-like environment in which users and companies compete to contribute the most cycles. Startups aiming to compensate users for their spare compute cycles will need to track how much is contributed. Depending on the type of problem to be computed, they might want to know more about the resources being offered, such as the speed of the computer, the amount of available memory, and the bandwidth of the connection.

We can see then that some of the key business battlegrounds for peer-to-peer will be in defining the standards for metadata, the protocols for describing and discovering network based resources and services, and owning the "namespaces" that are used to identify those resources.

Returning to Napster, though, it's also clear that the core competencies required of successful peer-to-peer projects will include seamless communication and connectivity, facilities that support self-organizing systems, and the management of trust and expectations.

Related Articles:

In Praise of Freeloaders

What Is P2P ... And What Isn't

What's On Freenet?

Free Radical: Ian Clark has Big Plans for the Internet

A Directory of Peer-to-Peer Projects

How Ray Ozzie Got His Groove Back

Peer-to-Peer Makes the Internet Interesting Again

More from the P2P DevCenter Previous Features

Ultimately, peer-to-peer is about overcoming the barriers to the formation of ad-hoc communities, whether of people, of programs, of devices, or of distributed resources. It's about decoupling people, data and services from specific machines, using redundancy to replace reliability of connections as the key to consistency. If we get it right, peer-to-peer can help to break the IT bottleneck that comes with centralized services. Decentralization and user empowerment enable greater productivity. Edge services allow more effective use of Internet resources.

We're just at the beginning of a process of discovery. To get this right, we'll need a lot of experimentation. But if we can learn lessons from Internet history, we also need to remember to focus on interoperability, rather than treating this as a winner-takes-all game in which a single vendor can establish the standard for the network platform.

The peer-to-peer landscape is changing daily. New companies, applications, and projects appear faster than they can be catalogued. Especially with all the hype around peer-to-peer, the connections between these projects can be fairly tenuous. Is it marketing buzz or is it substance when everyone tries to join the parade?

While there's a danger in casting the net too widely, there's also a danger in limiting it. I believe that the story I've told gives us a good starting point in understanding an emergent phenomenon: the kind of computing that results when networking is pervasive, resources are abundant (and redundant) and the barriers are low to equal participation by any individual network node.

Tim O'Reilly is the founder and CEO of O'Reilly Media, Inc., thought by many to be the best computer book publisher in the world. In addition to Foo Camps ("Friends of O'Reilly" Camps, which gave rise to the "un-conference" movement), O'Reilly Media also hosts conferences on technology topics, including the Web 2.0 Summit, the Web 2.0 Expo, the O'Reilly Open Source Convention, the Gov 2.0 Summit, and the Gov 2.0 Expo. Tim's blog, the O'Reilly Radar, "watches the alpha geeks" to determine emerging technology trends, and serves as a platform for advocacy about issues of importance to the technical community. Tim's long-term vision for his company is to change the world by spreading the knowledge of innovators. In addition to O'Reilly Media, Tim is a founder of Safari Books Online, a pioneering subscription service for accessing books online, and O'Reilly AlphaTech Ventures, an early-stage venture firm.

Discuss this article in the O'Reilly Network General Forum.

Return to the P2P DevCenter.


P2P Weblogs

Richard Koman Richard Koman's Weblog
Supreme Court Decides Unanimously Against Grokster
Updating as we go. Supremes have ruled 9-0 in favor of the studios in MGM v Grokster. But does the decision have wider import? Is it a death knell for tech? It's starting to look like the answer is no. (Jun 27, 2005)

> More from O'Reilly Developer Weblogs

More Weblogs
FolderShare remote computer search: better privacy than Google Desktop? [Sid Steward]

Data Condoms: Solutions for Private, Remote Search Indexes [Sid Steward]

Behold! Google the darknet/p2p search engine! [Sid Steward]

Open Source & The Fallacy Of Composition [Spencer Critchley]