Clay Shirky -- Decoding P2P

Hailstorm: Open Web Services Controlled by Microsoft


Comment on this articleHow will Hailstorm and Passport change the face of P2P, web services, and the Net itself?
Post your comments

So many ideas and so many technologies are swirling around P2P -- decentralization, distributed computing, web services, JXTA, UDDI, SOAP -- that it's getting hard to tell whether something is or isn't P2P, and it's unclear that there is much point in trying to do so just for the sake of a label.

What there is some point in doing is evaluating new technologies to see how they fit in or depart from the traditional client-server model of computing, especially as exemplified in recent years by the browser-and-web-server model. In this category, Microsoft's Hailstorm is an audacious, if presently ill-defined, entrant. Rather than subject Hailstorm to some sort of P2P litmus test, it is more illuminating to examine where it embraces the centralization of the client-server model and where it departs by decentralizing functions to devices at the network's edge.

The design and implementation of HailStorm is still in flux, but the tension that exists within HailStorm between centralization and decentralization is already quite vivid.


HailStorm, which launched in March with a public announcement and a white paper, is Microsoft's bid to put some meat on the bones of its .NET initiative. It is a set of Web services whose data is contained in a set of XML documents, and which is accessed from the various clients (or "HailStorm endpoints") via SOAP (Simple Object Access Protocol.) These services are organized around user identity, and will include standard functions such as myAddress (electronic and geographic address for an identity); myProfile, (name, nickname, special dates, picture); myCalendar, myWallet; and so on.

HailStorm can best be thought of as an attempt to re-visit the original MS-DOS strategy: Microsoft writes and owns the basic framework, and third-party developers write applications to run on top of that framework.

Three critical things differentiate the networked version of this strategy, as exemplified by HailStorm, from the earlier MS-DOS strategy:


For more on Hailstorm and .NET, check out O'Reilly Network's new .NET Devcenter.

The key to shifting from a machine-centric application model to a distributed computing model is to shift the central unit away from the computer and towards the user. In a machine-centric system, the software license was the core attribute -- a software license meant a certain piece of software could be legally run on a certain machine. Without such a license, that software could not be installed or run, or could only be installed and run illegally.

In a distributed model, it is the user and not the hardware that needs to be validated, so user authentication becomes the core attribute -- not "Is this software licensed to run on this machine?" but "Is this software licensed to run for this user?" To accomplish this requires a system that first validates users, and then maintains a list of attributes in order to determine what they are and are not allowed to do within the system.

HailStorm is thus authentication-centric, and is organized around Passport. HailStorm is designed to create a common set of services which can be accessed globally by authenticated users, and to this end it provides common definitions for:

or as Microsoft puts it:

From a technical perspective, HailStorm is based on Microsoft Passport as the basic user credential. The HailStorm architecture defines identity, security, and data models that are common to all HailStorm services and ensure consistency of development and operation.


The decentralized portion of HailStorm is a remarkable departure for Microsoft: they have made accessing HailStorm services on non-Microsoft clients a core part of the proposition. As the white paper puts it:

The HailStorm platform uses an open access model, which means it can be used with any device, application or services, regardless of the underlying platform, operating system, object model, programming language or network provider. All HailStorm services are XML Web SOAP; no Microsoft runtime or tool is required to call them.

To underscore the point at the press conference, they demonstrated HailStorm services running on a Palm, a Macintosh, and a Linux box.

Clay Shirky will speak in several sessions at the O'Reilly P2P and Web Services Conference, Nov. 5-8 in Washington, DC. David Chappell will speak on "Understanding Hailstorm", as well.

While Microsoft stresses the wide support for HailStorm clients, the relationship of HailStorm to the Web's servers is less clear. In the presentation, they suggested that servers running non-Microsoft operating systems like Linux or Solaris can nevertheless "participate" in HailStorm, though they didn't spell out how that participation would be defined.

This decentralization of the client is designed to allow Hailstorm applications to spread as quickly as possible. Despite their monopoly in desktop operating systems, Microsoft does not have a majority market share for any of the universe of non-PC devices -- PDAs, set-tops, pagers, game consoles, cell phones. This is not to say that they don't have some notable successes -- NT has over a third of the server market, the iPaq running the PocketPC operating system is becoming increasingly popular, and the XBox has captured the interest of the gaming community. Nevertheless, hardware upgrade cycles are long, so there is no way Microsoft can achieve market dominance in these categories as quickly.

Also in Clay Shirky -- Decoding P2P:


Interoperability, Not Standards

P2P Smuggled In Under Cover of Darkness

The Parable of Umbrellas and Taxicabs

The Case Against Micropayments

Enter HailStorm. HailStorm offers a way for Microsoft to sell software and services on devices that aren't using Microsoft operating systems. This is a big change -- Microsoft typically links its software and operating systems (SQLServer won't run outside an MS environment; Office is only ported to the Mac). By tying HailStorm to SOAP and XML rather than specific client environments, Microsoft says it is giving up its ability to control (or even predict) what software, running on which kinds of devices, will be accessing HailStorm services.

The embrace of SOAP is particularly significant, as it seems to put HailStorm out of reach of many of its other business battles -- vs. Java, vs. Linux, vs. PalmOS, and so on -- because, according to Microsoft, any device using SOAP will be able to participate in HailStorm without prejudice -- "no Microsoft runtime or tool" will be required, though the full effect of this client-insensitivity will be determined by how much Microsoft alters Kerberos or SOAP in ways that limit or prevent other companies from writing HailStorm-compliant applications.

HailStorm is Microsoft's most serious attempt to date to move from competing on unit sales to selling software as a service, and the announced intention to allow any sort of client to access HailStorm represents a remarkable decentralization for Microsoft.

It is not, however, a total decentralization by any means. In decentralizing their control over the client, Microsoft seeks to gain control over a much larger set of functions, for a much larger group of devices, than they have now. The functions that HailStorm centralizes are in many ways more significant than the functions it decentralizes.


In the press surrounding HailStorm, Microsoft refers to its "massively distributed" nature, its "user-centric" model, and even makes reference to its tracking of user presence as "peer-to-peer." Despite this rhetoric, however, HailStorm as described is a mega-service, and may be the largest client-server installation ever conceived.

Microsoft addressed the requirements for running such a mega-service, saying:

Reliability will be critical to the success of the HailStorm services, and good operations are a core competency required to ensure that reliability. [...] Microsoft is also making significant operational investments to provide the level of service and reliability that will be required for HailStorm services. These investments include such things as physically redundant data centers and common best practices across services.

This kind of server installation is necessary for HailStorm, because Microsoft's ambitions for this service are large: they would like to create the world's largest address registry, not only of machines but of people as well. In particular, they would like to host the identity of every person on the Internet, and mediate every transaction in the consumer economy. They will fail at such vast goals of course, but succeeding at even a small subset of such large ambitions would be a huge victory.

Because they have decentralized their support of the client, they must necessarily make large parts of HailStorm open, but always with a caveat: while HailStorm is open for developers to use, it is not open for developers to build on or revise. Microsoft calls this an "Open Access" model -- you can access it freely, but not alter it freely.

This does not mean that HailStorm cannot be updated or revised by the developer community; it simply means that any changes made to HailStorm must be approved by Microsoft, a procedure they call "Open Process Extensibility." This process is not defined within the white paper, though it seems to mean revising and validating proposals from HailStorm developers, which is to say, developers who have paid to participate in HailStorm.

With HailStorm, Microsoft is shifting from a strategy of controlling software to controlling transactions. Instead of selling units of licensed software, Hailstorm will allow them to offer services to other developers, even those working on non-Microsoft platforms, while owning the intellectual property which underlies the authentications and transactions, a kind of "describe and defend" strategy.

"Describe and defend" is a move away from "software as unit" to "software as service," and means that their control of the HailStorm universe will rely less on software licenses and more on patented or copyrighted methods, procedures, and database schema.

While decentralizing client-code, Microsoft centralizes the three core aspects of the service:

Identity: The goal with Passport is simple -- ubiquity. As Bill Gates put it at the press conference: "So it's our goal to have virtually everybody who uses the Internet to have one of these Passport connections."

HailStorm provides a set of globally useful services which, because they are authentication-centric, requires all users to participate in its Passport program. This allows Microsoft to be a gatekeeper at the level of individual participation -- an Internet user without a Passport will not exist within the system, and will not be able to access or use Passport services. Because users pay to participate in the HailStorm system, in practice this means that Microsoft will control a user's identity, leasing it to them for use within HailStorm for a recurring fee.

It's not clear how open the Passport system will be. Microsoft has a history of launching web initiatives with restrictive conditions, and then dropping the restrictions that limit growth: the original deployment of Passport required users to get a Hotmail account, a restriction that was later dropped when this adversely affected the potential size of the Passport program. You can now get a Passport with any email address, and since an email address is guaranteed to be globally unique, any issuer of email addresses is also issuing potentially valid Passport addresses.

The metaphor of a passport suggests that several different entities agree to both issue and honor passports, as national governments presently do with real passports. There are several entities who have issued email addresses to millions or tens of millions of users -- AOL, Yahoo, ATT, British Telecom, et al. Microsoft has not spelled out how or whether these entities will be allowed to participate in HailStorm, but it appears that all issuing and validation of Passports will be centralized under Microsoft's control.

Security: Authentication of a HailStorm user is provided via Kerberos, a secure method developed at MIT for authenticating a request for a service in a computer network. Last year, Microsoft added its own proprietary extension to Kerberos, which creates potential incompatibilities between clients running non-Microsoft versions of Kerberos and servers running Microsoft's versions.

Microsoft has published the details of its version of Kerberos, but it is not clear if interoperability with the Microsoft version of Kerberos is required to participate in HailStorm, or if there are any licensing restrictions for developers who want to write SOAP clients that use Kerberos to access HailStorm services.

Definitions and Descriptions: This is the most audacious aspect of HailStorm, and the core of the describe-and-defend strategy. Microsoft wants to create a schema which describes all possible user transactions, and then copyright that schema, in order to create and manage the ontology of life on the Internet. In HailStorm as it was described, all entities, methods, and transactions will be defined and mediated by Microsoft or Microsoft-licensed developers, with Microsoft acting as a kind of arbiter of descriptions of electronic reality:

The initial release of HailStorm provides a basic set of possible services users and developers might need. Beyond that, new services (for example, myPhotos or myPortfolio) and extensions will be defined via the Microsoft Open Process with developer community involvement. There will be a single schema for each area to avoid conflicts that are detrimental to users (like having both myTV and myFavoriteTVShows) and to ensure a consistent architectural approach around attributes like security model and data manipulation. Microsoft's involvement in HailStorm extensions will be based on our expertise in a given area.

The business difficulties with such a system are obvious. Will the airline industry help define myFrequentFlierMiles, copyright Microsoft, when Microsoft also runs the Expedia travel service? Will the automotive industry sign up to help the owner of CarPoint develop myDealerRebate?

Less obvious but potentially more dangerous are the engineering risks in a single, global schema, because there are significant areas where developers might legitimately disagree about how resources should be arranged. Should business users record the corporate credit card as a part of myWallet, alongside their personal credit card, or as part of myBusinessPayments, alongside their EDI and purchase order information? Should a family's individual myCalendars be a subset of ourCalendar, or should they be synched manually? Is it really so obvious that there is no useful distinction between myTV (the box, through which you might also access DVDs and even WebTV) and myFavorite TVShows (the list of programs to be piped to the TiVo)?

Microsoft proposes to take over all the work of defining the conceptual entities of the system, promising that this will free developers to concentrate their efforts elsewhere:

By taking advantage of Microsoft's significant investment in HailStorm, developers will be able to create user-centric solutions while focusing on their core value proposition instead of the plumbing.

Unmentioned is what developers whose core value proposition is the plumbing are to do with HailStorm's global schema. With Hailstorm, Microsoft proposes to divide the world into plumbers and application developers, and to take over the plumbing for itself. This is analogous to the split early in its history when Microsoft wrote the DOS operating system, and let other groups write the software that ran on top of DOS.

Unlike DOS, which could be tied to a single reference platform -- the "IBM compatible" PC -- HailStorm is launching into a far more heterogeneous environment. However, this also means that the competition is far more fragmented, and given the usefulness of HailStorm to developers who want to offer Web services without rethinking identity or authentication from the ground up (one of the biggest hurdles to widespread use of Sun's JXTA), and the possible network effects that a global credentials schema could create, HailStorm could quickly account for a plurality of Internet users. Even a 20% share of every transaction made by every Internet user would make Microsoft by far the dominant player in the world of Web services.

Non-Microsoft Participation

Also in Clay Shirky -- Decoding P2P:


Interoperability, Not Standards

P2P Smuggled In Under Cover of Darkness

The Parable of Umbrellas and Taxicabs

The Case Against Micropayments

With HailStorm, Microsoft has abandoned tying its major software offerings to its client operating systems. Even if every operating system it has -- NT/Win2k, PocketPC, Stinger, et al -- spreads like kudzu, the majority of the world's non-PC devices will still not be controlled by Microsoft in any short-term future. By adopting open standards such as XML and SOAP, Microsoft hopes to attract the world's application developers to write for the HailStorm system now or soon, and by owning the authentication and schema of the system, they hope to be the mediator of all HailStorm users and transactions, or the licenser of all members of the HailStorm federation.

Given the decentralization on the client-side, where a Java program running on a Linux box could access Hailstorm, the obvious question is "Can a HailStorm transaction take place without talking to Microsoft owned or licensed servers?"

The answer seems to be no, for two, and possibly three, reasons.

First, you cannot use a non-Passport identity within HailStorm, and at least for now, that means that using HailStorm requires a Microsoft-hosted identity.

Second, you cannot use a non-Microsoft copyrighted schema to broker transactions within HailStorm, nor can you alter or build on existing schema without Microsoft's permission.

Third, developers might not be able to write HailStorm services or clients without using the Microsoft-extended version of Kerberos.

At three critical points in HailStorm, Microsoft is using an open standard (email address, Kerberos, SOAP) and putting it into a system it controls, not through software licensing but through copyright (Passport, Kerberos MS, HailStorm schema). By making the system transparent to developers but not freely extensible, Microsoft hopes to gain the growth that comes with openness, while avoiding the erosion of control that also comes with openness.

This is a strategy many companies have tried before -- sometimes it works and sometimes it doesn't. Compuserve collapsed while pursuing a partly open/partly closed strategy, while AOL flourished. Linux has spread remarkably with a completely open strategy, but many Linux vendors have suffered. Sun and Apple are both wrestling with "open enough to attract developers, but closed enough to stave off competitors" strategies with Solaris and OS X respectively.

Hailstorm will not be launching in any real way until 2002, so it is too early to handicap Microsoft's newest entrant in the "open for users but closed for competitors" category. But if it succeeds at even a fraction of its stated goals, Hailstorm will mark the full-scale arrival of Web services and set the terms of both competition and cooperation within the rest of the industry.

Copyright © 2017 O'Reilly Media, Inc.