.NET from the Enterprise Perspective:SOAP

In my earlier article, Microsoft .NET vs J2EE: How Do They Stack Up, I compared .NET and J2EE as development frameworks. In our follow-up conversation, Tim O'Reilly and I discussed the larger picture around .NET, namely how .NET proposes to change the game of enterprise applications into a world where services talk to each over a common backplane, defined by SOAP, in a vein similar to CORBA's intentions with IIOP and IDL.

In this article, I'd like to continue the enterprise-level discussion. In one sense this gives some closure to the discussion, but in another sense I hope it further fuels the larger discourse that's being held online and around watercoolers.

Looked upon as an application development environment (the perspective of my earlier article), .NET will be a reengineering of Microsoft's world for developers. But what innovations is .NET offering in a world where the Web is increasingly emerging as a global information, interaction, and commerce space? The answer revolves around SOAP (Simple Object Access Protocol), and SOAP-based Web services, so it's important to get at what SOAP really brings to the table. For those of you unfamiliar with where SOAP came from, let's indulge in a bit of background.

The Road to SOAP

When you as an individual or organization want to engage in an interaction of some kind with another individual or organization, there are two things that need to get sorted out up-front: 1) What is the structure and syntax of the language we are going to speak (i.e., what messages and data are we going to exchange, and how); and 2) What are the underlying semantics behind this language (i.e., in a real-world sense, what does it mean when I give you a value of 58 13/16 with a label of "GE", or tell you to closeAccountEvents(09834))?

This need for structure and semantics arises whether the interaction is between two people, a person and a computing device, or two computing devices. The context that has the most people in a lather these days is the interaction between two computing devices, where money is involved in the conversation. This is the basis for electronic commerce, which is really just a euphemism for "fast money."

Don't miss The .NET/C# Roundup for more in-depth articles. You can also listen to Getting to the Bottom of .Net, a discussion in which Tim O'Reilly and Jim Farley talk about the ideas behind--and the opportunities beyond--Microsoft's .Net Framework.

Until very recently, online interactions were typically done in one of two ways. A digital exchange can occur in a direct, tightly coupled connection, where the structure and syntax of the messages are encoded into object interfaces and the parties engage in remote method calls (over CORBA/IIOP, RMI, DCOM, etc.). Alternatively, the connection can be more loosely coupled, defined in terms of GET/PUT arguments on well-defined URLs (e.g., validate a credit card transaction by making a request in the form of POST arguments to a particular SSL-enabled URL).

Data exchanges are handled within these contexts in various ways: as method arguments; as URL arguments; as structured data streams generated from either of these sources; or sometimes even as out-of-band direct database transactions. The semantics of these exchanges can be very local and customized, or in rare cases there may be well-known, high-level APIs in play, such as an e-commerce component library or widely-published and well-documented EDI-based protocols.

Enter XML, which arose to address the need for a common, flexible context for defining the structure and syntax of messages and data. This was really a return to the SGML roots of HTML, which by this time (1996) had lots of presentation-specific details incorporated into its syntax. In an XML context, DTDs and XML schemas provide a well-defined format for specifying (and, more importantly, sharing) the structure and syntax of an exchange. The semantics and rules of the exchange are agreed upon as part of the ancillary elements of the DTD/schema documentation. For example, "a 'stockquote' tag will contain data representing the market price of a stock within five minutes of the time the tag was generated at the source," or, conversely, "when asked for a stock price, a compliant AcmeXML participant will respond with a well-formed 'stockquote' tag."

SOAP: Back to the Future

It's only natural to think of using XML in an online-messaging or remote-method context, and that's what happened next. XML-RPC came around 1998 as a way to encode remote method calls and responses in an XML-based format, and as a way to transmit these remote method payloads over HTTP. SOAP evolved out of the same work (and people) that created XML-RPC, and was submitted as a note to the W3C earlier this year, for its comment.

The interesting thing about this effort is that it's really a move back to the days before distributed object protocols were developed. The RPC protocol is a scheme for encoding remote procedure calls into a standard representation ( XDR), then serializing these calls onto the wire and transmitting them to a remote RPC peer, where they are deserialized, processed, and results are similarly encoded and returned. Distributed object protocols came about as a way to dissolve the interface between RPC capabilities and object-oriented environments like Java and C++. Once you've done the up-front work to define a remote object and implement its methods, you make remote method calls in your code by calling methods on remote object "stubs," which you've obtained from a remote service. No more messing around with RPC/XDR encodings of method arguments and responses: The distributed object system handles all this for you when you make a method call on a remote object stub.

XML-RPC and SOAP roll the clock back to RPC, then move it forward again using XML as the encoding context instead of XDR or object interfaces. Then they specify a way to deliver XML-encoded data and RPCs over HTTP. The idea here is that you encapsulate your services at a different level and export an XML face to the world, rather than object interfaces.

Why is this Cool?

First off, SOAP (like RPC and IIOP before it) decouples the encoding and communications protocol from the runtime environment. Your Web service can receive a SOAP payload from a remote service, and the platform details of the source are entirely irrelevant. Anything can generate XML, from Perl scripts to C++ code to J2EE app servers. In fact, DevelopMentor, one of the original developers of the SOAP specification, is developing SOAP implementations for Perl, Java, and IONA, and others plan to include a SOAP layer on top of their CORBA products. So, as of the 1.1 version of the SOAP specification, anyone and anything can participate in a SOAP conversation, with a relatively low barrier to entry.

SOAP is also a simple way to accomplish remote object/component/service communications (hence its name). It formalizes the vocabulary definition in a form that's now familiar, popular, and accessible (XML). If you know XML, you can figure out the basics of SOAP encoding pretty quickly.

In these regards, SOAP has an edge on the predominant remote object protocols (RMI, IIOP, DCOM). RMI is very straightforward if you know Java, but it requires Java on either end, hence it puts additional platform restrictions on the participants. CORBA decouples the protocol from the runtime environment (which could be ADA running on a Linux box, or Java running on a Windows box, or C++ running on an AS400, or ...). But the framework of CORBA is pretty complex and there is a learning curve to invest in before diving into enterprise CORBA systems. Microsoft COM/DCOM also has platform restrictions (It effectively requires Windows), despite the availability of Software AG's Unix ports of DCOM. But the real dividing line here is how the vocabulary is defined between the parties. In the case of RMI, CORBA, and DCOM, how you speak to a remote service is encoded in the object interfaces that it exports, along with the semantics defined behind these interfaces. So I have to know and understand the Java, IDL, or MIDL definitions for these interfaces in order to interact with them. With SOAP, I still need to know the interface to your service (What requests do you respond to? What compound data types can you digest?), but you can give it to me in the form of XML.

What's Not So Cool About SOAP?

So, from an enterprise computing standpoint, .NET's adoption (and evangelizing) of SOAP into its framework has some promise. But there's no magic bullet here. As the sayings go, nothing is free, and nothing is perfect, and you should be aware that SOAP has its fair share of trade-offs and warts. First, the warts.

The SOAP specification contains no mention of security facilities. This is not a good thing. One of the advantages of SOAP is that it runs over HTTP, which eliminates firewall problems. Well, firewalls are there for a reason. No enterprise will want to open up a channel to make direct, unprotected method calls on their Web services. Some will build custom security measures on top of SOAP, to ensure that authentication, authorization, and accountability are preserved. This tosses a great deal of the interoperability of SOAP out the window. Others will solve this gap at the network level, sniffing HTTP traffic passing through their firewalls, and restricting SOAP payloads to privileged IPs/ports. But now we're trading network administration overhead for portability. If and when SOAP payload filters become common fare with firewall vendors, this overhead will go away. But this only makes sense if SOAP traffic is well-defined and detectable.

This leads us to another issue. The current version of the SOAP specification (1.1) does not specify a default encoding for the message body. There is an encoding defined in the spec, but it is not required that you use this encoding to be compliant: Any custom encoding that you choose can be specified in the encodingStyle attribute of the message or of individual elements in the message. This is not a good thing. The default encoding spelled out in the spec may become a de facto standard by SOAP implementations, but this standard needs to be made explicit so that SOAP interoperability can be well-defined and testable. This may sound like picking nits, since the change here is really just a few sentences in the 1.1 specification. But as vendor activity heats up around this, some will start to throw around the "SOAP compliant" label rather loosely, and if they do indeed start to use custom encoding styles, the adoption of SOAP will suffer from lack of interoperability. This smells a lot like the mistakes made with CORBA in its early days by not specifying a standard wire protocol. CORBA suffered for this lack of interoperability, and SOAP (and .NET) run the same risk by leaving this hole unfilled.

The Cost of Simplicity

Now what about the trade-offs? SOAP is simple, accessible, and very portable. But what are you giving up for this?

SOAP is very simple compared to RMI, CORBA, and DCOM because it does not deal with certain ancillary but important aspects of remote object systems. I've already mentioned the lack of security provisions. In addition, the specification itself explicitly excludes distributed garbage collection, objects-by-reference, and remote activation as being "not part of the core SOAP specification." The SOAP model also does not include any provisions for object lifecycles, session/state management, or distributed transactions. There are ways to add custom header entries to a SOAP message to address some of these issues (the SOAP spec includes examples that show custom transaction-oriented header fields), but these custom services layered on top of the standard SOAP model also severely limit interoperability.

I'm not suggesting that SOAP be expanded to include all of these services. If it tried to do this, it would bloat significantly, and it would get much more complicated. Rather, I would hope that SOAP would expand to include the strictly necessary elements, like security and perhaps distributed transactions. And for the other issues, like component lifecycles and session management, the SOAP spec should be amended to include the underlying assumptions about the models in use by SOAP agents. In other words, define the high-level contracts that SOAP participants need to satisfy with their underlying implementations and leave vendors and developers the freedom to continue to use the tools that they want, as long as they honor the specified contracts. SOAP should stay as simple as possible, and it does this by limiting its target domain to messaging situations where its simplified runtime model is sufficient. But this runtime model needs to be spelled out explicitly.

The upshot of all this is that SOAP nicely offers industry support for a new suite of design and implementation patterns, and a way to quickly establish interactions between online services. But SOAP-like messaging is appropriate for some situations and not others, and it should probably stay that way. The messy details of remote method calls and distributed enterprises do arise in many situations because of the nature of the problem being solved. In these situations, there needs to be a way to shift into a more structured (and more complex) remote computing environment, like J2EE or CORBA.

Which Way to Nirvana from Here?

As I've suggested above, one of the first issues to be addressed is firming up the SOAP specification. And we can all hope that the industry leaders (IBM, Microsoft, Sun, Oracle, and so on) will keep the center of gravity centered around protocols, like SOAP and ebXML. Platform- and language-specific frameworks, like .NET and J2EE, should be a separate concern. The factors playing into the decisions that an organization makes to use one or the other to build their internal infrastructure should be orthogonal to the decision to publish Web services under SOAP.

As an example, for the development processes and architectures that I typically face, the ideal framework for me in a post-.NET/SOAP world would be a J2EE environment enhanced with SOAP support for interservice messaging (either over HTTP or JMS), and a graceful path from XML to JMS, or EJB, or IIOP, or RMI, or even sockets. This would let me keep the cross-platform abilities of Java, and let me easily connect to other services using the protocol appropriate for the purpose at hand.

Someone else may prefer the multi-language capabilities that Microsoft is trying to build into .NET's common language runtime, or they may be building a primarily CORBA enterprise and want to use SOAP-enabled CORBA servers from IONA or another vendor. The ability of organizations to make the right decisions for themselves, and the ability for multiple players to offer solutions that cater to different needs is one of the biggest payoffs of open standards. So from the enterprise perspective, the ubiquitous SOAP support that is being promoted by the announcement of .NET is a boon to developers and technology strategists alike.

Bibliography

• Microsoft .NET vs J2EE: How Do They Stack Up, Jim Farley, java.oreilly.com, August 2000.

• Getting to the Bottom of .NET, Tim O'Reilly and Jim Farley, windows.oreilly.com, August 2000.

• SOAP: Simple Object Access Protocol, MDSN Online.

• XML-RPC home page.

• Simple Object Access Protocol (SOAP) 1.1, W3C Note, May 2000.

• W3C Team Comment on the "SOAP" Submission, May 2000.

• RPC: Remote Procedure Call Protocol Specification Version 2, IETF RFC 1831, August 1995.

• XDR: External Data Representation Standard, IETF RFC 1832, August 1995.

• DevelopMentor's SOAP area.

• IONA Orbix 2000 Product Information.

• Software AG's EntireX product technical description.