<document>
  <header>
    <product>resin-ee</product>
    <resin-2.0>$ee/ejb-ref/burlap-notes.xtp</resin-2.0>
    <title>Burlap Notes</title>
    <date>December 31, 2001</date>
    <description>
      <p>As described in the <a href="protocols|burlap-draft-spec">Burlap draft spec</a>, 
        we created Burlap to implement Enterprise Java Beans (EJB) using
        an XML-based protocol with reasonable performance.  Although many
        RPC protocols already exist, including several based on XML, none met
        our application's needs.  The name "Burlap" was chosed for a simple
        reason: it's boring.  Unlike the exciting protocols defining "Internet 3.0",
        SOAP and XML-RPC, Burlap is just boring text-based protocol to make
        testing and debugging EJB a little bit easier.</p>

      <p>Because we're an engineering-driven company and lack the resources
        to effectively lobby for Burlap as the standard internet
        services wire-protocol,
        we have the opportunity to write these design notes from an
        engineering perspective, devoid of marketing hype.  Several of the
        examples use <a href="http://www.w3.org/TR/SOAP/">SOAP</a> and
        <a href="http://www.xmlrpc.com/spec">XML-RPC</a> to
        contrast Burlap's design decisions, but as those protocols have
        different goals, the contrasts should not be taken as criticisms
        of the protocols.  Most of the design of Burlap, after all, is merely a
        modification of SOAP and XML-RPC for EJB's needs.</p>

    </description>
  </header>

  <body>
    <summary/>



<section title="Design Goals">

<p>The Burlap protocol was created to solve a specific problem: to
provide remote procedure calls for Java Enterprise Java Beans (EJB)
using an XML-based protocol without limiting the
protocol to Java servers and clients.

<p>EJB is often used for distributed computing on a small scale.  One
development team controls the design for both clients and servers.
Most often, the application uses a single subnet for all
communication.  As long as the RPC protocol is fast and reliable, most
EJB developers don't care what protocol is used.  "Small-scale" might
be misleading: a site like eBay using EJB internally would be small-scale
distributed computing, but it needs large-scale performance
and reliability.</p>

<ul>
<li>It must have sufficient power to support EJB.
<li>It must be simple so it can be effectively tested.
<li>It must be as fast as possible.
<li>It must only require Java introspection.  It must not require external
IDL or schema definitions.
<li>It must use a subset of XML.
<li>It must allow EJB servers to deployed as a Servlet.
<li>It should support transaction contexts.
<li>It should allow non-Java clients to use EJB servers.
</ul>

<section title="Sufficiency">

<p>Fortunately, we know exactly how powerful Burlap needs to be: it needs to
serialize and deserialize arbitrary Java objects.  In contrast, SOAP and
XML-RPC have the much tougher responsibility of an open-ended sufficiency
requirement.  Although Burlap may be sufficiently powerful
for essentially any service, it's not a design goal.

<p>Specific requirements:</p>

<ul>
<li>Serialize Java primitive types (boolean, int, long, double, String).
<li>Support nulls.
<li>Permit shared and circular data structures.
<li>Support objects, arrays, lists, and maps.
<li>Deserialize subclassed objects, including Object variables..
<li>Support remote object references (EJBHome and EJBObject references.)
</ul>

<p>Surprisingly, many of the existing protocols are insufficiently
powerful to handle these requirements.  Even IIOP/CORBA needed protocol
additions to support full Java serialization.

<p>XML-RPC is missing a number of the features needed to support
EJBs: 64-bit integer (Java long), nulls, shared data structures, subclassed
objects, and remote object references.

<p>SOAP explicitly does not support remote object references because it
was perceived as too difficult. In other respects SOAP seems to be
sufficient.  In fact, it defines a large number of types unusable for
EJB, including everying in the
<a href="http://www.w3.org/TR/xmlschema-2/">XML Schema</a>: sparse arrays,
enumerations, integer subranges, durations.

<p>Maps, including Java Hashtables and HashMaps, are not explicitly
supported by other protocols, even though they are very common in
Java programming.  All the protocols support special String to Object
maps, i.e. structures, but none support Object to Object maps.  Of course,
it's always possible to serialize the Hashtable structure itself, as RMI
does, but that exposes the implementation details of the hashtable.  So a
Hashtable would look very different from a HashMap serialization, making
it difficult for different languages to interoperate.

</section>

<section title="Testability">

<p>Testability is the key design goal of Burlap.  At this early stage
of development, we have 43k lines in the EJB test suite, but only 28k
lines of code in the EJB implementation.  Granted, the EJB
implementation uses Resin libraries heavily which weren't included
in the count, but it should give an idea of the work involved in testing.

<p>The Burlap spec is as small as possible to reduce the
test suite size.  For example, eliminating XML attributes, namespaces, and
<a href="http://www.w3.org/TR/2001/PR-xmlschema-1-20010316">XML Schema</a>
radically reduces the test suite size and complexity without losing any power.
The EJB spec only requires CORBA/IIOP as a wire protocol, but
IIOP is a huge ungainly beast and is a binary protocol.  Because
IIOP is huge, testing it is a large task and leaves many places where
bugs can spawn.  By using Burlap as the primary wire protocol, we can
make it more reliable because it's more testable, and when we do implement
IIOP we can easily localize the bugs.

<p>Since ambiguity and complexity make testing difficult, both have
been eliminated where possible in Burlap.  Even simple ambiguity can make
testing more difficult.  For example, XML-RPC allows either &lt;int>
or &lt;i4> to represent 32-bit integers.  In theory, an implementation
could parse integers using different code for &lt;int> and for &lt;i4>,
so &lt;int> might be carefully tested and &lt;i4> spot-checked, but the
&lt;i4> parse might be buggy.  If a ServerA implementation
generally uses &lt;int>, a fully conforming ClientB might use &lt;i4> and
run into a bug undetected for months.  So ClientB needs to use
&lt;int> for ServerA.  The XML-RPC case is trivial, but for more complicated
and verbose specs like IIOP/CORBA and SOAP, full testing may be impossible,
forcing bake-off testing so the mutually-incompatible implementations can
be forced to work together.

<p>Ambiguity is not an academic problem.  Every HTTP server
implementor needs to work around various non-conforming clients.  
Clients interpret the HTTP cookie spec in many strange an
non-conformant ways.  For example, one client needs spaces in a
";&nbsp;secure" attribute, but another client can't deal with the spaces.
This is a real example.  If browsers mess up the simple cookie spec, the
SOAP, XML Schema, SOAP Attachment, WSDL, and more! specs, are likely
to cause interoperability and testing problems, like CORBA/IIOP did.
Burlap tries to eliminate ambiguity so these problems never show up
in the first place.

<p>The primary motivation for using a text-based protocol, like XML, is the
resulting simplicity of each test.  The following is
one of Caucho's tests for integer serialization.</p>

<example title='ejb/0004.qa'>
&lt;title>burlap: server int&lt;/title>
&lt;file file='file:/tmp/caucho/qa/WEB-INF/classes/test/MyBean.java'>
package test;

import com.caucho.burlap.*;

public class MyBean extends BurlapServlet {
  public int add(int a, int b) { return a + b; }
}
&lt;/file>

&lt;script out='stdout'>

http = new caucho.server.http.TestHttp(File("file:/tmp/caucho/qa"));
http.request(@&lt;&lt;END, out);
POST /servlet/test.MyBean HTTP/1.0
Content-Length: 120

&lt;burlap:call>
&lt;method>add&lt;/method>
&lt;int>32000&lt;/int>
&lt;int>-1000&lt;/int>
&lt;/burlap:call>
END

http.close();

&lt;/script>

&lt;compare file='/stdout'>
HTTP/1.0 200 OK
Server: Resin/1.1
Content-Length: 60
Date: Fri, 08 May 1998 09:51:31 GMT

&lt;burlap:reply>&lt;value>&lt;int>31000&lt;/int>&lt;/value>&lt;/burlap:reply>
&lt;/compare>
</example>

<p>Testability is a true measure of the complexity of a spec.  Writing a
sample server easy for any spec, but writing a
fully-conforming, tested servers that works with all the clients is a
much larger issue.  Unfortunately, most spec writers are "white-paper
engineers" who don't even implement their design, much less test it.
Given the computer industry's reputation for buggy products, it just
seems prudent to design the spec to minimize testing and bugs.

</section>

</section>

<section title='Implementation Decisions'>

<section title='XML vs SML'>

<p>This decision is obvious.  Burlap doesn't need the additional
complexity with XML or namespaces or schema, so there's no
need to add them.  It would just explode the test-suite, complicate the
parsers, and make conformance harder.  And all that complexity gives
zero added power.</p>

<p>The XML-specs give no added power because Burlap is an existence
proof for the sufficiency of SML.  Here's a rough sketch of a formal proof:</p>

<ul>
<li>It encodes Java primitives: &lt;boolean>, &lt;int>, &lt;long>, &lt;double>,
&lt;string>, &lt;null>, and is non-lossy for the additional <var/byte/>,
<var/short/>, <var/char/>, <var/float/> primitives.
<li>It encodes Java arrays (&lt;list> with &lt;type>[foo&lt;/type>)
<li>It encodes Java classes (serialization with &lt;map>)
<li>It supports Java inheritance (&lt;type> for &lt;list> and
&lt;map>).
<li>All Java objects are generated using a combination of the above.
</ul>

<p>(The additional tags &lt;date>, &lt;base64>, &lt;remote>,
using &lt;list> for List objects and &lt;map> for Map objects are
not strictly necessary, but make the protocol Java-independent.)</p>

<p>Since SML is sufficient, any added XML "features" can add nothing to
the expressive power because there's nothing left to add.  But adding
XML attributes and namespaces and have a huge cost in conformance, testing
and performance.  It's a good idea to remember that Burlap will always be
machine-generated.  Unlike HTML, no graphic designer will
ever write Burlap documents.</p>

<p>The trivial &lt;null> tag has only one encoding in Burlap as follows.</p>

<example title='SML-mandated null'>
&lt;null>&lt;/null>
</example>

<p>Having only one option not only simplifies the tests and
interoperability, but it improves the parser performance.  The
following is explicitly forbidden in Burlap, though allowed in XML:</p>

<results title='XML null (forbidden in Burlap)'>
&lt;null/>
</results>

<p>At first glance, the XML encoding appears more efficient, because
it saves 5 characters.  In reality, that savings of network bandwidth
is in the noise performance-wise.  When a typical page even over
a 56k modem can be 100k with additional gifs, 5 characters
doesn't matter.  Burlap's use is for single subnets, usually
running at 100Mb or possibly 1Gb.  The miniscule savings aren't worth
the complexity.</p>

<p>For Burlap's encoding it's easy to write an efficient parser, knowing
what kind of results are expected:</p>

<example>
switch (is.readToken()) {
case NULL:
  is.readEndTag("null");
  break;

case STRING:
  String v = is.readString();
  is.readEndTag("string");
  break;

...
}
</example>

<p>With the XML short-tag "feature", the code would need to
return a different token for "null-with-trailing-slash", the test
suite would need to add cases and all servers and clients would
need to parse both (with possible buggy clients only understanding
one form).  This example is relatively simple; the addition of
attributes and namespaces would increase the complexity.</p>

</section>

<section title="DTD vs XML Schema">

<p>XML has two main use patterns: a static syntax described by a DTD (document
type descriptor), and an extensible syntax using XML namespaces and
presumably defined by an XML schema.  XML-RPC choose the DTD
and SOAP choose XML schema.  Burlap choose the DTD.

<p>A major advantage of the DTD choice is that any Burlap request or
response is verifiable without any external schema.  The DTD defines
the entire grammar for Burlap.  So a tester can use that grammar to validate
that a Burlap client or server is conformant.  The grammar serves as
a testplan.

<p>The schema route gives more flexibility without adding expressive power.
The same object might have two different serializations if the
XML schema differs.  With Burlap or XML-RPC, the class has a
unique serialization.  With SOAP the different representations leave open
the possibility of interoperability issues.

</section>

<section title="Object Identifiers (URL)">

<p>Burlap uses a URL to locate the server object.  Usually, this will
be an HTTP url, but nothing in the Burlap protocol requires the
use of HTTP.  Since URLs are sufficient and already familiar, it seems
obvious to use them.

<p>An advantage of the particular encoding used for
EJB is that intelligent load balancers can use it to direct the
request to an owning server.  It's not required that the URL for
Burlap servers follow the same conventions as the EJB encoding.  In
the EJB encoding, the bean home interface is also a URL, easily
determined by removing the object identifier as the last name.  This
makes getting meta information easier.

<p>XML-RPC, in contrast, uses a combination of an HTTP URL and an
object identifier embedded in the request.  The object identifier is
part of the &lt;methodName> item.</p>

<example title="http://host/RPC2">
&lt;?xml version="1.0"?>
&lt;methodCall>
  &lt;methodName>examples.getStateName&lt;/methodName>
  &lt;params>
    &lt;param>
      &lt;value>&lt;i4>41&lt;/i4>&lt;/value>
    &lt;/param>
  &lt;/params>
&lt;/methodCall>
</example>

<p>SOAP also uses the URL to identify the target object.  The primary
model for SOAP appears to be more service-based rather than
object-based.  In other words, it appears that most SOAP servers will
only have a single URL and not have separate URLs for object
instances.  In constrast, Burlap heavily uses sub-URLs (path-info) to
identify object instances.  Nothing in the SOAP protocol precludes
using sub-URLs, so this isn't a limitation of the SOAP spec.  It just
appears to be counter to the culture of SOAP to use lots of URLs.  As
the SOAP spec says, objects-by-reference are not part of
the SOAP specification.

</section>

<section title="Method Selection and Overloading">

<p>Since EJB requires method overloading, Burlap supports it.  Burlap
implements method overloading with "method name mangling".  The type
of the method arguments becomes part of the method name.  To support
simpler clients, like script-based clients, one of the methods will
also typically respond to an unadorned method.</p>

<example title="add(2,3) call">
&lt;burlap:call>
  &lt;method>add_int_int&lt;/method>
  &lt;int>2&lt;/int>
  &lt;int>3&lt;/int>
&lt;/burlap:call>
</example>

<p>Because Burlap's encoding is lossy, some overloaded methods are
indistinguishable.  For example <var/add(int)/> and <var/add(short)/>
are both encoded as integer argument.  So only the integer version
will be callable.   The main reason for the lossy limitation is to
make Burlap less Java-dependent.  <var/int/>, <var/long/>, and
<var/double/> are supported by most languages.  We expect this won't
be a large limitation.  Few objects have methods with aliased methods.

<p>XML-RPC doesn't specifically address this issue.  Since its
method selection is very similar to Burlap's, name mangling would
be easy to implement in XML-RPC.

<p>SOAP's method selection is very different.  Each method has its
own XML element and namespace, presumably defined in some external
schema.  It could also support overloading with mangled element
names.  SOAP adds additional complexity to the method call with the
addition of a namespace.  It's not clear what value this adds, other
than additional complications, more tests, and interoperability
issues.

<p>Since a service/remote-object has a single set of methods,
selecting the object implicitly selects the method set or object
signature.  In theory, namespaces would allow a client to choose a
different object signature, but it's not clear what application would
need that.  Java and EJB doesn't need that complexity.</p>

<example title="SOAP add(2,3) call">
...
&lt;m:add xmlns:m="http://www.caucho.com/soap/calculator">
  &lt;a>2&lt;/a>
  &lt;b>3&lt;/b>
&lt;/m:add>
...
</example>

</section>

<section title="Method Parameters">

<p>Burlap method parameters are order-based like Java and essentially
every major programming language.  The number of parameters is fixed
for each method.  Varying-length parameters and extra parameters are
forbidden.  It's easily tested, unambiguous, and it's easy to write a fast
parsers.  The choice seems obvious and hardly worth discussion, but
SOAP dismissed position-based arguments.

<p>SOAP's method arguments are named by scheme-defined elements and
can appear in any order:</p>

<example>
...
&lt;m:myMethodCall xmlns:m="my-namespace">
  &lt;z>13&lt;/z>
  &lt;a>4&lt;/a>
  &lt;m>13&lt;/m>
&lt;/m:myMethodCall>
</example>

<p>For an EJB-implementation, the SOAP method arguments creates
several problems.  What name should the first argument be given?  Does
a client need an external schema or IDL to map the SOAP argument to
the function call?  How intelligent does a client or server have to be
to handle all the possible argument name assignments?  And how can all
this be tested?  The SOAP designers must have had some reason for this
choice, but it directly opposes the needs of an EJB server and client.

</section>

<section title='Inheritance (&lt;type>)'>

<p>Although most of Burlap's design avoids Java-specific requirements,
solving the inheritance problem requires the deserializer know what
Java class to create.  Burlap's solution makes Java implementations
easy, but doesn't significantly impede other languages and is
no less language-independent than SOAP or XML-RPC.

<p>Since Java is an object-oriented language, the type of a value may not
equal the declared type.  For example, an Object field might contain a
Car object or a Truck object.  To serialize the Car or Truck, Burlap
needs to add the object type in the protocol.  XML-RPC, in contrast,
has no accompanying type information, so a Java XML-RPC implementation
can't to create a Car from information in the protocol.  The Java serializer
also need to know if an array will be a Java array, Vector, or
ArrayList.  That's done with the &lt;type> tag in &lt;list> and &lt;map>.</p>

<example>
&lt;map>
&lt;type>com.caucho.example.Car&lt;/type>
&lt;string>color&lt;/string>&lt;string>red&lt;/string>
&lt;/map>
</example>

<p>This is a thorny problem.  The serializer must know the Java type,
but we'd prefer to keep Java-specific information out of the
protocol.  Burlap's solution allows non-Java clients with a
little extra work, and makes Java implementations easy.

<p>The heart of the problem how to map a type key in the
protocol to a language-specific type.  That map might appear in a
repository, in some external schema like IDL, or it can be encoded
in the protocol itself for a specific language.  Some simple clients
may not need this mapping.  A simple client in Perl, JavaScript or
Java client might use generic types, like ArrayList and Hashtables to
read the serialized values.  When they write request to a Java server,
they'll need to add the type, but protocol writing is much easier than
protocol parsing.

<p>It's not clear whether SOAP uses an external schema or a
repository service (<a href="http://www.w3.org/TR/wsdl">WSDL</a>) to
map to language types, but it's clear something external is required.
The following is a typical SOAP serialization of the Car:</p>

<example title="Car in SOAP">
&lt;e:Car xmlns:e="http://www.caucho.com/soap/cars.xml">
&lt;color>red&lt;/color>
&lt;/e:Car>
</example>

<p>How do you map from "e:Car" with namespace
"http://www.caucho.com/soap/cars" to the Java class
com.caucho.example.Car?  Since that's not part of the SOAP spec, there
must be an additional spec defining that mapping.  Since our
requirements required a testable implementation (repository is extra
work), no external IDL (so schema is out), the technique used by SOAP
was not appropriate for Burlap.

<p>Burlap doesn't preclude clients in other languages from using
their own types.  It would just require the addition of a repository
service or an external IDL.  In other words, it's no more complicated
than the type lookup SOAP requires for every language.  So Burlap
makes Java implementation easy, but doesn't make other languages hard.
When a Python client first encountered a "com.caucho.example.Car", it could
either use "Car" as a Python type, or query some yet-to-be-defined
service mapping to Python.  In other words, it's a trade-off, but it's
unavoidable and we believe the solution is fair to other languages.

</section>

<section title='Shared References (&lt;ref>)'>

<p>Shared references are an integral vital for serializing any
significant data structure.  Serializing a tree, for example, needs to
link a node's children and the parent in a circular set of
references.  Shared references are one of the necessary capabilities
missing from XML-RPC that Burlap adds.

<p>Burlap implements shared references with an implicit array of all
&lt;list> and &lt;map> objects.  Linking to an old object can then
just refer to the object's position in the array.  An advantage of
Burlap's approach is that it only requires a single pass through the
object.</p>

<example title="Burlap References">
&lt;map>
  &lt;type>com.muggle.Car&lt;/type>

  &lt;string>model&lt;/string>
  &lt;string>Ford Anglia&lt;/string>
&lt;/map>
...
&lt;map>
  &lt;type>com.muggle.Person&lt;/type>

  &lt;string>name&lt;/string>
  &lt;string>Arthur Weasley&lt;/string>

  &lt;string>car&lt;/string>
  &lt;ref>1&lt;/ref>
&lt;/e:Person>
</example>

<p>SOAP takes a different approach for shared references.  Declaring an object
uses an <var/id/> attribute and referring to the object uses a
<var/href/> attribute:</p>

<example title="SOAP references">
&lt;e:Car id="1">
  &lt;e:Model>Ford Anglia&lt;/e:Model>
&lt;/e:Car>
...
&lt;e:Person>
  &lt;e:Name>Arthur Weasley&lt;/e:Name>
  &lt;e:Car href="1"/>
&lt;/e:Person>
</example>

<p>The SOAP approach has the advantage that the receiving server
doesn't need to keep track of every object it's received.  In
addition, it can refer to any object, including strings.  With Burlap,
only &lt;map> and &lt;list> can share references, strings and byte
array can't be shared.  As a disadvantage, SOAP requires two passes
over the object before sending it and SOAP's approach requires the
use of XML attributes.

</section>

</section>

<section title="Conclusion">

<p>The Burlap design aimed at reducing the testing and implementation
complexity, and should give decent performance.  Because Burlap is a
wire protocol, EJB users don't care about the protocol details and we
could tailor Burlap to the specific requirements needed to support
EJB.

<p>The small size of the Burlap specification should not be confused
with its expressive power.  Burlap is fully capable of handling
Java/EJB calls.  The added complexity and "flexibility" of a
spec like SOAP just introduces ambiguity and adds to the testing
requirements without adding expressive power.

</section>
  </body>
</document>
