<document>
  <header>
    <product>resin</product>
    <resin-2.0>$article/xslt-benchmark.xtp</resin-2.0>
    <title>XSLT Benchmark</title>
  </header>

  <body>
    <summary/>

<section title='Introduction'>

As XML and XSLT move from the cutting edge into the mainstream,
they are used in more real applications, like the rendering of web
sites.  That transition now means that XSLT users need to pay
attention to the performance of their stylesheets, especially if the
XSLT engine is run for every web page served.

<sidebar>
Resin's XSLT engine was about three times faster than
Xalan and about 80% faster than Saxon.
</sidebar>

<p/>DataPower Technology has created a comprehensive XSLT benchmark,
called <a href="http://www.datapower.com/XSLTMark/">XSLTMark</a> to
evaluate the performance of XSLT engines.  Like all synthetic
benchmarks, it's not as good as testing your own application, but it
is comprehensive and gives a good indication of XSLT engine
performance.

<p/>Although Resin has included an XSLT engine since 1998, Resin's
reputation as a fast servlet and JSP engine has overshadowed its
XSLT implementation.  This comparison reintroduces Resin's XSLT
engine and compares its performance with the other popular Java
XSLT engines.

<p/>For this benchmark, Resin's XSLT engine was about three times faster than
Xalan and about 80% faster than Saxon.

</section>

<section title='Data'>

The following Java XSLT engines were tested:
<ul>
<li><a href="http://www.caucho.com">Resin 2.0.5</a>
<li><a href="http://xml.apache.org/xalan-j/index.html">Xalan 2.2 D14</a>
<li><a href="http://saxon.sourceforge.org">Saxon 6.5</a>
<li>XT
</ul>

<p/>The XSLT benchmark drivers was taken from the XSLTMark website and all
comparisons were done on the same RedHat 7.2 system (dual 800Mhz
Pentium III).

<p/>XSLTMark summarizes the results from the individual tests by
calculating a number of means.  For benchmarks of this type, the
geometric mean is the most valid, since the geometric mean weighs all
test equally.  In other words, even though <em>queens</em>
is slow and <em>identity</em> is fast for every engine, the geometric
mean treats them both identically.

<p/>Using the geometric mean, Resin is:
<ul>
<li>10% faster than XT
<li>80% faster than Saxon
<li>3 times faster than Xalan
</ul>

<p/><table border=2>
<tr><th>&nbsp;<th colspan=3>Mean
<tr><th>Engine<th>Arithmetic<th>Geometric<th>Harmonic
<tr><td>Resin<td align='right'>548.64<td align=right>383.67<td align=right>196.55
<tr><td>XT<td align=right>465.58<td align=right>323.85<td align=right>162.83
<tr><td>Saxon<td align=right>331.35<td align=right>209.25<td align=right>94.94
<tr><td>Xalan<td align=right>204.16<td align=right>121.12<td align=right>53.87
</table>

<p/>The following graph shows the performance for each test.  The
units are KB/s.

<p/><ct:img src='xslt-perf.gif'/>

<section title='Resin 2.0.5'>

<pre>
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:22:11 PST, 11 Jan 2002

Results for driver ResinDriver

                          prep  exec                                       
Test Component     iter   ms    ms     KB In     KB Out    KB/s      status

alphabetize         10    2021  1292   196       156       136.66    OK
attsets             100   1846  578    121       254       324.9     OK
avts                100   1897  1398   1966      1155      1116.34   OK
axis                100   1994  297    35        57        155.36    OK
backwards           100   1965  486    260       260       535.6     OK
bottles             100   1923  568    4         1130      999       OK
breadth             100   1823  558    415       199       551.11    OK
brutal              100   2504  1076   130       451       270.64    OK
chart               100   2168  748    121       467       393.76    OK
creation            50    1891  1191   983       1520      1050.81   OK
current             100   1904  165    18        27        141.45    OK
dbonerow            10    2096  10677  19621     9         919.3     OK
dbtail              100   1909  1109   1966      502       1112.83   OK
decoy               100   2133  3500   1966      1972      562.67    OK
depth               100   4     394    588       371       1219.46   OK
encrypt             10    1973  286    196       196       687.38    OK
functions           100   1957  4429   1966      616       291.5     OK
game                100   2087  420    260       53        373.54    OK
html                100   1913  149    29        53        275.6     CHK OUTPUT
identity            10    1741  1421   1962      1962      1381.06   OK
inventory           100   2129  378    184       200       509.6     OK
metric              100   1903  371    88        61        202.42    CHK OUTPUT
number              100   1956  1163   18        79        42.32     NO REF
oddtemplate         100   1855  279    16        13        54.95     OK
patterns            100   1920  2552   1966      1972      771.68    OK
prettyprint         10    1958  1342   196       1405      596.75    OK
priority            100   2174  328    91        55        223.6     OK
products            100   2013  442    196       43        271.54    OK
queens              10    2023  440    0         17        19.73     OK
reverser            100   1937  260    129       125       491.29    OK
stringsort          10    2001  1613   1962      1933      1207.59   OK
summarize           100   1936  417    712       27        887.57    OK
total               100   1915  126    121       36        625.47    OK
tower               20    2037  1000   1         1381      691.05    OK
trend               100   1867  3147   176       804       155.92    OK
union               100   1931  73     16        10        185.28    OK
xpath               100   1858  97     28        18        244.64    OK
xslbench1           100   2385  295    36        732       1303.13   CHK OUTPUT
xslbench2           20    2191  1487   2920      2109      1691.3    CHK OUTPUT
xslbench3           20    19    1101   2920      25        1337.72   CHK OUTPUT

Aggr. Results       2740  67346 44250  38601     19490     656.41    
Arithmetic Mean           1924.17                            548.64    
Geometric Mean                                             383.67    
Harmonic Mean                                              196.55    
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL)      35/40 = 87.5% 
</pre>

</section>

<section title='XT'>

<pre>
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:24:46 PST, 11 Jan 2002

Results for driver XTDriver

                          prep  exec                                       
Test Component     iter   ms    ms     KB In     KB Out    KB/s      status

alphabetize         10    247   992    196       156       178.07    OK
attsets             100   7     724    121       255       260.53    OK
avts                100   6     1788   1966      1156      873.31    OK
axis                100   10    314    35        60        153.02    OK
backwards           100   5     1301   260       261       200.72    CHK OUTPUT
bottles             100   9     1044   4         1130      543.52    OK
breadth             100   3     767    415       200       402.02    OK
brutal              100   8     678    130       394       386.88    OK
chart               100   8     596    121       359       403.32    OK
creation            50    5     1610   983       1520      777.6     OK
current             100   10    343    18        29        71.18     OK
dbonerow            10    7     14146  19621     8         693.85    OK
dbtail              100   3     1682   1966      503       734.22    OK
decoy               100   12    3959   1966      1974      497.68    OK
depth               100   2     850    588       373       566.23    OK
encrypt             10    3     326    196       196       603.3     OK
functions           100   7     7437   1966      618       173.76    OK
game                100   8     292    260       43        520.22    OK
html                100   3     174    29        40        199.52    CHK OUTPUT
identity            10    2     1735   1962      1962      1131.17   OK
inventory           100   3     425    184       202       455.19    OK
metric              0     0     0      0         0         ?         ERROR
number              100   9     1060   18        79        46.43     NO REF
oddtemplate         100   3     147    16        15        109.95    OK
patterns            100   3     3172   1966      1974      621.16    OK
prettyprint         10    11    1681   196       1405      476.41    OK
priority            100   6     209    91        57        354.88    OK
products            100   3     951    196       40        124.41    OK
queens              10    13    669    0         17        13.1      OK
reverser            100   2     1151   129       129       112.63    OK
stringsort          10    101   2828   1962      1933      688.8     OK
summarize           100   7     605    712       29        613.14    OK
total               100   2     221    121       33        350.63    OK
tower               20    147   2168   1         1381      318.83    OK
trend               100   2     5103   176       806       96.34     OK
union               100   3     127    16        11        113.04    OK
xpath               100   2     125    28        20        196.48    OK
xslbench1           100   7     287    36        663       1220.7    OK
xslbench2           20    3     1838   2920      2129      1373.68   OK
xslbench3           20    2     1517   2920      25        970.86    OK

Aggr. Results       2780  686   63567  44218     21904     520.11    
Arithmetic Mean           18.54                            465.58    
Geometric Mean                                             323.85    
Harmonic Mean                                              162.83    
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL)      37/40 = 92.5% 
</pre>

</section>

<section title='Saxon 6.5'>

<pre>
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:38:35 PST, 11 Jan 2002

Results for driver SaxonDriver

                          prep  exec                                       
Test Component     iter   ms    ms     KB In     KB Out    KB/s      status

alphabetize         10    32    1472   196       156       120       OK
attsets             100   17    1244   121       255       151.59    OK
avts                100   12    2247   1966      1156      694.89    OK
axis                100   8     694    35        60        69.16     OK
backwards           100   5     1704   260       261       153.22    OK
bottles             100   22    1626   4         1130      348.97    OK
breadth             100   6     1574   415       200       195.87    OK
brutal              100   19    1323   130       623       285       OK
chart               100   8     1295   121       726       327.39    OK
creation            50    12    2134   983       1520      586.65    OK
current             100   5     786    18        29        31        OK
dbonerow            10    8     12633  19621     12        777.11    OK
dbtail              100   7     2049   1966      503       602.69    OK
decoy               100   10    4898   1966      1974      402.26    OK
depth               100   3     1266   588       373       380.13    OK
encrypt             10    170   599    196       196       328.33    OK
functions           100   9     5321   1966      618       242.85    OK
game                100   6     518    260       79        327.94    OK
html                100   10    421    29        77        126.88    OK
identity            10    4     2531   1962      1962      775.41    OK
inventory           100   5     797    184       201       242.67    OK
metric              100   5     735    88        65        104.83    OK
number              100   10    815    18        79        60.39     NO REF
oddtemplate         100   5     859    16        15        18.76     OK
patterns            100   5     3900   1966      1974      505.2     OK
prettyprint         10    12    3779   196       1405      211.91    OK
priority            100   7     422    91        57        175.64    OK
products            100   4     638    196       62        202.43    OK
queens              10    17    1064   0         17        8.23      OK
reverser            100   4     3613   129       129       35.87     OK
stringsort          10    12    2850   1962      1933      683.48    OK
summarize           100   4     800    712       29        463.62    OK
total               100   4     385    121       51        224.23    OK
tower               20    10    2367   1         1381      292.02    OK
trend               100   4     3913   176       806       125.62    OK
union               100   4     309    16        11        46.3      OK
xpath               100   4     330    28        20        74.28     OK
xslbench1           100   198   583    36        877       784.01    OK
xslbench2           20    6     2225   2920      2556      1230.76   OK
xslbench3           20    5     1765   2920      32        836.45    OK

Aggr. Results       3080  698   78484  44596     23631     434.66    
Arithmetic Mean           17.45                            331.35    
Geometric Mean                                             209.25    
Harmonic Mean                                              94.94     
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL)      40/40 = 100%
</pre>

</section>

<section title='Xalan 2.2 D14'>

<pre>
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:09:09 PST, 11 Jan 2002

Results for driver Xalan2JDriver

                          prep  exec                                       
Test Component     iter   ms    ms     KB In     KB Out    KB/s      status

alphabetize         10    64    3221   196       156       54.84     OK
attsets             100   50    1792   121       255       105.26    OK
avts                100   22    4993   1966      1156      312.73    OK
axis                100   29    1042   35        60        46.11     OK
backwards           100   11    1977   260       261       132.09    OK
bottles             100   31    2597   4         1130      218.5     OK
breadth             100   17    2341   415       200       131.72    OK
brutal              100   72    1768   130       394       148.36    OK
chart               100   43    2111   121       359       113.87    OK
creation            50    11    3684   983       1520      339.83    OK
current             100   15    909    18        29        26.86     OK
dbonerow            10    17    29714  19621     8         330.32    OK
dbtail              100   11    2920   1966      503       422.93    OK
decoy               100   227   11657  1966      1974      169.02    OK
depth               100   12    2036   588       373       236.39    OK
encrypt             10    11    845    196       196       232.75    OK
functions           100   25    22354  1966      618       57.81     OK
game                100   14    911    260       43        166.74    OK
html                100   14    828    29        47        45.94     CHK OUTPUT
identity            10    17    5043   1962      1962      389.17    OK
inventory           100   11    1305   184       202       148.24    OK
metric              100   14    1523   88        65        50.66     OK
number              100   16    1872   18        79        26.29     NO REF
oddtemplate         100   15    1268   16        15        12.75     OK
patterns            100   14    8132   1966      1974      242.29    OK
prettyprint         10    39    2134   196       1405      375.28    OK
priority            100   19    881    91        57        84.19     OK
products            100   16    2532   196       40        46.73     OK
queens              10    74    1799   0         17        4.87      OK
reverser            100   23    970    129       129       133.65    OK
stringsort          10    16    5591   1962      1933      348.4     OK
summarize           100   12    1339   712       29        277.03    OK
total               100   10    639    121       33        121.27    OK
tower               20    27    2796   1         1381      247.22    OK
trend               100   9     32396  176       806       15.17     OK
union               100   13    574    16        11        25.01     OK
xpath               100   112   611    28        20        40.2      OK
xslbench1           100   95    870    36        665       403.65    OK
xslbench2           20    75    3735   2920      2139      677.33    OK
xslbench3           20    16    1407   2920      25        1046.76   OK

Aggr. Results       2980  1325  174289 44567     22244     191.67    
Arithmetic Mean           33.97                            204.16    
Geometric Mean                                             121.12    
Harmonic Mean                                              53.87     
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL)      39/40 = 97.5% 
</pre>

</section>

</section>

<section title="Conclusion">

Resin's three times performance gain over Xalan and 80% gain
over Saxon is a significant advantage for web sites using Resin to
render their pages from XML.  Besides the obvious improved response
times, the better performance means that more web sites can take
advantage of XSLT instead of avoiding it because of performance
worries.

<p/>As XML becomes more important for web services, the core engine
performance will also become more important.  Resin's fast XML and
XSLT support strengthens Resin's position as a leading Java XML
application server.

</section>
  </body>
</document>
