\subsection{1-d Synthetic Sine Data}
\label{sec:sin}

<<echo=false,results=hide>>=
library(tgp)
##options(width=65)
seed <- 0; set.seed(seed)
@ 

Consider 1-dimensional simulated data which is partly a mixture of
sines and cosines, and partly linear.
\begin{equation}
    z(x) = \left\{ \begin{array}{cl}
    \sin\left(\frac{\pi x}{5}\right)
        + \frac{1}{5}\cos\left(\frac{4\pi x}{5}\right) & x < 9.6 \\
	x/10-1 & \mbox{otherwise}
   \end{array} \right. 
   \label{e:sindata}
\end{equation}

The {\sf R} code below obtains $N=100$ evenly spaced samples from this
data in the domain $[0,20]$, with noise added to keep things
interesting.  Some evenly spaced predictive locations {\tt XX} are
also created.
<<>>=
X <- seq(0,20,length=100)
XX <- seq(0,20,length=99)
Ztrue <- (sin(pi*X/5) + 0.2*cos(4*pi*X/5)) * (X <= 9.6)
lin <- X>9.6; 
Ztrue[lin] <- -1 + X[lin]/10
Z <- Ztrue + rnorm(length(Ztrue), sd=0.1)
@ 

By design, the data is clearly nonstationary in its mean.  Perhaps not
knowing this, a good first model choice for this data might be a GP.
<<>>=
sin.bgp <- bgp(X=X, Z=Z, XX=XX, verb=0)
@ 
\begin{figure}[ht!]
\centering
<<label=sin-bgp,fig=TRUE,echo=TRUE,width=7,height=5,include=FALSE>>=
plot(sin.bgp, main='GP,', layout='surf')
lines(X, Ztrue, col=4, lty=2, lwd=2)
@
<<echo=false,results=hide>>=
graphics.off()
@
\includegraphics[trim=0 25 0 25]{tgp-sin-bgp}
%\vspace{-0.25cm}
\caption{Posterior predictive distribution using {\tt
    bgp} on synthetic sinusoidal data: mean and 90\% pointwise credible
  interval.  The true mean is overlayed with a dashed line.}
\label{f:sin:bgp}
\end{figure}
Figure \ref{f:sin:bgp} shows the resulting posterior predictive
surface under the GP.  Notice how the (stationary) GP gets the
wiggliness of the sinusoidal region, but fails to capture the
smoothness of the linear region.  The true mean (\ref{e:sindata}) is
overlayed with a dashed line.

So one might consider a Bayesian treed linear model (LM) instead.
<<>>=
sin.btlm <- btlm(X=X, Z=Z, XX=XX)
@
MCMC progress indicators show successful {\em grow} and {\em prune}
operations as they happen, and region sizes $n$ every 1,000 rounds.
Specifying {\tt verb=3}, or higher will show echo more successful tree
operations, i.e., {\em change}, {\em swap}, and {\em rotate}.
 
\begin{figure}[ht!]
\centering
<<label=sin-btlm,fig=TRUE,echo=TRUE,width=7,height=5,include=FALSE>>=
plot(sin.btlm, main='treed LM,', layout='surf')
lines(X, Ztrue, col=4, lty=2, lwd=2)
@
<<echo=false,results=hide>>=
graphics.off()
@
\includegraphics[trim=0 25 0 25]{tgp-sin-btlm}
%\vspace{-0.25cm}
<<label=sin-btlmtrees,fig=TRUE,echo=TRUE,width=12,height=10>>=
tgp.trees(sin.btlm)
@
<<echo=false,results=hide>>=
graphics.off()
@
\vspace{-1cm}
\caption{{\em Top:} Posterior predictive distribution
  using {\tt btlm} on synthetic sinusoidal data: mean and 90\%
  pointwise credible interval, and MAP partition ($\hat{\mathcal{T}}$).
  The true mean is overlayed with a dashed line. {\em
    Bottom:} MAP trees for each height encountered in the Markov
  chain showing $\hat{\sigma}^2$ and the number of observation $n$,
  at each leaf.}
\label{f:sin:btlm}
\end{figure}
Figure \ref{f:sin:btlm} shows the resulting posterior predictive
surface ({\em top}) and trees ({\em bottom}).  The MAP partition
($\hat{\mathcal{T}}$) is also drawn onto the surface plot ({\em top})
in the form of vertical lines.  The treed LM captures the smoothness
of the linear region just fine, but comes up short in the sinusoidal
region---doing the best it can with piecewise linear models.

The ideal model for this data is the Bayesian treed GP because it can
be both smooth and wiggly.
<<>>=
sin.btgp <- btgp(X=X, Z=Z, XX=XX, verb=0)
@ 
\begin{figure}[ht!]
\centering
<<label=sin-btgp,fig=TRUE,echo=FALSE,width=7,height=5,include=FALSE>>=
plot(sin.btgp, main='treed GP,', layout='surf')
lines(X, Ztrue, col=4, lty=2, lwd=2)
@
<<echo=false,results=hide>>=
graphics.off()
@
\includegraphics[trim=0 25 0 25]{tgp-sin-btgp}
%\vspace{-1cm}
\caption{Posterior predictive distribution
  using {\tt btgp} on synthetic sinusoidal data: mean and 90\%
  pointwise credible interval, and MAP partition ($\hat{\mathcal{T}}$)
  \label{f:sin:btgp}.  The true mean is overlayed with a dashed line.}
\end{figure}
Figure \ref{f:sin:btgp} shows the resulting posterior predictive
surface ({\em top}) and MAP $\hat{\mathcal{T}}$ with height=2.

Finally, speedups can be obtained if the GP is allowed to jump to the
LLM \cite{gra:lee:2008}, since half of the response surface is {\em
  very} smooth, or linear.  This is not shown here since the results
are very similar to those above, replacing {\tt btgp} with {\tt
  btgpllm}.  Each of the models fit in this section is a special case
of the treed GP LLM, so a model comparison is facilitated by fitting
this more general model.  The example in the next subsection offers
such a comparison for 2-d data.  A followup in Appendix
\ref{sec:traces} shows how to use parameter traces to extract the
posterior probability of linearity in regions of the input space.
