\documentclass[nogin,a4paper]{article}
%\usepackage[utf8]{inputenc} % doesn not work with check --as-cran

%\VignetteIndexEntry{sos4R: Accessing Sensor Observation Services from R}
%\VignetteDepends{xtable}

% RUN WITH 
% Sweave(file = "/home/daniel/workspace/sos4R/inst/doc/sos4R.Rnw")
% OR
% Sweave(file = "D:/workspace/sos4R/inst/doc/sos4R.Rnw")
% OR
% library(cacheSweave)
% Sweave("sos4R.Rnw", driver = cacheSweaveDriver)

\usepackage{graphicx}
\usepackage{color}
\usepackage{Sweave}
\usepackage{csquotes}

\newcommand{\strong}[1]{{\normalfont\fontseries{b}\selectfont #1}}
\newcommand{\fivetwon}{$52\,^{\circ}{\rm North}$ }
\newcommand{\code}[1]{{\tt #1}}
\let\pkg=\strong

\usepackage[colorlinks=true,urlcolor=blue,linkcolor=blue]{hyperref}

% otherwise removes line breaks etc.
\SweaveOpts{keep.source=TRUE}

% some global options and helper function
<<options,echo=FALSE,results=hide,print=FALSE>>= 
.goOnline <- FALSE # triggers whether data is downloaded and stored in /vignettes folder
.verbose <- TRUE

#library("cacheSweave")
#setCacheDir("/tmp/cacheSweave/sos4R")

.getFilePath <- function(name) {
	if(regexpr(pattern = ".RData", text = name) > 0)
		.fileEnding <- ""
	else .fileEnding <- ".xml"

	# since Vignette is in /vignettes/ this doesn't seem to work anymore...
#	.path <- paste(.find.package("sos4R", lib.loc = NULL), "/vignettes/",
#			name, .fileEnding, sep = "")
	.path <- paste0(name, .fileEnding)
	if(.verbose) cat("Loading file", .path, "\n")
	
	return(.path)
}
@

% sets the figure size globally
\setkeys{Gin}{width=0.6\textwidth} 

\begin{document}
\SweaveOpts{concordance=TRUE}

% http://www.stat.auckland.ac.nz/~stat782/downloads/Sweave-customisation.pdf
\DefineVerbatimEnvironment{Sinput}{Verbatim} {xleftmargin=2em}
\DefineVerbatimEnvironment{Soutput}{Verbatim}{xleftmargin=2em}
\DefineVerbatimEnvironment{Scode}{Verbatim}{xleftmargin=2em}
\fvset{listparameters={\setlength{\topsep}{0pt}}}
\renewenvironment{Schunk}{\vspace{\topsep}}{\vspace{\topsep}}
<<echo=false>>=
options(width=60)
options(SweaveHooks=list(fig=function()
					par(mar=c(5.1, 4.1, 1.1, 2.1))))
@


\title{Accessing Data from Sensor Observation Services:\\ the \pkg{sos4R} Package}
\author{Daniel N\"{u}st\footnote{Institute for Geoinformatics, University of Muenster, Germany.}\\ {\tt daniel.nuest@uni-muenster.de}\\ {\tt \url{http://www.nordholmen.net/sos4r}}}

\date{\today}

\maketitle

\begin{abstract}
The sos4R package provides simple yet powerful access to OGC Sensor Observation Service instances. The package supports both encapsulation and abstraction from the service interface for novice users as well as powerful request building for specialists.

sos4R is motivated by the idea to close the gap between the Sensor Web and tools for (geo-)statistical analyses. It implements the core profile of the SOS specification and supports temporal, spatial, and thematical filtering of observations. This document briefly introduces the SOS specification. The package's features are explained extensively: exploration of service metadata, request building with filters, function exchangeability, result data transformation.

The package is published under GPL 2 license within the geostatistics community of \fivetwon Initiative for Geospatial Open Source Software.

\end{abstract}

\tableofcontents


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}

The \pkg{sos4R} package provides classes and methods for retrieving data from an OGC Sensor Observation Service (Na, 2007). The goal of this package is to provide easy access with a low entry threshold for everyone to information available via SOSs. The complexity of the service interface shall be shielded from the user as much as possible, while still leaving enough possiblities for advanced users. This package uses S4 classes and methods style (Chambers, 1998).

At the current state, the output is fixed to a standard data.frame with attributed columns for metadata. In future releases a tighter integration is planned with upcoming space-time packages regarding data structures and classes. 

The motivation to write this package was born out of perceiving a missing link between the Sensor Web community (known as Sensor Web Enablement (SWE) Initiative\footnote{\url{http://www.opengeospatial.org/projects/groups/sensorweb}} in the OGC realm) and the community of (geo-)statisticians (N\"ust, 2011). While the relatively young SWE standards get adopted more by data owners (like governmental organizations), we see a high but unused potential for more open data and spatio-temporal analyses based on it. \pkg{sos4R} can help enabling this.

The project is part of the geostatistics community\footnote{\url{http://52north.org/communities/geostatistics/}} of the \fivetwon Initiative for Geospatial Open Source Software\footnote{\url{http://52north.org/}}. \pkg{sos4R} is available, or will be available soon, on CRAN\footnote{\url{http://cran.r-project.org/}} (the Comprehensive R Archive Network).

On the package home page, \url{http://www.nordholmen.net/sos4r/}, you can stay updated with the development blog and find example code and services.

This software is released under a GPL 2 license\footnote{\url{http://www.gnu.org/licenses/gpl-2.0.html}} and contributions are very welcome---please see section \ref{sec:developing}.

The package \pkg{sos4R} is loaded by 
<<load,echo=TRUE>>= 
library("sos4R")
@

This document was build for \textbf{package version \Sexpr{sessionInfo()[["otherPkgs"]][["sos4R"]][["Version"]]}}.

\subsection{Quick Start}

If you are familiar with the OGC SOS standard specification, know how to use content assist in your favourite R editor, and you do not need to extend the functionality of \pkg{sos4R}, then feel free to fast forward to Section \ref{sec:gettingstarted} and get started straightaway.

\subsection{Related Specifications}

The Open Geospatial Consortium\footnote{\url{http://www.opengeospatial.org/}} (OGC) is an organisation which provides standards for handling geospatial data on the internet, thereby ensuring interoperability. The \textbf{Sensor Observation Service (SOS)} is such a standard and provides a well-defined interface for data warehousing of measurements and observations made by all kinds of sensors. This vignette describes the classes, methods and functions provided by \pkg{sos4R} to request these observations from a SOS.

Providing data via web services is more powerful than local file copies (with issues like being outdated, redundancy, \ldots). Flexible filtering of data on the service side reduces download size. That is why SOS operations can comprise flexible subsetting in temporal, spatial and thematical domain. For example \enquote{Get measurements from sensor urn:mySensor:001 for the time period from 01/12/2010 to 31/12/2010 where the air temperature below zero degrees}.

In general, the SOS supports two methods of requesting data: (i) HTTP GET as defined in the OOSTethys best practice document\footnote{This best-practice paper takes the place of a section in the specification that was left out by mistake. It is well established and (loosely) followed by several SOS implementations. See \url{http://www.oostethys.org/best-practices/best-practices-get}.} with key-value-pair (KVP) encoding of request, and (ii) POST as defined in the standard document with requests encoded in eXtensible Markup Language (XML)\footnote{\url{http://www.w3.org/XML/}}. Both request types always returns XML documents as response.

OGC Standards that are referenced and used, by the SOS standard are as follows.

\begin{description}
\item[Observations and Measurements (O\&M)] O\&M (Cox, 2007) defines the markup of sensor measurements results. An observation consists of information about the observerd geographic feature, the time of observation, the sensor, the observed phenomenon, and the observation's actual result.
\item[Sensor Model Language (SensorML)] SensorML (Botts, 2007) is used for sensor metadata descriptions (calibration information, inputs and outputs, maintainer).
\item[Geography Markup Language (GML)] (Portele, 2003) defines markup for geographical features (points, lines, polygons, \ldots).
\item[SweCommon] SWE Common defines data markup. It is contained in the SensorML specification (see above).
\item[Filter Encoding] Filter Encoding (Vretanos, 2005) defines operators and operands for filtering values.
\item[OWS Common] OGC Web Services Common (Whiteside, 2007) models service related elements that are reusable across several service specifications, like exception handling.
\end{description}

\subsection{Terms and Definitions}

The OGC has a particular set of well-defined terms that might differ from usage of words in specific domains. The most important are as follows\footnote{Based on \url{http://de.wikipedia.org/wiki/Sensor_Observation_Service}}.

\begin{description}
\item[Feature of Interest (FOI)] The FOI represents the geo-object, for which measurements are made by sensors. It is ordinarily used for the spatial referencing of measuring points, i.e. the geoobject has coordinates like latitude, longitude and height. The feature is project specific and can be anything from a point (e.g. the position of a measuring station) or a real-world object (e.g. the region that is observed).
\item[Observation] The observation delivers a measurement (result) for a property (phenomenon) of an observed object (FOI). The actual value is created by a sensor or procedure. The phenomenon was measured at a specific time (sampling time) and the value was generated at a specific point in time (result time). These often coincide so in practice the sampling time is often used as the point in time of an obsrevation.
\item[Offering] The offering is a logical collection of related observations (similar to a layer in mapping applications) which a service offers together.
\item[Phenomenon] A phenomenon is a property (physical value) of a geographical object, e.g. air temperature, wind speed, concentration of a pollutant in the atmosphere, reflected radiation in a specific frequency band (colours).
\item[Procedure] A procedure creates the measurement value of an observation. The source can be a reading from a sensor, simulation or a numerical process.
\end{description}

A more extensive discussion is available in the the O\&M specification (Cox, 2007). The Annex B of that document contains the examples of applicating some terms to specific domains, aerosol analysis and earth observations, which are repeated here for elaboration in table \ref{tab:terms}.

\begin{center}
\begin{table}[ht]
\begin{tabular}{p{0.3\textwidth}p{0.3\textwidth}p{0.3\textwidth}}
	\textbf{O\&M} & \textbf{Particulate Matter\newline 2.5 Concentrations} & \textbf{Earth Observations} \\
	\hline
	\hline\noalign{\smallskip}
	result & 35 ug/m3 & observation value, measurement value
	\\
	\hline\noalign{\smallskip}
	procedure & U.S. EPA Federal Reference Method for PM 2.5
	& method, sensor \\
	\hline\noalign{\smallskip}
	observedProperty & Particulate Matter 2.5 & parameter, variable
	\\
	\hline\noalign{\smallskip}
	featureOfInterest & troposphere & media (air, water,
	\ldots), Global Change Master Directory \enquote{Topic} \\
	\hline
\end{tabular}
\caption{Domain specific variants of O\&M terms.}
\label{tab:terms}
\end{table}
\end{center}

A good and extensive \textbf{introduction into the whole field of SWE}, including its history, and an analysis of the current state of the art and future developments is provided in a recent paper (Br\"oring, 2011).


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Supported Features}
\label{sec:supported}

The package provides accessor functions for the supported parameters. It is recommended to access options from the lists returned by these functions instead of hard-coding them into scripts.

This section only lists the possibilities. Explanations follow in this document or can be found in the SOS specification.

<<supported01,echo=TRUE,print=TRUE>>=
SosSupportedOperations()
SosSupportedServiceVersions()
SosSupportedConnectionMethods()
SosSupportedResponseFormats()
@

The response format \enquote{text/csv} is not standard conform, but used by services as a well established alternative to XML encodings.

<<supported02,echo=TRUE,print=TRUE>>=
SosSupportedResponseModes()
SosSupportedResultModels()
@

The output of the following calls are named lists (the name being the same as the value) which are simplified here for brevity using \code{toString()}.

<<supported03,echo=TRUE,eval=FALSE,print=FALSE>>=
SosSupportedSpatialOperators()
@
<<supported04,echo=FALSE,eval=TRUE,print=TRUE>>=
toString(SosSupportedSpatialOperators())
@

<<supported05,echo=TRUE,eval=FALSE,print=FALSE>>=
SosSupportedTemporalOperators()
@
<<supported06,echo=FALSE,eval=TRUE,print=TRUE>>=
toString(SosSupportedTemporalOperators())
@


\subsection{Supported Implementations}

\pkg{sos4R} supports the core profile of the SOS specification. But the possible markups for observations is extremely manifold due to the flexibility of the O\&M specification. Sadly, there is no common application profile for certain types of observations, like simple measurements.

Therefore, the undocumented profile of the \boldmath{\fivetwon}\textbf{ SOS implementation}\footnote{\url{http://52north.org/communities/sensorweb/sos/}} was used as a guideline. It is not documented outside of the source code. Observations returned by instances of this implementation are most likely to be processed out of the box.

In the author's experience, \textbf{OOSThetys SOS implementations}\footnote{\url{http://www.oostethys.org/}} utilise the same or at least very similar profile, so responses of these service instances are probably parsed without further work as well.

An incomplete list of \textbf{tested services} can be found in section \ref{sec:gettingstarted}. Please share your experiences with other SOS implementations with the developers and users of \pkg{sos4R} (see section \ref{sec:support}).
 

\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Default Options}
\label{sec:default}

Two kinds of default values can be found in (function calls in) \pkg{sos4R}: (i) default depending on other function parameters, and (ii) global defaults. Global defaults can be inspected (not changed!) using the following functions. If you want to use a different value please change the respective argument in function calls.

<<default,echo=TRUE,print=TRUE>>=
SosDefaultConnectionMethod()
SosDefaults()
@

The process of data download also comprises (i) building requests, (ii) decoding responses, and (iii) applying the correct R data type to the respective data values. This mechanism is explained in detail in see section \ref{sec:exchangeability}. The package comes with a set of predefined encoders, decoders and converters (output not shown here as it is very extensive).

<<converterFunc,echo=TRUE,eval=FALSE>>=
SosEncodingFunctions()
SosParsingFunctions()
SosDataFieldConvertingFunctions()
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Creating a SOS connection}
\label{sec:creatingconnection}

The method \code{SOS()} is a construction method for classes encapsulating a connection to a SOS. It prints out a short statement when the connection was successfully established (i.e. the capabilities document was received) and returns an object of class \code{SOS}.


<<conn,echo=FALSE,print=FALSE,eval=TRUE>>=
.fileMySOS <- "mySOS.RData" 
if(.goOnline) {
	mySOS <- SOS(url = "http://v-swe.uni-muenster.de:8080/WeatherSOS/sos")
	save(mySOS, file = .fileMySOS)
} else {
	load(.fileMySOS)
}
@

<<conn,echo=TRUE,print=FALSE,eval=FALSE>>= 
mySOS <- SOS(url = "http://v-swe.uni-muenster.de:8080/WeatherSOS/sos")
@

To create a SOS connection you only need the URL of the service (i.e. the URL which can be used for HTTP GET or POST requests). The service connection created above is used for all examples throughout this document.

All parameters except the service endpoint are optional and use default settings (see also section \ref{sec:default}):
 
\begin{itemize}
    \item \code{method}: The transport protocol. Currently available are \Sexpr{toString(names(SosSupportedConnectionMethods()))}, the default is \Sexpr{SosDefaultConnectionMethod()}. \code{GET} is less powerful, especially regarding filtering operations. Section \ref{subsec:converters} contains an example of such a connection, whereas the majority of examples is based on a \code{POST} connection.
    \item \code{version}: The service version. Currently available version(s) is/are \Sexpr{SosSupportedServiceVersions()}.
    \item \code{parsers}: The list of parsing functions. See section \ref{subsec:parsers}.
    \item \code{encoders}: The list of encoding functions. See section \ref{subsec:encoders}.
    \item \code{dataFieldConverters}:  The list of conversion functions. See section \ref{subsec:converters}.
    \item \code{curlHandle}, \code{curlOptions}: Settings of the package \pkg{RCurl}, which is used for HTTP connections. Please consult the packags specification before using this.
    \item \code{timeFormat}: The time format to be used or decoding and encoding time character strings to and from \code{POSIXt} classes, the default is \Sexpr{sosDefaultTimeFormat}.
    \item \code{verboseOutput}: Trigger parameter for extensive debugging information on the console, see section \ref{subsec:verbose}.
	\item \code{switchCoordinates}: Switches all coordinates that are encountered during the \textbf{parsing phase}, such as in an element like \code{<gml:lowerCorner>117.3 -41.5</gml:lowerCorner>}.
\end{itemize}

There are accessor methods for the slots of the class. The encoders, parsers and converters are described extensively in section \ref{sec:exchangeability}.

<<connDetails1,echo=TRUE,eval=FALSE>>= 
sosUrl(mySOS)
sosTitle(mySOS)
sosAbstract(mySOS)
sosVersion(mySOS)
sosTimeFormat(mySOS)
sosMethod(mySOS)
@

<<connDetails2,echo=TRUE,eval=FALSE>>= 
sosEncoders(mySOS)
sosParsers(mySOS)
sosDataFieldConverters(mySOS)
@

Print and summary methods are available for important classes, like \code{SOS}.

<<connDetails3,echo=TRUE,eval=TRUE>>= 
mySOS
summary(mySOS)
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{SOS Operations}

\pkg{sos4R} implements the SOS core profile of version 1.0.0 comprising the operations GetCapabilities, DescribeSensor and GetObservation. This document focusses on the practical usage of the operations, so the reader is refered to the specification document for details.

The methods mirroring the SOS operations all contain debugging parameters \code{inspect} and \code{verbose} as described in section \ref{subsec:inspect}.

\subsection{GetCapabilities}
\label{subsec:getcap}

The GetCapabilities operations is automatically conducted during the connecting to a SOS instance. The response is the \textbf{capabilities document}, which contains a detailed description of the services capabilities. It's sections describe: service identification, service provider, operations metadata (parameter names, \ldots), filter capabilities, and contents (a list of offering descriptions). Please see section 8.2.3 of the SOS specification for details. If you want to inspect the original capabilities document it can be re-requested using

<<capsOriginal,echo=TRUE,eval=FALSE>>= 
sosCapabilitiesDocumentOriginal(sos = mySOS)
@

The actual operation can be started with the following function. It returns an object of class \code{SosCapabilities} which can be accessed later on by the function \code{sosCaps()} from an object of class \code{SOS}.

<<getCap1,echo=TRUE,eval=FALSE>>= 
getCapabilities(sos = mySOS)
@

The parameters of the operation are:

\begin{itemize}
    \item \code{sos}: The SOS connection to request the capabilities document from.
    \item \code{inspect} and \code{verbose}: See section \ref{subsec:inspect}.
\end{itemize}


\subsubsection{Exploring the Capabilities Document}
\label{subsubsec:exploringcapabilities}

The respective \textbf{parts of the capabilities document} are modelled as R classes and can be accessed with these functions:

<<getCap2,echo=TRUE,eval=FALSE>>= 
sosServiceIdentification(mySOS)
sosServiceProvider(mySOS)
sosFilter_Capabilities(mySOS)
sosContents(mySOS)
@

The first four functions extract clearly structured, self-explanatory parts of the document, so no further discussion is made here. The contents part however is described in detail in section \ref{subsubsec:metadataExtraction}, as it can (and should) be used to extract query parameters.

The function \code{sosTime()} returns the time period for which observations are available within the service. To be precise, it accesses the ows:Range element of the parameter eventTime in the description of the GetObservation operation.

<<getCap3,echo=TRUE,eval=TRUE>>= 
sosTime(mySOS)
@

The operations supported by the SOS are listed in the ows:OperationsMetadata element, which is modelled as an R class, \code{OwsOperationsMetadata}, which contains a list of objects of class \code{OwsOperation} which in turn describe the allowed parameter values for calls to the operation. The operations metadata and individual operations can be inspected with the following functions.

<<getCap4,echo=TRUE,eval=FALSE>>= 
sosOperationsMetadata(mySOS)
sosOperation(mySOS, "GetCapabilities")
sosOperation(mySOS, sosGetCapabilitiesName)
@

The allowed response formats (the file format/encoding of the response), the response modes (for example inline or as attachment) and the result models (a qualified XML name of the root element of the response) differ for every operation of the service. The following accessor methods return either (i) a list (named by the operation names) of vectors (with the actual allowed parameter values), or (ii) with the \code{unique} parameter set to \code{TRUE}, a unique list of all allowed values. Please be aware that these are not allowed for all operations, not are all options supported by \pkg{sos4R}.

<<getCap5,echo=TRUE,eval=FALSE>>=
sosResponseFormats(mySOS)
sosResponseMode(mySOS)
sosResultModels(mySOS)
@

Some exemplary outputs of the operations are as follows (unnamed lists are simplified with \code{toString()}). Note the missing values for some operations (where options are not required they might not be available). 

<<getCap6a,echo=TRUE,eval=FALSE>>=
sosResponseMode(mySOS, unique = TRUE)
@
<<getCap6b,echo=FALSE,eval=TRUE>>=
toString(sosResponseMode(mySOS, unique = TRUE))
@

<<getCap7,echo=TRUE,eval=TRUE>>=
sosResultModels(mySOS)[1:3]
@

<<getCap8a,echo=TRUE,eval=FALSE>>=
sosResponseMode(mySOS)[[sosGetObservationByIdName]]
@
<<getCap8b,echo=FALSE,eval=TRUE>>=
toString(sosResponseMode(mySOS)[[sosGetObservationByIdName]])
@

<<getCap9a,echo=TRUE,eval=FALSE>>=
sosResultModels(mySOS)[[sosGetObservationName]][3:4]
@
<<getCap9b,echo=FALSE,eval=TRUE>>=
toString(sosResultModels(mySOS)[[sosGetObservationName]])
@

<<getCap10,echo=TRUE,eval=FALSE>>=
sosResponseFormats(mySOS)[[sosGetObservationByIdName]]
@
<<getCap10,echo=FALSE,eval=TRUE>>=
toString(paste(sosResponseFormats(mySOS)[[sosGetObservationByIdName]]))
@

\subsubsection{Spatial Reference Systems}
\label{subsubsec:getCapSRS}

For future analyses, but also for correct plotting, one must know the coordinate reference system (CRS) or spatial reference system (SRS)\footnote{\url{http://en.wikipedia.org/wiki/Spatial_referencing_system}} or the returned data. You can get this information using the method \code{sosGetCRS()} from variuos objects.

The function utilizes the EPSG code\footnote{\url{http://www.epsg-registry.org/}} in GML attributes like \code{srsName="urn:ogc:def:crs:EPSG:4326"} to initialize an object of class \code{CRS} from the package \pkg{sp}. For \code{SOS} and code{SosObservationOffering} objects these are taken from the bounding box given in the \code{gml:boundedBy} element.

<<sosGetCRS,echo=TRUE,eval=TRUE>>=
sosGetCRS("urn:ogc:def:crs:EPSG:4326")

# returns the CRS of offering(s) based on the CRS 
# used in the element gml:boundedBy:
sosGetCRS(mySOS)[1:2]

sosGetCRS(sosOfferings(mySOS)[[1]])
@

Mre examples for \code{sosGetSRS()} can be found in section \ref{subsubsec:getObsSRS}.

\subsubsection{Plotting SOS and Offerings}
\label{subsubsec:plottingSosAndOfferings}

The content of the capabilities document allows the plotting of a service's offerings. The following example uses the packages \pkg{maps}, \pkg{mapdata} and \pkg{maptools} to create a background map. Plotting functions exist for objects of class \code{SOS} (see Figure \ref{fig:mySOS}) and \code{SosObservationOffering}, so offerings can also be plotted separately.

<<sosPlot,include=FALSE>>= 
# background map:
library(maps); library(mapdata); library(maptools)
data(worldHiresMapEnv)
crs <- sosGetCRS(mySOS)[[1]]
worldHigh <- pruneMap(
		map(database = "worldHires",
			region = c("Germany", "Austria", "Netherlands"),
			plot = FALSE))
worldHigh.lines <- map2SpatialLines(worldHigh, proj4string = crs)

# the plot:
plot(worldHigh.lines, col = "grey50")
plot(mySOS, add = TRUE, lwd = 3)
title(main = paste("Offerings by '", sosTitle(mySOS), "'", sep = ""),
		sub = toString(names(sosOfferings(mySOS))))
@

\begin{figure}[ht]
\label{fig:mySOS}

\begin{center}
<<sosPlotFigure,fig=TRUE,echo=FALSE>>=
<<sosPlot>>
@
\end{center}
\caption{Plot of a SOS object.}
\end{figure}

See the demos (section \ref{subsec:demos}) for more detailed examples of plotting.

\subsection{DescribeSensor}
\label{sub:describesensor}

The DescribeSensor operation is specified in clause 8.3 of the SOS specification and its response is modeled in Sensor Model Language\footnote{\url{http://www.opengeospatial.org/standards/sensorml}} (SensorML) and Transducer Markup Language\footnote{\url{http://www.opengeospatial.org/standards/tml}} (TML) specifications.

\begin{quotation}
The DescribeSensor operation is useful for obtaining detailed information of sensor characteristics encoded in either SensorML or TML. The sensor characteristics can include lists and definitions of observables supported by the sensor. [...]
\end{quotation}

The parameters of the operation are as follows. Please see section \ref{sec:supported} and \ref{subsubsec:exploringcapabilities} of this document for supported values respectively allowed values of request parameters.

<<defaultValue,echo=FALSE,print=FALSE,eval=TRUE>>=
defaultOutputFormatDescribeSensor <- gsub(pattern = "&quot;", replacement = "'", x = sosDefaultDescribeSensorOutputFormat)
@

\begin{itemize}
    \item \code{sos}: The SOS connection to request a sensor description from.
    \item \code{procedure}: The identifier of the sensor, so one of the character strings returned by \code{sosProcedures()}.
    \item \code{outputFormat}: The format in which the sensor description is to be returned. The default is \code{\Sexpr{defaultOutputFormatDescribeSensor}}.
    \item \code{inspect} and \code{verbose}: See section \ref{subsec:inspect}.
 	\item \code{saveOriginal}: Saves a copy of the response document in the current working directory. See section \ref{subsec:getObsById} for an example. Accepts boolean values (\code{TRUE} will automatically create file name with time stamp) or character string to be used as file name.
\end{itemize}

A simple example is as follows.

<<describeSensor1a,echo=FALSE,results=hide,print=FALSE>>= 
.sensorFile <- "mySensor"
if (.goOnline) {
	mySensor <- describeSensor(sos = mySOS, # verbose = TRUE
			procedure = "urn:ogc:object:feature:OSIRIS-HWS:3d3b239f-7696-4864-9d07-15447eae2b93",
			saveOriginal = .sensorFile)
} else {
	# mySensor <- parseFile(mySOS, .getFilePath(.sensorFile), verbose = TRUE)
	#mySensor <- parseFile(mySOS, paste0(.sensorFile, ".xml")) #, verbose = TRUE)
	mySensor <- parseFile(mySOS, .getFilePath(.sensorFile), verbose = .verbose)
}
@
<<describeSensor1b,echo=TRUE,print=TRUE,eval=FALSE>>= 
# manual assignment used because procedure order might change:
mySensor <- describeSensor(sos = mySOS,
		procedure = "urn:ogc:object:feature:OSIRIS-HWS:3d3b239f-7696-4864-9d07-15447eae2b93")

# using procedure referencing:
mySensor <- describeSensor(sos = mySOS,
		procedure = sosProcedures(obj = mySOS)[[1]][[1]])
@
<<describeSensor1c,echo=FALSE,print=FALSE,eval=TRUE>>= 
mySensor
@

All additional information presented in the following depends on compliance of the sensor description with the SensorML Profile for Discovery\footnote{\url{http://portal.opengeospatial.org/files/?artifact_id=37944}}).

The coordinates data frame of a sensor description can be accessed with the common method \code{sosCoordinates()}.

<<describeSensor2,echo=TRUE,print=TRUE,eval=FALSE>>=
sosCoordinates(mySensor)
@

Other possibly useful parts of the sensor description can be accessed as well:

<<describeSensor3,echo=TRUE,print=TRUE>>=
sosId(mySensor)
sosName(mySensor)
sosAbstract(mySensor)
@

This includes the coordinates with unit and reference system information in the attributes of the returned object. The observed bounding box is also available.

<<describeSensorCoords,echo=TRUE,print=TRUE>>=
mySensor.coords <- sosCoordinates(mySensor)
attributes(mySensor.coords)
sosBoundedBy(mySensor)
@

The coordinates also allow the plotting of the sensor positions (see Figure \ref{fig:desribeSensorPlot}). Here it is assumed that the spatial reference system of the SOS is the same for data from the first offering and the sensor positions!

<<describeSensorPlotCodeForText,echo=TRUE,eval=FALSE>>=
library(maps); library(mapdata); library(maptools)
data(worldHiresMapEnv)

# get sensor descriptions
procs <- unique(unlist(sosProcedures(mySOS)))
procs.descr <- lapply(X = procs, FUN = describeSensor, sos = mySOS)

sensors.crs <- unique(sosGetCRS(procs.descr))[[1]]
worldHigh <- pruneMap(map(database = "worldHires",
				region = c("Germany", "Austria", "Netherlands",
						"Italy"),
				plot = FALSE))
worldHigh.lines <- map2SpatialLines(worldHigh, proj4string = sensors.crs)

plot(worldHigh.lines, col = "grey50", ylim = c(44.0, 54.8))
for(x in procs.descr)
	plot(x, add = TRUE, pch = 19)
text(sosCoordinates(procs.descr)[c("x", "y")],
		labels = sosId(procs.descr), pos = 4, cex = 0.8)
title(main = paste("Sensors of", sosTitle(mySOS)))
@

\begin{figure}[ht]
\label{fig:desribeSensorPlot}

\begin{center}
<<describeSensorPlot,fig=TRUE,echo=FALSE,print=FALSE,width=8>>=
library(maps); library(mapdata); library(maptools)
data(worldHiresMapEnv)

# get sensor descriptions
.fileProcs <- "procs.descr.RData"
if (.goOnline) {
	procs <- unique(unlist(sosProcedures(mySOS)))
	procs.descr <- lapply(X = procs, FUN = describeSensor, sos = mySOS)
	save(procs.descr, file = .fileProcs)
} else {
	load(.fileProcs)
}

sensors.crs <- unique(sosGetCRS(procs.descr))[[1]]
worldHigh <- pruneMap(map(database = "worldHires",
				region = c("Germany", "Austria", "Netherlands",
						"Italy"),
				plot = FALSE))
worldHigh.lines <- map2SpatialLines(worldHigh, proj4string = sensors.crs)

plot(worldHigh.lines, col = "grey50", ylim = c(44, 55))
# only plot the procedures where I can find the id (and then probably the
# position can also be found
for(x in procs.descr)
	plot(x, add = TRUE, pch = 19)
text(sosCoordinates(procs.descr)[c("x", "y")],
		labels = sosId(procs.descr), pos = 4, cex = 0.6)
title(main = paste("Sensors of", sosTitle(mySOS)))
@
\end{center}
\caption{Plot of procedure positions and identifiers}
\end{figure}


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{GetObservation}
\label{sub:getobservation}

The GetObservation operation is specified in clause 8.4 of the SOS specification. In this section, all matters around requesting data are explained --- from extracting query parameters from metadata, and sending the request, till finally extracting data values and coordinates from the response.

A few utility functions exist to minize a user's amount of work to create usual requests. They accept normal R types as input and return the respective class from \pkg{sos4R} with useful default settings. These function's names follow the pattern with \code{sosCreate [name of object] ()} and exist for spatial and temporal filters.

\subsubsection{Metadata Extraction for Request Building}
\label{subsubsec:metadataExtraction}

It is recommended to extract the identifiers of procedures et cetera that are to be used for queries from the metadata description provided by the service, the capabilities document (see section \ref{subsec:getcap}. This often ensures forward compatiblity and minimizes typing errors. The offerings are the \enquote{index} of the service and therefore we concentrate on the contents section of the capabilities here.

The class \code{SosContents} simply contains a list of objects of the class \code{SosObservationOffering} which one can get directly from the connection object:

<<metadataExtraction1,echo=TRUE,eval=FALSE>>= 
sosOfferings(mySOS)
sosOfferings(mySOS, name = "Rain")
@

The output when printing this list is quite extensive, so we concentrate on just on element of it in the following examples. Printing and summary methods are available of objects of the class \code{SosObservationOffering}.

<<metadataExtraction1,echo=TRUE,eval=TRUE>>= 
summary(sosOfferings(mySOS)[[1]])
@

The offerings list is named with the offering identifier, so the following statements return the same list.

<<metadataExtraction2,echo=TRUE,eval=FALSE>>= 
sosOfferingIds(mySOS)
names(sosOfferings(mySOS))
sosName(sosOfferings(mySOS))
@

The offering identifier is is used in the example below to extract the offering description of temperature measurements. The offerings list is a standard R list, so all subsetting operations are possible.

\textbf{Note:} The order of the offering list (as all other lists, e.g. procedures or observed properties) is not guaranteed to be the same upon every connection to a service. So indexing by name (though counteracting the mentioned forward compatibility, as names might change) is recommended at at least one point in the analysis so that changes in the contents of a service result in an error.

<<metadataExtraction3,echo=TRUE,eval=TRUE,print=TRUE>>= 
off.temp <- sosOfferings(mySOS)[["ATMOSPHERIC_TEMPERATURE"]]
@

Metadata about the whole \textbf{offering} are identifier, name, and spatial and temporal extends.

<<metadataExtraction4,echo=TRUE,eval=TRUE,print=TRUE>>= 
off.temp.id <- sosId(off.temp)
off.temp.name <- sosName(off.temp)
@

The offerings also contains metadata about the format and model that are supported.

<<metadataExtraction4b,echo=TRUE,eval=TRUE,print=TRUE>>= 
sosResultModels(off.temp)
sosResponseMode(off.temp)
sosResponseFormats(off.temp)
@

The \textbf{spatial extend} is given as a rectangular bounding box with two coordinates. The structure of the bounding box is kept flexible, as it simply returns a named list of lower and upper corner.

<<metadataExtraction5,echo=TRUE,eval=TRUE,print=TRUE>>= 
off.temp.boundedBy <- sosBoundedBy(off.temp)
@

The optional attribute \code{bbox} can be used to obtain a bounding box matrix as used by package \pkg{sp}.

<<metadataExtraction6,echo=TRUE,eval=TRUE,print=TRUE>>= 
off.temp.boundedBy.bbox <- sosBoundedBy(off.temp, bbox = TRUE)
@

The \textbf{temporal extend} is modeled as an object of the respective class of the element in the offering description, which normally is a gml:TimePeriod, but does not have to be. The last two statements in the following snipped show how one can access the actual data and what their class is.

<<metadataExtraction7,echo=TRUE,eval=TRUE,print=TRUE>>= 
off.temp.time <- sosTime(off.temp)
str(off.temp.time)

off.temp.time@beginPosition@time
off.temp.time@endPosition@time
class(off.temp.time@endPosition@time)
@

The structure of these elements is very flexible (with some of optional elements) and not self-explanatory. Therefore the parameter \code{convert} can be used to try to create R objects and return these instead. Please be aware that this might not work for temporal elements returned by all service.

<<metadataExtraction8,echo=TRUE,eval=TRUE,print=TRUE>>= 
off.temp.time.converted <- sosTime(off.temp, convert = TRUE)
str(off.temp.time.converted)
@

Furthermore the offering comprises \textbf{lists of procedures}, \textbf{observed properties}, and \textbf{features of interest}. In our example the feature and procedure identifiers are the same --- this does not have to be the case.

\textbf{Important Note:} The order of these lists is not guaranteed to be the same upon every connection to a service.

<<metadataExtraction9,echo=TRUE,eval=TRUE,print=TRUE>>= 
sosProcedures(off.temp)
sosObservedProperties(off.temp)
sosFeaturesOfInterest(off.temp)
@

All of the above can not only be requested for single offerings but also for complete SOS connections or for lists of offerings. The following examples only print out a part of the returned lists.

<<metadataExtraction10,echo=TRUE,eval=TRUE,print=FALSE>>= 
sosProcedures(mySOS)[1:2]
sosObservedProperties(mySOS)[1:2]
sosFeaturesOfInterest(mySOS)[1:2]
@

Also (parts of) a list of offerings are possible with these functions:

<<metadataExtraction10,echo=TRUE,eval=TRUE,print=FALSE>>= 
sosProcedures(sosOfferings(mySOS)[4:5])
sosObservedProperties(sosOfferings(mySOS)[4:5])
sosFeaturesOfInterest(sosOfferings(mySOS)[3:4])
@

Please carefully inspect the structure in each case, as these functions will return named lists of lists and not combine procedures from different offerings. Consequently, some procedures could appear several times, but the association to the offering is still intact which is preferred at this stage.

\subsubsection{Basic Request}

<<getObservation,eval=FALSE,echo=TRUE>>= 
getObservation(sos = mySOS, offeringy = myOffering, ...)
@

The mandatory attributes are \code{sos}, \code{offering}, \code{observedProperty} and \code{responseFormat}. The other parameters are set to \code{NA} and not used when building the request.

Please see section 8.4.2 of the SOS specification for details, and section \ref{sec:supported} and \ref{subsubsec:exploringcapabilities} of this document for supported values respectively allowed values of request parameters. Note that different implementations might respond differently to missing parameters.

<<defaultValue,echo=FALSE,print=FALSE,eval=TRUE>>=
defaultResponseFormatGetObs <- gsub(pattern = "&quot;", replacement = "'", x = sosDefaultGetObsResponseFormat)
@

\begin{itemize}
  \item \code{sos}: The service connection to be used, an object of class \code{SOS}.
  \item \code{offering}: The offering to be used, either the identifier as a character string or an object of class \code{SosObservationOffering}.
  \item \code{observedProperty}: The observed property of the desired observations. The default is all observed property of the offering, \code{sosObservedProperties(obj = offering)}.
  \item \code{responseFormat}: The format of the response document. The default is \Sexpr{defaultResponseFormatGetObs}.
  \item \code{srsName}: The name of the spatial reference system that should be used for the geometries in the response.
  \item \code{eventTime}: A list of objects of class \code{SosEventTime} which specify the time period(s) for which observations are requested. See section \ref{subsubsec:temporalfiltering} for more information.
  \item \code{procedure}: A list of procedure identifiers for which observations are requested. See section \ref{subsubsec:spatialfiltering} for more information.
  \item \code{featureOfInterest}: An object of class \code{SosFeatureOfInterest} which specifies the feature for which observations are requested. See sections \ref{subsubsec:spatialfiltering} and \ref{subsubsec:featurefiltering} for more information.
  \item \code{result}: An object of class \code{OgcComparisonOps} for result filtering with filter expressions from Filter Encoding.  See section \ref{subsubsec:valuefiltering} for more information.
  \item \code{resultModel}: The qualified XML name of the root element of the response, e.g. \code{om:Measurement}. The available models of a service can be found in the service metadata using \code{sosResultModel()}.
  \item \code{responseMode}: The response mode defines the form of the response, e.g. inline, out-of-band, or attached. The available models of a service can be found in the service metadata using \code{sosResponseMode()}.
  \item \code{BBOX}: A bounding box to be used only in HTTP GET connections (parameter is discarded for POST connections). The format must one character string with \code{minlon,minlat,maxlon,maxlat,srsURI?}, the spatial reference system is optional.
  \item \code{latest}: A boolean parameter to request the latest observation only (see example below) --- this is not standard conform but only supported by \fivetwon SOS.
  \item \code{saveOriginal}: Saves a copy of the response document in the current working directory. See section \ref{subsec:getObsById} for an example. Accepts boolean values (\code{TRUE} will automatically create file name with time stamp) or character string to be used as file name.
\end{itemize}

The returned data of all GetObservation operations is an XML document of type om:Observation, om:Measurement, or om:ObservationCollection which holds a list of the former two. All three of these have corresponding S4 classes, namely \code{OmObservation}, \code{OmMeasurement}, or \code{OmObservationCollection}.

The most straightforward (and most simple to use) methods to query certain observations are to request one (or several) specific \textbf{observed property} (phenomenon) or \textbf{procedure} (sensor). Note that the procedures and observed properties have to match the given offering.

<<getObsPropPhen,eval=FALSE,echo=TRUE>>= 
obs.temp.procedure.1 <- getObservation(sos = mySOS,
	offering = off.temp,
	procedure = sosProcedures(off.temp)[[2]])

obs.temp.offering.34 <- getObservation(sos = mySOS,
	offering = off.temp,
	procedure = sosProcedures(off.temp)[3:4],
	observedProperty =
		sosObservedProperties(mySOS)[3:4])
@

These request would potentially retrieve a lot of data, since there is no temporal (or thematical/spatial) limitation. The following example requests data for about one day of temperature data and stores it in the object \code{obs.temp}. This feature is described extensively in section \ref{subsubsec:temporalfiltering}.

<<loadOrDownload_ObsTemp,eval=TRUE,echo=FALSE,results=hide>>= 
.obsFile <- "obs.temp"
if (.goOnline) {
	obs.temp <- getObservation(sos = mySOS,
		offering = off.temp,
		eventTime = sosCreateTime(sos = mySOS, time = "2009-08-20::2009-08-21"),
		saveOriginal = .obsFile)
} else {
	obs.temp <- parseFile(mySOS, .getFilePath(.obsFile))
}
@
<<getObs0a,eval=FALSE>>=
obs.temp <- getObservation(sos = mySOS,
		offering = off.temp,
		eventTime = sosCreateTime(sos = mySOS, time = "2009-08-20::2009-08-21"),
		saveOriginal = .obsFile)
@
% Fake the response:
<<getObs0b,eval=TRUE,echo=FALSE>>=
.results <- lapply(obs.temp, sosResult)
.resultLength <- sapply(.results, nrow)
cat("[sos4R] Received response (size: 27056 bytes), starting parsing ...\n",
	"[sos4R] Finished getObservation to", sosUrl(mySOS), "\n",
	"	--> received", length(obs.temp), "observation(s) having",
	sum(.resultLength), "result values [", toString(.resultLength), "].")
@

The logging output above starting with \code{[sos4R]} informs the user when the download of data is complete and when the parsing has finished. It even contains some information about the data, if possible. In following requests, this ouput is \textbf{not} included for brevity.

The response \code{obs.temp} of this request is the base for the next sections.

<<getObs1,eval=TRUE,echo=TRUE>>=
class(obs.temp)
str(obs.temp, max.level = 2)
@

\subsubsection{Response Subsetting}

Subsetting of elements in an \code{OmObservationCollection} can be done just like in a normal list (in fact, it just wraps at list of observations at this point), i.e. with the operators \code{[} and \code{[[}. Summary functions are available for single observations or an observation collection.

<<getObs1,eval=TRUE,echo=TRUE>>=
length(obs.temp)
obs.temp[[1]]
summary(obs.temp)
summary(obs.temp[[1]])
@

The collection can also be subset in parts:

<<getObs1,eval=FALSE,echo=TRUE>>=
obs.temp[2:3]
@

\textbf{Observation collection indexing} is possible with identifiers of procedure(s), observed property(ies), and feature(s) of interest.

<<getObs1,eval=TRUE,echo=TRUE>>=
index.foiId <- sosFeatureIds(obs.temp)[[1]]
index.foiId
obs.temp[index.foiId]

index.obsProp <- sosObservedProperties(off.temp)
obs.temp[index.obsProp]

index.proc <- sosProcedures(obs.temp)[1:4]
index.proc.alternative1 <- sosProcedures(off.temp)[1:4]
index.proc.alternative2 <- sosProcedures(mySOS)
obs.temp[index.proc]
@

\subsubsection{Result Extraction}

\textbf{Data Values} can be extracted from observations, measurements and observation collections with the function \code{sosResult()}.  The function returns an object of class \code{data.frame}. In the case of collections, it automatically binds the data frames (you can turn this off by adding \code{bind = FALSE} as a parameter).

<<getObs2,eval=TRUE,echo=TRUE,print=FALSE>>= 
obs.temp.result.2 <- sosResult(obs.temp[[2]])
obs.temp.result <- sosResult(obs.temp[1:2])
@

Additional metadata, like units of measurement or definitions, is accessible via \code{attributes()} for every column of the data frame. 

<<getObs3,eval=TRUE,echo=TRUE,print=TRUE>>= 
temperature.attrs <- attributes(
	obs.temp.result[["urn:ogc:def:property:OGC::Temperature"]])
@

\textbf{Spatial Information} can be stored in an observation in several ways: (i) as a usual data attribute which is directly contained in the result \code{data.frame}, (ii) within a feature collection in the observation. In the latter case the utility functions \code{sosCoordinates()} and \code{sosFeatureIds()} can be used to extract the coordinates respectively the identifiers from \code{OmObservationCollection} or \code{OmObservation} classes. A variety of feature types gml:Point or sa:SamplingPoint are supported by \code{sosCoordinates()}.

<<getObsSpatial1,eval=TRUE,echo=TRUE>>= 
obs.temp.foiIDs <- sosFeatureIds(obs.temp)
obs.temp.coords <- sosCoordinates(obs.temp)
obs.temp.coords.1 <- sosCoordinates(obs.temp[[1]])
@

An observation collection also contains a bounding box of the contained observations, which can be extracted with the function \code{sosBoundedBy()}. The optional attribute \code{bbox} can be used to obtain a bounding box matrix as used by package \pkg{sp}.

<<getObsSpatial1,eval=TRUE,echo=TRUE>>= 
sosBoundedBy(obs.temp)
sosBoundedBy(obs.temp, bbox = TRUE)
@

The combination of data values and coordinates strongly depends on the use case and existing spatial information. In the case of coordinates encoded in the features, a matching of the two data frames can easily be accomplished manually with the function \code{merge()}.

<<getObsSpatial2,eval=TRUE,echo=TRUE,print=FALSE>>= 
result.names <- names(obs.temp.result)
coords.names <- names(obs.temp.coords)
print(toString(result.names))
print(toString(coords.names))

obs.temp.data <- merge(
	x = obs.temp.result,
	y = obs.temp.coords,
	by.x = result.names[[2]],
	by.y = coords.names[[4]])
@

The default column name for the feature identifiers is \code{\Sexpr{sosDefaultColumnNameFeatureIdentifier}}. If the name of the feature identifier attribute in the data table matches (which is the case for \fivetwon SOS), \code{merge} does not need additional information. In that case, the merging reduces to the following code:

<<getObsSpatial3,eval=TRUE,echo=TRUE>>=
obs.temp.data <- merge(x = obs.temp.result,
	y = obs.temp.coords)
str(obs.temp.data, max.level = 2)
@

And in that case, you can even save that step by specifying the attribute \code{coordinates} of the function \code{sosResult} which includes the merge of data values and coordinates as shown above.

<<getObsSpatial4,eval=FALSE,echo=TRUE,print=FALSE>>= 
sosResult(obs.temp, coordinates = TRUE)
@

\subsubsection{Temporal Filtering}
\label{subsubsec:temporalfiltering}

The possibly most typical temporal filter is a period of time for which measurements are of interest.

<<temporalFiltering1a,eval=TRUE,echo=TRUE>>= 
# temporal interval creation based on POSIXt classes:
lastWeek.period <- sosCreateTimePeriod(sos = mySOS,
	begin = (Sys.time() - 3600 * 24 * 7),
	end = Sys.time())

oneWeek.period <- sosCreateTimePeriod(sos = mySOS,
		begin = as.POSIXct("2010/01/01"),
		end = as.POSIXct("2010/01/07"))
oneWeek.eventTime <- sosCreateEventTimeList(oneWeek.period)
@

Please note that the create function \code{sosCreateEventTimeList()} wraps the created objects in a list as required by the method \code{getObservation()}.

The most comfortable creation function for event times is \code{sosCreateTime()}. It supports time intervals with starttime and endtime as character strings seperated by \code{::} or \code{/} as defined by ISO 8601\footnote{\url{http://en.wikipedia.org/wiki/ISO_8601}, section Time intervals}. The respective time stamps have to be parsable by \code{as.POSIXct()}. If either one of the time stamps is missing, a \code{GmlTimePosition} wrapped in the appropriate relative temporal operator, e.g. \enquote{before}.

<<temporalFiltering1b,eval=TRUE,echo=TRUE>>= 
sosCreateTime(sos = mySOS, time = "2007-07-07 07:00::2008-08-08 08:00")
sosCreateTime(sos = mySOS, time = "2007-07-07 07:00/2010-10-10 10:00")

sosCreateTime(sos = mySOS, time = "::2007-08-05")
sosCreateTime(sos = mySOS, time = "2007-08-05/")
@

\textbf{Example:} What was the minimum, average and maximum temperature during one week?
	
<<loadOrDownload_TemporalFiltering2,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "obs.oneWeek"
if(.goOnline) {
	obs.oneWeek <- getObservation(sos = mySOS,
		offering = off.temp,
		procedure = sosProcedures(off.temp),
		eventTime = oneWeek.eventTime,
		saveOriginal = .obsFile)
} else {
	obs.oneWeek <- parseFile(mySOS, .getFilePath(.obsFile))
}
@
<<temporalFiltering2a,eval=FALSE,echo=TRUE>>= 
obs.oneWeek <- getObservation(sos = mySOS,
	offering = off.temp,
	# actually not required, as default is 'all procedures':
	procedure = sosProcedures(off.temp),
	eventTime = oneWeek.eventTime)
@
<<temporalFiltering2b,eval=TRUE,echo=TRUE>>=
obs.oneWeek.result <- sosResult(obs.oneWeek)
summary(obs.oneWeek.result[,"urn:ogc:def:property:OGC::Temperature"])
@

The default temporal operator is \enquote{during}, but others are supported as well (see section \ref{sec:supported}). The next example shows how to create a temporal filter for all observations taken \textbf{after} a certain point in time. Here the creation function creates just one object of class \code{SosEventTime} which must be added to a list manually before passing it to \code{getObservation()}.

<<temporalFiltering3,eval=TRUE,echo=TRUE>>=
lastDay.instant <- sosCreateTimeInstant(
	time = as.POSIXct(Sys.time() - 3600 * 24), sos = mySOS)
lastDay.eventTime <- sosCreateEventTime(time = lastDay.instant,
	operator = SosSupportedTemporalOperators()[["TM_After"]])
print(lastDay.eventTime)
@

\subsubsection{Spatial Filtering}
\label{subsubsec:spatialfiltering}

The possibly most typical spatial filter is a bounding box\footnote{\url{http://en.wikipedia.org/wiki/Bounding_box}} within which measurements of interest must have been made. Here the creation function returns an object of class \code{OgcBBOX}, which can be wrapped in an object of class \code{SosFeatureOfInterest}, which is passed into the get-observation call.

<<spatialFiltering1,eval=TRUE,echo=TRUE>>=
sept09.period <- sosCreateTimePeriod(sos = mySOS,
	begin = as.POSIXct("2009-09-01 00:00"),
	end = as.POSIXct("2009-09-30 00:00"))
sept09.eventTimeList <- sosCreateEventTimeList(
	sept09.period)
@

<<loadOrDownload_SpatialFiltering,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "obs.sept09"
.obsFile2 <- "obs.sept09.bbox"
if(.goOnline) {
	obs.sept09 <- getObservation(sos = mySOS,
		offering = off.temp,
		eventTime = sept09.eventTimeList,
		saveOriginal = .obsFile)

	request.bbox <- sosCreateBBOX(lowLat = 50.0, lowLon = 5.0,
			uppLat = 55.0, uppLon = 10.0,
			srsName = "urn:ogc:def:crs:EPSG:4326")
	request.bbox.foi <- sosCreateFeatureOfInterest(
			spatialOps = request.bbox)
	
	obs.sept09.bbox <- getObservation(sos = mySOS,
			offering = off.temp,
			featureOfInterest = request.bbox.foi,
			eventTime = sept09.eventTimeList,
			saveOriginal = .obsFile2)
} else {
	obs.sept09 <- parseFile(mySOS, .getFilePath(.obsFile))
	obs.sept09.bbox <- parseFile(mySOS, .getFilePath(.obsFile2))
}
@
<<spatialFiltering2,eval=FALSE,echo=TRUE>>= 
obs.sept09 <- getObservation(sos = mySOS,
	offering = off.temp,
	eventTime = sept09.eventTimeList)
@

<<spatialFiltering3,eval=FALSE,echo=TRUE>>=
request.bbox <- sosCreateBBOX(lowLat = 50.0, lowLon = 5.0,
	uppLat = 55.0, uppLon = 10.0,
	srsName = "urn:ogc:def:crs:EPSG:4326")
request.bbox.foi <- sosCreateFeatureOfInterest(
	spatialOps = request.bbox)
obs.sept09.bbox <- getObservation(sos = mySOS,
	offering = off.temp,
	featureOfInterest = request.bbox.foi,
	eventTime = sept09.eventTimeList)
@

Unfiltered versus spatially filtered coordinates of the responses:

<<spatialFiltering1,eval=TRUE,echo=TRUE>>=
print(sosCoordinates(obs.sept09)[,1:2])
print(sosCoordinates(obs.sept09.bbox)[,1:2])
@

More advanced spatial filtering, for example based on arbitrary shapes et cetera, is currently not implemented. This could be implemented by implementing subclasses for \code{GmlGeometry} (including encoders) which must be wrapped in \code{OgcBinarySpatialOp} which extends \code{OgcSpatialOps} and can therefore be added to an object of class \code{SosFeatureOfInterest} as the spatial parameter.

\subsubsection{Feature Filtering}
\label{subsubsec:featurefiltering}

The feature can not only be used for spatial filtering, but also to query specific FOIs. The following example extracts the identifiers from an offering and then creates an object of class \code{SosFeatureOfInterest}, which is passed into the get-observation call. Here the encoding function is called to show how the content of the result element will look like.

<<featureFiltering1,eval=TRUE,echo=TRUE>>= 
off.temp.fois <- sosFeaturesOfInterest(off.temp)
request.fois <- sosCreateFeatureOfInterest(
	objectIDs = list(off.temp.fois[[1]]))
encodeXML(obj = request.fois, sos = mySOS)
@

An exemplary GetObservation operation is as follows.

<<loadOrDownload_FeatureFiltering,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "obs.oneWeek.fois"
if(.goOnline) {
	obs.oneWeek.fois <- getObservation(sos = mySOS,
		offering = off.temp,
		featureOfInterest = request.fois,
		eventTime = oneWeek.eventTime,
		saveOriginal = .obsFile)
} else {
	obs.oneWeek.fois <- parseFile(mySOS, .getFilePath(.obsFile))
}
@
<<featureFiltering1a,eval=FALSE,echo=TRUE,print=FALSE>>= 
obs.oneWeek.fois <- getObservation(sos = mySOS,
	offering = off.temp,
	featureOfInterest = request.fois,
	eventTime = oneWeek.eventTime)
@

<<featureFiltering1b,eval=TRUE,echo=TRUE>>=
print(sosFeaturesOfInterest(obs.oneWeek.fois))
@


\subsubsection{Value Filtering}
\label{subsubsec:valuefiltering}

Value Filtering is realized via the slot \code{result} in a GetObservation request. The filtering in the request is based on comparison operators and operands specified by OGC Filter Encoding (Vretanos, 2005).

The classes and methods of this specification are not yet implemented, but manual definition of the XML elements is possible with the methods of the package \pkg{XML}.

The following code example uses a literal comparison of a property. The elements names are taken from constants within \pkg{sos4R} (with the naming scheme \enquote{<namespace><ElementName>Name}), but can equally as well be put in directly.

<<valueFiltering1,eval=TRUE,echo=TRUE,print=FALSE>>= 
# result filtering
filter.value <- -2.3
filter.propertyname <- xmlNode(name = ogcPropertyNameName,
	namespace = ogcNamespacePrefix)
xmlValue(filter.propertyname) <-
		"urn:ogc:def:property:OGC::Temperature"
filter.literal <- xmlNode(name = ogcLiteralName,
	namespace = ogcNamespacePrefix)
xmlValue(filter.literal) <- as.character(filter.value)
filter.comparisonop <- xmlNode(
	name = ogcComparisonOpGreaterThanName,
	namespace = ogcNamespacePrefix,
	.children = list(filter.propertyname,
	filter.literal))
filter.result <- xmlNode(name = sosResultName,
	namespace = sosNamespacePrefix,
	.children = list(filter.comparisonop))
@

Please consult to the extensive documentation of the \pkg{XML} package for details. The commands above result in the following output which is inserted into the request without further processing.

<<valueFiltering2,eval=TRUE,echo=TRUE,print=TRUE>>= 
filter.result
@

Any object of class \code{OgcComparisonOpsOrXMLOrNULL}, which includes the class of the object returned by \code{xmlNode()}, i.e. \code{XMLNode}. These object can be used in the GetObservation request as the \code{result} parameter.

First, we request the unfiltered values for comparison, then again with the filter applied. The length of the returned results is compared in the end.

% has been requested before
<<valueFiltering3a,eval=FALSE,echo=TRUE>>= 
obs.oneWeek <- getObservation(sos = mySOS,
	eventTime = oneWeek.eventTime,
	offering = sosOfferings(mySOS)[["ATMOSPHERIC_TEMPERATURE"]])
@

<<loadOrDownload_ValueFiltering,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "obs.oneWeek.filter"
if(.goOnline) {
	obs.oneWeek.filter <- getObservation(sos = mySOS,
		eventTime = oneWeek.eventTime,
		offering = sosOfferings(mySOS)[["ATMOSPHERIC_TEMPERATURE"]],
		result = filter.result,
		saveOriginal = .obsFile)
} else {
	obs.oneWeek.filter <- parseFile(mySOS, .getFilePath(.obsFile))
}
@
<<valueFiltering3b,eval=FALSE,echo=TRUE>>= 
# request  values for the week with a value higher than 0 degrees:
obs.oneWeek.filter <- getObservation(sos = mySOS,
	eventTime = oneWeek.eventTime,
	offering = sosOfferings(mySOS)[["ATMOSPHERIC_TEMPERATURE"]],
	result = filter.result)
<<valueFiltering3c,eval=TRUE,echo=TRUE>>= 
print(paste("Filtered:", dim(sosResult(obs.oneWeek.filter))[[1]], 
	"-vs.- Unfiltered:", dim(sosResult(obs.oneWeek))[[1]]))
@

\subsubsection{Result Exporting}

A tighter integration with data structures of packages \pkg{sp} or \pkg{spacetime} (both available on CRAN) is planned for the future. Please consult the developers for the current status.

As an example the following code creates a \code{SpatialPointsDataFrame} (can only contain one data value per position!) based on the features of a result.

<<resultExporting1,eval=TRUE,echo=TRUE>>=
library("sp")
@

% ran before:
<<resultExporting2,eval=FALSE,echo=TRUE>>=
obs.oneWeek <- getObservation(sos = mySOS,
	offering = off.temp,
	procedure = sosProcedures(off.temp),
	eventTime = oneWeek.eventTime)
@

<<resultExporting3,eval=TRUE,echo=TRUE>>=
# Create SpatialPointsDataFrame from result features
coords <- sosCoordinates(obs.oneWeek[[1]])
crs <- sosGetCRS(obs.oneWeek[[1]])
spdf <- SpatialPointsDataFrame(coords = coords[,1:2],
	data = data.frame(coords[,4]), proj4string = crs)
str(spdf)
@

\subsubsection{Spatial Reference Systems}
\label{subsubsec:getObsSRS}

For following analyses and plotting, the \textbf{spatial reference system} can be extracted as follows (see section \ref{subsubsec:getCapSRS} for a general description).

<<getObsCRS,echo=TRUE,eval=TRUE>>=
sosGetCRS(obs.temp)
sosGetCRS(obs.oneWeek)
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{GetObservationById}
\label{subsec:getObsById}

The operation GetObservationById is defined in clause 10.1 of the SOS specification and not part of the core profile. But it is implemented as it is quite simple. The response is the same as described in the previous section. Optional parameters, and their defaults and supported values (see sections \ref{sec:supported} and \ref{subsubsec:exploringcapabilities}), are normally the same as in GetObservation requests.

In this case the returned observation collection contains an om:Measurement element, which contains just one measured value and is parsed to an object of class \code{OmMeasurement}.

The result extraction works the same as with objects of class \code{OmObservation}.

<<loadOrDownload_GetObsById,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "obsId"
if(.goOnline) {
	obsId <- getObservationById(sos = mySOS,
		observationId = "o_3508493",
		saveOriginal = .obsFile)
} else {
	obsId <- parseFile(mySOS, .getFilePath(.obsFile))
}
@
<<getObsById1a,eval=FALSE,echo=TRUE>>= 
obsId <- getObservationById(sos = mySOS,
	observationId = "o_3508493")
@
<<getObsById1b,eval=TRUE,echo=TRUE>>= 
sosResult(obsId, coordinates = TRUE)
@

Just as for \code{getObservation()} you can \textbf{save the orginal} response document with an automatically generated name or a selected one. It is saved into the current working directory and the name starts with the observation identifier. You can also read it back using the function \code{sosParse()}.

<<getObsById3,eval=FALSE,echo=TRUE>>= 
# generated file name, find file in working directory:
obsId <- getObservationById(sos = mySOS,
	observationId = "o_3508493",
	saveOriginal = TRUE)
.files <- list.files(getwd())
.observationFiles <- c()
for(.f in .files) { # %in% not working with Sweave
	if(length(grep("^o_", .f, value=TRUE)) > 0)
		.observationFiles <- c(.observationFiles, .f)
}
obsId <- parseFile(sos = mySOS,
	file = .observationFiles[[1]])

<<getObsById3,eval=FALSE,echo=TRUE>>=
# manually selected file name:
obsId <- getObservationById(sos = mySOS,
	#verbose = TRUE,
	observationId = "o_3508493",
	saveOriginal = "myObservation")
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Changing Handling Functions}
\label{sec:exchangeability}

The flexibility of the specifications that model the markup requests and responses, especially the observation encoding, is too high to handle all possible cases within \pkg{sos4R}. Thus an equally flexible mechanism for users to adopt the steps of encoding and decoding documents to their needs is needed.

The process of data download comprises (i) building request, (ii) encoding requests, (iii) sending and receiving data, (iv) decoding responses, and (v) applying the correct R data type to the respective data values. This can be seen as a fixed, ordered workflow a user has to follow where each step build upon the input of the previous. To ensure flexibility within these steps of the workflow but also to maximize reusability of existing functionality, a mechanism to exchange the functions that are used in these steps is provided. 

Step (i), the building of requests, i.e. the assembly of the request parameters into an R object, is documented in section \ref{sub:getobservation}. Step (iii), the sending of the sending and receiving of documents to respectively from a service, does not need to be influenced directly but the user (apart from the connection method).

In the remainder of this section it is explained how this applies to the steps (ii), (iv) and (v) of the fixed workflow. 


\subsection{Include and Exclude Functions}
\label{subsec:inexclusion}

The functions used in the exchangeable steps are organized in lists. To base your own list of functions on the existing ones, thereby not having to start from scratch, you can combine the default list of functions with your own. Use the following functions:

To add your own function, simply add it as a named argument. You can add as many as you like in the \code{...} parameter. If a function with that identifier already exists in the default list it will be replaced by your function.
For further adjustments you can explicitly include and exclude functions by identifier. Please be aware that inclusion is applied first, then exclusion. It is also important that you also have to include that functions you just added manually!

Examples of function list generation with parsing functions:

<<inclusionExclusion,echo=TRUE,eval=TRUE>>=
parsers <- SosParsingFunctions(
	"ExceptionReport" = function() {
		return("Got Exception!")
	},
	include = c("GetObservation", "ExceptionReport"))
print(names(parsers))

parsers <- SosParsingFunctions(
		"ExceptionReport" = function() {
			return("Got Exception!")
		},
		include = c("GetCapabilities"))
print(names(parsers))
@

The following snipped shows how to remove a large part of parsers using \code{exclude} and then plots the names of the remaining ones.

parsers <- SosParsingFunctions(
		exclude = names(SosParsingFunctions())[5:29])
print(names(parsers))
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Encoders}
\label{subsec:encoders}

The current list of a connection's encoders can be accessed with

<<encoders1,echo=TRUE,eval=FALSE>>=
sosEncoders(mySOS)
@

A complete list of the existing encoders names:

<<encoders2,echo=TRUE,eval=TRUE>>=
names(sosEncoders(mySOS))
@

Here the idea of organizing the encoding functions becomes clear: One base encoding function is given, which is a generic method that must exist for alle elements that need to be encoded.

<<encoders3,echo=TRUE,eval=FALSE>>=
myPostEncoding <- function(object, sos, verbose) {
	return(str(object))
}
# Connection will not be establihsed because of mising objects
mySOS2 = SOS(sosUrl(mySOS),
	encoders = SosEncodingFunctions("POST" = myPostEncoding))
@

Encoding functions can be overriden for many specific objects. The signature of the encoding function consists of the object, \code{obj}, a SOS object, \code{sos}, and the optional \code{verbose} parameter.

<<encoders4,echo=TRUE,eval=TRUE>>=
showMethods("encodeXML")
showMethods("encodeKVP")
@

A useful example can be overriding the encoding method for time classes (\code{POSIXt}) as presented below--see the demo \textbf{southesk} (section 
\ref{subsec:demos}) for the application of this code.

<<encoders5,echo=TRUE,eval=FALSE>>=
setMethod(f = "encodeXML",
  signature = signature(obj = "POSIXt", sos = "SOS"),
    def = function(obj, sos, verbose) {
      if(verbose) cat("Using my own time encoding... ")

      # time zone hack to fix that the time format option
      # %z does not work on windows machines:
      .time <- obj + 11 * 60 * 60 # add 11 hours
      .formatted <- strftime(x = .time,
        format = sosTimeFormat(sos))
      .formatted <- paste(.formatted, 
        "+11:00", sep = "")	# append 11:00

      if(verbose) cat("Formatted ", toString(obj),
        " to ", .formatted, "\n")
      return(.formatted)
    }
)
@

All later calls for encoding any classes with time will then reference this newly defined method. Be aware that this changes the encoding \textbf{globally}, in contrast to converters and parsers which can be changed for every object of class \code{SOS}.


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Parsers/Decoders}
\label{subsec:parsers}

The terms parsing and decoding are used as synonyms for the process of processing an XML document to create an R object. XML documents are made out of hierarchical elements. That is why the parsing functions are organized in a listed, whose names are the elements' names that can be parsed.

The current list of a connection's parsers can be accessed with the following function.

<<parsers1,echo=TRUE,eval=FALSE>>=
sosParsers(mySOS)
@

A complete list of the elements with existing encoders is shown below. These are not only names of XML elements, but also MIME types\footnote{\url{http://en.wikipedia.org/wiki/Internet_media_type}}. Here the idea of organizing the encoding functions becomes clear: For every XML element or document type that must be parsed there is a function given in the list.

<<parsers2,echo=TRUE,eval=TRUE>>=
names(sosParsers(mySOS))
@

Parser selection can also be based on the \textbf{mimeType} of the returned document. Please be aware that this also can be a problem if you want to exchange a parse by \textbf{operation name}, which is done \textbf{after} switching the function based on the mime type. In other words, the exchange by operation name only works if the response type is as expected.

If you want to \textbf{replace only selected parsers} use the include parameter as described above. You can also base your own parsing functions on a variety of existing parsing functions. For example you can replace the base function for om:ObservationCollectionm, named \code{ObservationCollection}, but still use the parsing function for om:Observation within your own function if you include it in the parser list. The existing parsing functions are all named in the pattern \code{parse<ElementName>()}. Please be aware that some parsers contain require a parameter of class \code{SOS} upon which they might rely for example for formatting information.

<<loadOrDownload_Parsers3,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "err.response.RData"
.sos2 <- "mySOS2.RData"
if(.goOnline) {
	myER <- function(xml) {
		return("EXCEPTION!!!11")
	}
	myParsers <- SosParsingFunctions("ExceptionReport" = myER)
	mySOS2 <- SOS(sosUrl(mySOS), parsers = myParsers)
	save(mySOS2, file = .sos2)
	
	err.response <- getObservation(mySOS2,
		offering = sosOfferings(mySOS2)[[1]],
		observedProperty = list("Bazinga"))
	save(err.response, file = .obsFile)
} else {
	load(.sos2)
	load(.obsFile)
}
@
<<parsers3,echo=TRUE,eval=FALSE>>=
# Create own parsing function:
myER <- function(xml) {
	return("EXCEPTION!!!11")
}
myParsers <- SosParsingFunctions("ExceptionReport" = myER)
mySOS2 <- SOS(sosUrl(mySOS), parsers = myParsers)
# Triggers exception:
err.response <- getObservation(mySOS2, verbose = TRUE,
	offering = sosOfferings(mySOS2)[[1]],
	observedProperty = list("Bazinga!"))
@
<<parsers3,echo=TRUE,eval=TRUE>>=
print(err.response)
@

% TODO add exception output here

To disable all parsing, you can use the function \code{SosDisabledParsers()}. This effectively just \enquote{passes through} all received data because the list returned by the function only contains the top-most parsing functions for SOS operations and exception reports.

<<parsers4a,echo=TRUE,eval=FALSE>>=
SosDisabledParsers()
@

<<parsers4b,echo=TRUE,eval=TRUE>>=
names(SosDisabledParsers())
@

This is also the recommended way to start if you want to set-up your own parsers (given you have responses in XML) and an alternative to debugging if you want to inspect responses directly.

The next example shows how the response (in this case the request is intentionally incorrent and triggers an exception) is passed through as an object of class \code{XMLInternalDocument}:

<<loadOrDownload_Parsers5,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "response.noparsing.RData"
.sos2.dis <- "mySOS2.disabled.RData"
if(.goOnline) {
	mySOS2.disabled <- SOS(sosUrl(mySOS), # verboseOutput = TRUE,
			parsers = SosDisabledParsers())
	response.noparsing <- getObservation(mySOS2.disabled,
			offering = sosOfferings(mySOS2.disabled)[[1]],
			observedProperty = list("Bazinga!"))
	save(mySOS2.disabled, file = .sos2.dis)
#	save(response.noparsing, file = .obsFile)
} else {
	load(.sos2.dis)
#	load(.obsFile)
}
@
<<parsers5a,echo=TRUE,eval=FALSE>>=
mySOS2.disabled <- SOS(sosUrl(mySOS),
		parsers = SosDisabledParsers())
response.noparsing <- getObservation(mySOS2.disabled,
	offering = sosOfferings(mySOS2.disabled)[[1]],
	observedProperty = list("Bazinga"))
@
% cannot save XML document
<<parsers5b,echo=FALSE,eval=FALSE>>=
class(response.noparsing)
print(xmlName(xmlRoot(response.noparsing)))
# (Using XML functions here for accesing the root of a
# document and the name of an element.)
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Data Converters}
\label{subsec:converters}

A list of named functions to be used by the parsing methods to convert data values to the correct R type, which are mostly based on the unit of measurement\footnote{\url{http://en.wikipedia.org/wiki/Units_of_measurement}} code.

The conversion functions always take two parameters: \code{x} is the object to be converted, \code{sos} is the service where the request was received from.

The available functions are basically wrappers for coercion functions, for example \code{as.double()}. The only method exploiting the second argument is the one for conversion of time stamps which uses the time format saved with the object of class \code{SOS} in a call to \code{strptime}.

<<converters0,eval=TRUE,echo=TRUE>>= 
value <- 2.0
value.string <- sosConvertString(x = value, sos = mySOS)
print(class(value.string))

value <- "2.0"
value.double <- sosConvertDouble(x = value, sos = mySOS)
print(class(value.double))

value <- "1"
value.logical <- sosConvertLogical(x = value, sos = mySOS)
print(class(value.logical))

value <- "2010-01-01T12:00:00.000"
value.time <- sosConvertTime(x = value, sos = mySOS)
print(class(value.time))
@

The full list of currently supported units can be seen below. It mostly contains common numerical units which are converted to type \code{double}.

<<converters1,echo=TRUE,print=TRUE>>=
names(SosDataFieldConvertingFunctions())
@

The current list of a SOS connection's converters can be accessed with

<<converters2,echo=TRUE,eval=FALSE>>=
sosDataFieldConverters(mySOS)
@

The following connection shows a typical workflow of connecting to a new SOS for the first time, what the errors for missing converters look like, and how to add them to the SOS connection.

In Addition, this service shows errorenous behaviour regarding the reponse format (even if it is correctly set), so that the parameter \code{responseFormat} is set to \code{NA\_character} to be excluded in the request encoding. This results in additional warnings.

<<loadOrDownload_Converters3,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "mbariObs1"
.mbari <- "mbari.RData"
if(.goOnline) {
	MBARI <- SOS("http://mmisw.org/oostethys/sos",
		method = SosSupportedConnectionMethods()[["GET"]])
	myOff <- sosOfferings(MBARI)[[1]]
	myProc <- sosProcedures(MBARI)[[1]]

	mbariObs1 <- try(
			getObservation(sos = MBARI, offering = myOff, verbose = TRUE,
				procedure = myProc, responseFormat = NA_character_,
				saveOriginal = .obsFile)
		)
	save(MBARI, file = .mbari)
} else {
	load(.mbari)
	mbariObs1 <- parseFile(MBARI, .getFilePath(.obsFile))
}
@
<<converters3a,eval=FALSE,echo=TRUE>>= 
# GET connection
MBARI <- SOS("http://mmisw.org/oostethys/sos",
	method = SosSupportedConnectionMethods()[["GET"]])
myOff <- sosOfferings(MBARI)[[1]]
myProc <- sosProcedures(MBARI)[[1]]
mbariObs1 <- try(
	getObservation(sos = MBARI, offering = myOff,
		procedure = myProc, responseFormat = NA_character_)
)
@

An excerpt from the warnings list with regard to the conversion reads as follows.

<<converters3b,eval=FALSE,echo=TRUE>>= 
warnings()
@
<<converters3c,eval=TRUE,echo=FALSE>>= 
# ran manually:
#converterWarnings <- warnings()[25:29]
#.convWarnFile <- "converterWarnings.Rdata"
#save(converterWarnings, file = .convWarnFile)
#load(.convWarnFile)
#print(converterWarnings)
cat("...")
cat("25: In FUN(X[[7L]], ...) :
		swe:Quantity given without unit of measurement: Salinity
26: In .valParser(values = obj[[sweValuesName]], fields = .fields,  ... :
				No converter for the unit of measurement  S/m  with the definition  http://mmisw.org/ont/cf/parameter/conductivity ! Trying a default, but you can add one when creating a SOS using SosDataFieldConvertingFunctions().
27: In .valParser(values = obj[[sweValuesName]], fields = .fields,  ... :
				No converter found! Skipping field Conductivity 
No converter found! Skipping field http://mmisw.org/ont/cf/parameter/conductivity 
No converter found! Skipping field S/m 

28: In .valParser(values = obj[[sweValuesName]], fields = .fields,  ... :
				No converter found for the given field Salinity, http://mmisw.org/ont/cf/parameter/sea_water_salinity, NA
29: In .valParser(values = obj[[sweValuesName]], fields = .fields,  ... :
				No converter found! Skipping field Salinity 
No converter found! Skipping field http://mmisw.org/ont/cf/parameter/sea_water_salinity 
No converter found! Skipping field NA 

30: In FUN(X[[7L]], ...) :
		swe:Quantity given without unit of measurement: Salinity")
cat("...")
@

This shows warnings about unknown units of measurement and a swe:Quantity element (which describes a numeric field) without a given unit of measurement (which it should have as a numeric field). The next example creates conversion functions for these fields and repeats the operation.

<<converters4,eval=TRUE,echo=TRUE>>= 
myConverters <- SosDataFieldConvertingFunctions(
	"S/m" = sosConvertDouble,
	"http://mmisw.org/ont/cf/parameter/sea_water_salinity"
			= sosConvertDouble)
@
<<loadOrDownload_Converters4,eval=TRUE,echo=FALSE,results=hide>>=
.obsFile <- "mbariObs2"
.mbari2 <- "mbari2.RData"
if(.goOnline) {
	MBARI2 <- SOS("http://mmisw.org/oostethys/sos",
		method = SosSupportedConnectionMethods()[["GET"]],
		dataFieldConverters = myConverters)
	mbariObs2 <- getObservation(sos = MBARI2, offering = myOff, #verbose = TRUE,
		procedure = myProc, responseFormat = NA_character_,
		saveOriginal = .obsFile)
	save(MBARI2, file = .mbari2)
} else {
	load(.mbari2)
	mbariObs2 <- parseFile(MBARI2, .getFilePath(.obsFile))
}
@
<<converters4a,eval=FALSE,echo=TRUE>>= 
MBARI2 <- SOS("http://mmisw.org/oostethys/sos",
	method = SosSupportedConnectionMethods()[["GET"]],
	dataFieldConverters = myConverters)
mbariObs2 <- getObservation(sos = MBARI2, offering = myOff,
	procedure = myProc, responseFormat = NA_character_)
@

Subsequently, the second request results in more fields in the result.
 
<<converters5,eval=TRUE,echo=TRUE,print=TRUE>>= 
toString(names(sosResult(mbariObs1)))
toString(names(sosResult(mbariObs2)))
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Exception Handling}
\label{sec:exception}

When working with sos4R, two kinds of errors must be handled: service exceptions and errors within the package. The former can occur when a request is invalid or a service encounters internal exceptions. The latter can mean a bug or illegal settings within the package.

To understand both types of errorenous states, this sections explains the contents of the exception reports returned by the service and the functionalities to investigate the inner workings of the package.

\subsection{OWS Service Exceptions}

The service exceptions returned by a SOS are described in OGC Web Services Common (Whiteside, 2007) clause 8. The classes to handle the returned exceptions in sos4R are \code{OwsExceptionReport}, which contains a list of exception reports,  and \code{OwsException}, which contains slots for the parameters exception text(s), exception code, and locator. These are defined as follows and can be implementation specific.

\begin{description}
\item[ExceptionText] Text describing specific exception represented by the exceptionCode.
\item[exceptionCode] Code representing type of this exception.
\item[locator] Indicator of location in the client's operation request where this exception was encountered.
\end{description}

The standard exception codes and meanings are accessible by calling 

<<exceptionData,eval=FALSE,echo=TRUE>>= 
OwsExceptionsData()
@

directly in \pkg{sos4R} and are shown in table \ref{tab:exceptions}. The original table also contains the respective HTTP error codes and messages.

% row name & exceptionCode & meaningOfCode & locator & httpStatusCode & httpMessage \\ 
<<exceptionTable,echo=FALSE,results=tex>>=
library(xtable)
print(xtable(x = OwsExceptionsData()[,1:3],
				caption = "Exception Data Table (without HTTP columns).",
				label = c("tab:exceptions"), table.placement = "tbp",
				align = c("l", "l", "p{5cm}", "p{3cm}"),
				caption.placement = "top"),
		include.rownames = FALSE)
@

<<exceptionWarning1,eval=FALSE,echo=TRUE,print=TRUE>>= 
response <- try(getObservationById(sos = mySOS,
	observationId = "o_not_there"))
@

If an exception is received then it is also saved as a warning message. In this case, it reads as follows.

<<exceptionWarning2b,eval=TRUE,echo=FALSE>>=
cat("Warning:
In .handleExceptionReport(sos, .response) :
  Object of class OwsExceptionReport; version: 1.0.0; lang: NA;
 1 exception(s) (code @ locator : text):
  NoApplicableCode @ NA :
	Error while creating observations from database query result set: ERROR: invalid input syntax for integer: \"not_there\" ")
@

<<exceptionWarning2c,eval=TRUE,echo=FALSE,results=hide>>=
.file <- "exceptionResponse.RData"
if(.goOnline) {
	response <- getObservationById(sos = mySOS,
					observationId = "o_not_there")
	save(response, file = .file)
} else {
	load(.file)
}
@

The exception is also stored in the \code{response} object.

<<exceptionWarning2d,eval=TRUE,echo=TRUE>>=
response
@


\subsection{Inspect Requests and Verbose Printing}
\label{subsec:inspect}
\label{subsec:verbose}

The package offers two levels of inspection of the ongoing operations indicated by two boolean parameters, \code{inspect} and \code{verbose}. These are available in all service operation calls.

\begin{description}
\item[inspect] prints the raw requests and responses to the console.
\item[verbose] prints not only the requests, but also debugging and processing statements (e.g. intermediate steps during parsing).
\end{description}

The option \code{verboseOutput} when using the method \code{SOS()} turns on the verbose setting for all subsequent requests made to the created connection unless deactivated in an operation call. By using \code{verboseOutput} you can also debug the automatic GetCapabilities operations when creating a new SOS connections.

The output with these parameters enabled is too extensive to show within this document for actual GetObservation request.

<<inspect,eval=FALSE,echo=TRUE>>= 
off.4 <- sosOfferings(mySOS)[[4]]
getObservation(sos = mySOS, offering = off.4,
	procedure = sosProcedures(off.4)[[1]],
	inspect = TRUE)

getObservation(sos = mySOS,	offering = off.4,
	procedure = sosProcedures(off.4)[[1]],
	vebose = TRUE)

verboseSOS <- SOS(sosUrl(mySOS), verboseOutput = TRUE)
@


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Getting Started}
\label{sec:gettingstarted}

\subsection{Installation}

<<install,echo=TRUE,eval=FALSE>>= 
install.packages("sos4R")
@

\subsection{Demos}
\label{subsec:demos}

The \textbf{demos} are a good way to get started with the package. Please be aware that you need an internet connection for these demos, the used SOSs might be temporarily unavailable, and some of the demos are under construction.

<<demo,echo=TRUE,eval=FALSE>>= 
# list available demos:
demo(package = "sos4R")

# run a demo:
demo("airquality")
@

\begin{description}
\item[ades] SOS with French groundwater level data - \textbf{under construction}.
\item[airquality] The Air Quality SOS by ifgi provides EEA AirBase\footnote{\url{http://www.eea.europa.eu/themes/air/airbase}} data for Germany (and other countries). It is used for this demo about \textbf{spatio-temporal interpolation with inverse distance weighting} of NO2 observation in Germany using the packages \pkg{gstat} and \pkg{spacetime}.
\item[austria] SOSs by Research Studios Austria - \textbf{under construction}.
\item[ioos] Example using SOS by the Integrated Ocean Observing System which provides data in different formats, of which \textbf{KML} and \textbf{CSV} are shown in the demo - \textbf{under construction}.
\item[marinemeta] SOS by Marine Metadata Interoperability Initiative - \textbf{under construction}.
\item[oceanwatch] SOS by NOAA/SWFSC/ERD - \textbf{under construction}.
\item[pegel] Water gauge data in Germany by Pegelonline, shows how to create an xyplot of a set of variables.
\item[southesk] SOSs from South Esk Testbed by CSIRO, focuses on \textbf{data consolidation/fusion} and \textbf{plotting}. It also contains an example for partially changing the encoding by overriding a specific \code{encodeXML} function.
\item[weathersos] Time series analysis demo with weather data by ifgi, includes examples for \textbf{DescribeSensor} and data extraction from and plotting of \textbf{SensorML sensor descriptions}.
\end{description}

\subsection{Services}

There also is an imcomplete list of services that have been tested or are currently evaluated on the project homepage in the \enquote{data} area\footnote{\url{http://www.nordholmen.net/sos4r/data/}}. If you find or can provide new SOS with data useful to others, please do not hesitate to leave a comment on that page so that it can be inlcuded.

Additionally, a set of SOS URLs are available via the function \code{SosExampleServices()}.

<<exampleServices,echo=TRUE,print=TRUE>>= 
SosExampleServices()
@

Please note that the author of this document does not control these services and does not guarantee for any factors like correctness of data or availability.

\subsection{Quick Reference}

The package comes with a one page quick reference card, also known as a "cheat sheet", which shows everything that you need to know in an extremely concise manner. This is suitable for users with some experience with \pkg{sos4R}. You can open the document by loading the package and calling

<<cheatSheet,echo=TRUE,print=TRUE,eval=FALSE>>= 
sosCheatSheet()
@

and then printing it or saving it for later use. The most useful functions are highlighted in \textbf{bold font}.

\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Getting Support}
\label{sec:support}

If you want to ask \textbf{questions} about using the software, please go first
to the \fivetwon \textbf{forum} for the geostatistics community at
\url{http://geostatistics.forum.52north.org/} and check if a solution is
described there. If you are a frequent user please consider subscribing to the
geostatistics \textbf{mailing list}
(\url{http://list.52north.org/mailman/listinfo/geostatistics}). The forum and
mailing list are synchronized, so you only have to join respectively post at
either one.

If you are sure you \textbf{found a bug}, please submit it to the \fivetwon
Bugzilla at
\url{https://bugzilla.52north.org/enter_bug.cgi?product=52N\%20Geostatistics}
for the component \textbf{sos4R}. You can also brows a list of existing
bugs\footnote{\url{https://bugzilla.52north.org/buglist.cgi?product=52N\%20Geostatistics&component=sos4R}}.


%\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Developing sos4R}
\label{sec:developing}

\subsection*{Code Repository}

You can download (and also browse) the source code of \pkg{sos4R} directly from
the \fivetwon repository:

\begin{itemize}
\item \textbf{SVN resource URL}:
\url{https://svn.52north.org/svn/geostatistics/main/sos4R}. Please read the
documentation (especially the posting guide) of the \fivetwon
repositories\footnote{\url{http://52north.org/resources/source-repositories/}}.
Anonymous access for download is possible.
\item \textbf{Web access}:
\url{https://svn.52north.org/cgi-bin/viewvc.cgi/main/sos4R/?root=geostatistics}
\end{itemize}

The latest changes for every version are documented in the file \textbf{NEWS} in
the package root directory, which you can directly print to the console by
calling \code{sosNews()}, respectively the file \textbf{CHANGES} and
\code{sosChanges()} for older entries.

\subsection*{Developer Documentation}

See the developer documentation at the \fivetwon Wiki for detailed information
on how to use the checked out source project:
\url{https://wiki.52north.org/bin/view/Geostatistics/Sos4R }. You will find a
detailed description of the folder and class structure, the file naming scheme,
and an extensive list of tasks for future development.

\subsection*{Contribute}
Please get in touch with the community
lead\footnote{\url{http://52north.org/communities/geostatistics/community-contact}}
of the geostatistics community if you want to \textbf{become a contributor}.


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Acknowledgements}
\label{sec:acknowledgements}

The start of the project was generously supported by the \fivetwon Student Innovation Prize for Geoinformatics 2010. Thanks go also to Edzer Pebesma for contributing analyses to the demos.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{References}
\label{sec:references}

\begin{description}
\item Botts, M., 2007, OGC Implementation Specification 07-000: OpenGIS Sensor Model Language (SensorML)- Open Geospatial Consortium, Tech. Rep.
\item Br\"oring A., Echterhoff J., Jirka S., Simonis I., Everding T., Stasch C., Liang S., Lemmens R. New Generation Sensor Web Enablement. Sensors. 2011; 11(3):2652-2699.
\item Chambers, J.M., 2008, Software for Data Analysis, Programming with R. Springer, New York.
\item Cox, S., 2007, OGC Implementation Specification 07-022r1: Observations and Measurements - Part 1 - Observation schema. Open Geospatial Consortium. Tech. Rep.
\item Cox, S., 2007, OGC Implementation Specification 07-022r3: Observations and Measurements - Part 2 - Sampling Features. Open Geospatial Consortium. Tech. Rep.
\item Na, A., Priest, M., Niedzwiadek, H. and Davidson, J., 2007, OGC Implementation Specification 06-009r6: Sensor Observation Service, \url{http://portal.opengeospatial.org/files/?artifact_id=26667}, Open Geospatial Consortium, Tech. Rep. 
\item N\"ust, D., Stasch, C. and Pebesma, E. J., 2011, Connecting R to the Sensor Web in Geertman, S., Reinhardt, W. and Toppen, F. (Eds.) Advancing Geoinformation Science for a Changing World, Springer Lecture Notes in Geoinformation and Cartography. 
\item Portele, C., 2003, OGC Implementation Specification 07-036: OpenGIS Geography Markup Language (GML) Encoding Standard, version: 3.00. Open Geospatial Consortium, Tech. Rep.
\item Vretanos, P.A., 2005, OGC Implementation Specification 04-095: OpenGIS Filter Encoding Implementation Specification. Open Geospatial Consortium, Tech. Rep.
\item Whiteside, A., Greenwood, J., 2008, OGC Implementation Specification 06-121r9: OGC Web Services Common Specification. Open Geospatial Consortium, Tech. Rep.
\end{description}

<<options,echo=FALSE,results=hide,print=FALSE>>= 
print("###################################################################")
if(.goOnline) {
	print("####### Data for this vignette was downloaded ONLINE. #############")
} else {
	print("####### Data for this vignette was loaded from LOCAL FILES. #######")
}
print("###################################################################")
@

\end{document}
