NMixMCMC                package:mixAK                R Documentation

_M_C_M_C _e_s_t_i_m_a_t_i_o_n _o_f (_m_u_l_t_i_v_a_r_i_a_t_e) _n_o_r_m_a_l _m_i_x_t_u_r_e_s _w_i_t_h _p_o_s_s_i_b_l_y
_c_e_n_s_o_r_e_d _d_a_t_a.

_D_e_s_c_r_i_p_t_i_o_n:

     This function runs MCMC for a model in which unknown density is
     specified as a normal mixture with either known or unknown number
     of components. With a prespecified number of components, MCMC is
     implemented through Gibbs sampling (see Diebolt and Robert, 1994)
     and dimension of the data can be arbitrary. With unknown number of
     components, currently only univariate case is implemented using
     the reversible jump MCMC (Richardson and Green, 1997).

     Further, the data are allowed to be censored in which case
     additional Gibbs step is used within the MCMC algorithm

_U_s_a_g_e:

     NMixMCMC(y0, y1, censor, scale, prior,
              init, init2, RJMCMC,
              nMCMC=c(burn=10, keep=10, thin=1, info=10),
              PED, keep.chains=TRUE, onlyInit=FALSE, dens.zero=1e-300)

     ## S3 method for class 'NMixMCMC':
     print(x, dic, ...)

     ## S3 method for class 'NMixMCMClist':
     print(x, ped, dic, ...)

_A_r_g_u_m_e_n_t_s:

      y0: numeric vector of length n or n x p matrix with observed
          data. It contains exactly observed, right-censored,
          left-censored data and lower limits for interval-censored
          data.

      y1: numeric vector of length n or n x p matrix with upper limits
          for interval-censored data. Elements corresponding to exactly
          observed, right-censored or left-censored data are ignored
          and can be filled arbitrarily (by 'NA''s) as well.

          It does not have to be supplied if there are no
          interval-censored data. 

  censor: numeric vector of length n or n x p matrix with censoring
          indicators. The following values indicate:


          _0 right-censored observation,

          _1 exactly observed value,

          _2 left-censored observation,

          _3 interval-censored observation.      

          If it is not supplied then it is assumed that all values are
          exactly observed. 

   scale: a list specifying how to scale the data before running MCMC.
          It should have two components:


          _s_h_i_f_t a vector of length 1 or p specifying shift vector m,

          _s_c_a_l_e a vector of length 1 or p specifying diagonal of the
               scaling matrix S.

          If there is no censoring, and argument 'scale' is missing
          then the data are scaled to have zero mean and unit
          variances, i.e., 'scale(y0)' is used for MCMC. In the case
          there is censoring and 'scale' is missing, 'scale$shift' is
          taken to be a sample mean of 'init$y' and 'scale$scale' are
          sample standard deviations of columns of 'init$y'.

          If you do not wish to scale the data before running MCMC,
          specify 'scale=list(shift=0, scale=1)'. 

   prior: a list with the parameters of the prior distribution. It
          should have the following components (for some of them, the
          program can assign default values and the user does not have
          to specify them if he/she wishes to use the defaults):

          _p_r_i_o_r_K a character string which specifies the type of the
               prior for K (the number of mixture components). It
               should have one of the following values:

               cr 'fixed'cr Number of mixture components is assumed
               to be fixed to K[max]. This is a *default* value.

               cr 'uniform'cr A priori K ~ Unif{1,...,K[max]}.

               cr 'tpoisson'cr A priori K ~ truncated-Poiss(lambda,
               K[max]).

          _p_r_i_o_r_m_u_Q a character string which specifies the type of the
               prior for mu[1], ..., mu[K[max]] (mixture means) and
               Q[1], ..., Q[K[max]] (inverted mixture covariance
               matrices). It should have one of the following values:

               cr 'independentC'cr = independent conjugate prior
               (this is a *default* value). That is, a priori

          (mu[j], Q[j]) ~ N(xi[j], D[j]) * Wishart(zeta, Xi)

               independently for j=1,...,K, where normal means
               xi[1],...,xi[K], normal variances D[1],...,D[K], and
               Wishart degrees of freedom zeta are specified further as
               'xi', 'D', 'zeta' components of the list 'prior'.

               cr 'naturalC'cr = natural conjugate prior. That is, a
               priori

    (mu[j], Q[j]) ~ N(xi[j], (c[j]Q[j])^(-1)) * Wishart(zeta, Xi)

               independently for j=1,...,K, where normal means
               xi[1],...,xi[K], precisions c[1],...,c[K], and Wishart
               degrees of freedom zeta are specified further as 'xi',
               'ce', 'zeta' components of the list 'prior'.

               cr For both, independent conjugate and natural conjugate
               prior, the Wishart scale matrix Xi is assumed to be
               diagonal with gamma[1],...,gamma[p] on a diagonal. For
               gamma[j]^(-1) (j=1,...,K) additional gamma hyperprior
               G(g[j], h[j]) is assumed. Values of g[1],...,g[p] and
               h[1],...,h[p] are further specified as 'g' and 'h'
               components of the 'prior' list.

          _K_m_a_x maximal number of mixture components K[max]. It must
               *always be specified* by the user.

          _l_a_m_b_d_a parameter lambda for the truncated Poisson prior on K.
               It must be positive and must *always be specified* if
               'priorK' is 'tpoisson'.

          _d_e_l_t_a parameter delta for the Dirichlet prior on the mixture
               weights w[1],...,w[K]. It must be positive. Its
               *default* value is 1.

          _x_i a numeric value, vector or matrix which specifies xi[1],
               ..., xi[K[max]] (prior means for the mixture means
               mu[1], ..., mu[K[max]]). *Default* value is a matrix
               K[max] x p with midpoints of columns of 'init$y' in rows
               which follows Richardson and Green (1997).

               cr    If p=1 and 'xi'=xi is a single value then
               xi[1]=...=xi[K[max]] = xi.

               cr If p=1 and 'xi'=boldsymbol{xi} is a vector of length
               K[max] then the j-th element of 'xi' gives xi[j]
               (j=1,...,K[max]).

               cr    If p>1 and 'xi'=xi is a vector of length p then
               xi[1]=...=xi[K[max]] = xi.

               cr If p>1 and 'xi' is a K[max] x p matrix then the j-th
               row of 'xi' gives xi[j] (j=1,...,K[max]).    

          _c_e a numeric value or vector which specifies prior precision
               parameters c[1],...,c[K[max]] for the mixture means
               mu[1], ..., mu[K[max]] when 'priormuQ' is 'naturalC'.
               Its *default* value is a vector of ones which follows
               Cappe, Robert and Ryden (2003).

               cr If 'ce'=c is a single value then
               c[1]=...=c[K[max]]=c.

               cr If 'ce'c is a vector of length K[max] then the j-th
               element of 'ce' gives c[j] (j=1,...,K[max]).      

          _D a numeric vector or matrix which specifies D[1], ...,
               D[K[max]] (prior variances or covariance matrices of the
               mixture means mu[1], ..., mu[K[max]] when 'priormuQ' is
               'independentC'.) Its *default* value is a diagonal
               matrix with squared ranges of each column of 'init$y' on
               a diagonal.

               cr      If p=1 and 'D'=d is a single value then
               d[1]=...=d[K[max]] = d.

               cr If p=1 and 'D'=d is a vector of length K[max] then
               the j-th element of 'D' gives d[j] (j=1,...,K[max]).

               cr      If p>1 and 'D'=D is a p x p matrix then
               D[1]=...=D[K[max]] = D.

               cr If p>1 and 'D' is a (K[max]*p) x p matrix then the
               the first p rows of 'D' give D[1], rows p+1,...,2p of
               'D' give D[2] etc.

          _z_e_t_a degrees of freedom zeta for the Wishart prior on the
               inverted mixture variances Q[1], ...,Q[K[max]].. It must
               be higher then p-1. Its *default* value is p + 1.    

          _g a value or a vector of length p with the shape parameters
               g[1],...,g[p] for the Gamma hyperpriors on
               gamma[1],...,gamma[p].         It must be positive. Its
               *default* value is a vector (0.2,...,0.2)'.

          _h a value or a vector of length p with the rate parameters
               h[1],...,h[p] for the Gamma hyperpriors on
               gamma[1],...,gamma[p]. It must be positive. Its
               *default* value is a vector containing 10/R[l]^2, where
               R[l] is a range of the l-th column of 'init$y'. 

    init: a list with the initial values for the MCMC. All initials can
          be determined by the program if they are not specified. The
          list may have the following components:

          _y a numeric vector or matrix with the initial values for the
               latent censored observations.

          _K a numeric value with the initial value for the number of
               mixture components.

          _w a numeric vector with the initial values for the mixture
               weights.

          _m_u a numeric vector or matrix with the initial values for the
               mixture means.

          _S_i_g_m_a a numeric vector or matrix with the initial values for
               the mixture variances.

          _L_i a numeric vector with the initial values for the Colesky
               decomposition of the mixture inverse variances.

          _g_a_m_m_a_I_n_v a numeric vector with the initial values for the
               inverted components of the hyperparameter gamma.

          _r a numeric vector with the initial values for the mixture
               allocations.      

   init2: a list with the initial values for the second chain needed to
          estimate the penalized expected deviance of Plummer (2008).
          The list 'init2' has the same structure as the list 'init'.
          All initials in 'init2' can be determined by the program
          (differently than the values in 'init') if they are not
          specified.

          Ignored when 'PED' is 'FALSE'. 

  RJMCMC: a list with the parameters needed to run reversible jump MCMC
          for mixtures with varying number of components. It does not
          have to be specified if the number of components is fixed.
          Most of the parameters can be determined by the program if
          they are not specified. The list may have the following
          components:

          _P_a_c_t_i_o_n probabilities (or proportionalit constants) which are
               used to choose an action of the sampler within each
               iteration of MCMC to update the mixture related
               parameters. Let 'Paction' = (p[1], p[2], p[3])'. Then
               with probability p[1] only steps assuming fixed k
               (number of mixture components) are performed, with
               probability p[2] split-combine move is proposed and with
               probability p[3] birth-death move is proposed.

               If not specified (default) then in each iteration of
               MCMC, all sampler actions are performed.

          _P_s_p_l_i_t a numeric vector of length 'prior$Kmax' giving
               conditional probabilities of the split move given k as
               opposite to the combine move.

               Default value is (1, 0.5, ..., 0.5, 0)'.

          _P_b_i_r_t_h a numeric vector of length 'prior$Kmax' giving
               conditional probabilities of the birth move given k as
               opposite to the death move.

               Default value is (1, 0.5, ..., 0.5, 0)'.

          _p_a_r._u_1 a two component vector with parameters of the beta
               distribution used to generate an auxiliary value u[1].

               A default value is 'par.u1' = (2, 2)', i.e., u[1] ~
               Beta(2, 2).    

          _p_a_r._u_2 a two component vector (for p=1) or a matrix (for p >
               1) with two columns with parameters of the distributions
               of the auxiliary values u[2,1],...,u[2,p] in rows.

               A default value leads to u[2,d] ~ Unif(-1, 1)
               (d=1,...,p-1), u[2,p] ~ Beta(1, 2p).


          _p_a_r._u_3 a two component vector (for p=1) or a matrix (for p >
               1) with two columns with parameters of the distributions
               of the auxiliary values u[3,1],...,u[3,p] in rows.

               A default value leads to u[3,d] ~ Unif(0, 1)
               (d=1,...,p-1), u[3,p] ~ Beta(1, p).     

   nMCMC: numeric vector of length 4 giving parameters of the MCMC
          simulation. Its components may be named (ordering is then
          unimportant) as:

          _b_u_r_n length of the burn-in (after discarding the thinned
               values), can be equal to zero as well.

          _k_e_e_p length of the kept chains (after discarding the thinned
               values), must be positive.

          _t_h_i_n thinning interval, must be positive.

          _i_n_f_o interval in which the progress information is printed on
               the screen.      

          In total (M[burn] + M[keep]) * M[thin] MCMC scans are
          performed. 

     PED: a logical value which indicates whether the penalized
          expected deviance (see Plummer, 2008 for more details) is to
          be computed (which requires two parallel chains). If not
          specified, 'PED' is set to 'TRUE' for models with fixed
          number of components and is set to 'FALSE' for models with
          numbers of components estimated using RJ-MCMC. 

keep.chains: logical. If 'FALSE', only summary statistics are returned
          in the resulting object. This might be useful in the model
          searching step to save some memory.

onlyInit: logical. If 'TRUE' then the function only determines
          parameters of the prior distribution, initial values, values
          of 'scale' and parameters for the reversible jump MCMC.

dens.zero: a small value used instead of zero when computing deviance
          related quantities.

       x: an object of class 'NMixMCMC' or 'NMixMCMClist' to be
          printed.

     dic: logical which indicates whether DIC should be printed. By
          default, DIC is printed only for models with a fixed number
          of mixture components. 

     ped: logical which indicates whether PED should be printed. By
          default, PED is printed only for models with a fixed number
          of mixture components. 

     ...: additional arguments passed to the default 'print' method.

_D_e_t_a_i_l_s:

     See accompanying paper (Kom&#225rek, 2009). In the rest of the
     helpfile, the same notation is used as in the paper, namely, n
     denotes the number of observations, p is dimension of the data, K
     is the number of mixture components, w[1],...,w[K] are mixture
     weights,  mu[1],...,mu[K] are mixture means, Sigma_1,...,Sigma_K
     are mixture variance-covariance matrices, Q_1,...,Q_K are their
     inverses.

     For the data y[1],...,y[n] the following g(y) density is assumed

 g(y) = |S|^(-1) * sum[j=1]^K w[j] phi(S^(-1)*(y - m) | mu[j], Sigma[j]),

     where phi(. | mu, Sigma) denotes a density of the (multivariate)
     normal distribution with mean mu and a~variance-covariance matrix
     Sigma. Finally, S is a pre-specified diagonal scale matrix and m
     is a pre-specified shift vector. Sometimes, by setting m to sample
     means of components of y and diagonal of S to sample standard
     deviations of y (considerable) improvement of the MCMC algorithm
     is achieved.

_V_a_l_u_e:

     An object of class 'NMixMCMC' or class 'NMixMCMClist'. Object of
     class 'NMixMCMC' is returned if 'PED' is 'FALSE'. Object of class
     'NMixMCMClist' is returned if 'PED' is 'TRUE'. 

     Objects of class 'NMixMCMC' have the following components: 

    iter: index of the last iteration performed.

   nMCMC: used value of the argument 'nMCMC'.

     dim: dimension p of the distribution of data

   prior: a list containing the used value of the argument 'prior'.

    init: a list containing the used value of the argument 'init'.

  RJMCMC: a list containing the used value of the argument 'RJMCMC'.

   scale: a list containing the used value of the argument 'scale'.

   state: a list having the components labeled 'y', 'K', 'w', 'mu',
          'Li', 'Q', 'Sigma', 'gammaInv', 'r' containing the last
          sampled values of generic parameters.

   freqK: frequency table of K based on the sampled chain.

   propK: posterior distribution of K based on the sampled chain.

     DIC: a 'data.frame' having columns labeled 'DIC', 'pD', 'D.bar',
          'D.in.bar' containing values used to compute deviance
          information criterion (DIC). Currently only DIC[3] of Celeux
          et al. (2006) is implemented. 

   moves: a 'data.frame' which summarizes the acceptance probabilities
          of different move types of the sampler.

       K: numeric vector with a chain for K (number of mixture
          components).

       w: numeric vector or matrix with a chain for w (mixture
          weights). It is a matrix with K columns when K is fixed.
          Otherwise, it is a vector with weights put sequentially after
          each other.

      mu: numeric vector or matrix with a chain for mu (mixture means).
          It is a matrix with p*K columns when K is fixed. Otherwise,
          it is a vector with means put sequentially after each other.

       Q: numeric vector or matrix with a chain for lower triangles of
          Q (mixture inverse variances). It is a matrix with
          (p*(p+1)2)*K columns when K is fixed. Otherwise, it is a
          vector with lower triangles of Q matrices put sequentially
          after each other.

   Sigma: numeric vector or matrix with a chain for lower triangles of
          Sigma (mixture variances). It is a matrix with (p*(p+1)2)*K
          columns when K is fixed. Otherwise, it is a vector with lower
          triangles of Sigma matrices put sequentially after each
          other.

      Li: numeric vector or matrix with a chain for lower triangles of
          Cholesky decompositions of Q matrices. It is a matrix with
          (p*(p+1)2)*K columns when K is fixed. Otherwise, it is a
          vector with lower triangles put sequentially after each
          other.

gammaInv: matrix with p columns with a chain for inverses of the
          hyperparameter gamma. 

   order: numeric vector or matrix with order indeces of mixture
          components. It is a matrix with K columns when K is fixed.
          Otherwise it is a vector with orders put sequentially after
          each other.

    rank: numeric vector or matrix with rank indeces of mixture
          components. It is a matrix with K columns when K is fixed.
          Otherwise it is a vector with ranks put sequentially after
          each other.

 mixture: 'data.frame' with columns labeled 'y.Mean.*', 'y.SD.*',
          'y.Corr.*.*', 'z.Mean.*', 'z.SD.*', 'z.Corr.*.*' containing
          the chains for the means, standard deviations and
          correlations of the distribution of the original ('y') and
          scaled ('z') data based on a normal mixture at each
          iteration. 

deviance: 'data.frame' with columns labeles 'LogL0', 'LogL1',
          'dev.complete', 'dev.observed' containing the chains of
          quantities needed to compute DIC. 

    pm.y: a 'data.frame' with p columns with posterior means for
          (latent) values of observed data (useful when there is
          censoring).

    pm.z: a 'data.frame' with p columns with posterior means for
          (latent) values of scaled observed data (useful when there is
          censoring).

pm.indDev: a 'data.frame' with columns labeled 'LogL0', 'LogL1',
          'dev.complete', 'dev.observed', 'pred.dens' containing
          posterior means of individual contributions to the deviance. 

pred.dens: a numeric vector with the predictive density of the data
          based on the MCMC sample evaluated at data points.

          Note that when there is censoring, this is not exactly the
          predictive density as it is computed as the average of
          densities at each iteration evaluated at sampled values of
          latent observations at iterations. 

poster.comp.prob1: a matrix which is present in the output object if
          the number of mixture components in the distribution of
          random effects is fixed and equal to K. In that case,
          'poster.comp.prob1' is a matrix with K columns and n rows
          with estimated posterior component probabilities - posterior
          means of the components of the underlying 0/1 allocation
          vector.

          These can be used for possible clustering of the subjects. 

poster.comp.prob2: a matrix which is present in the output object if
          the number of mixture components in the distribution of
          random effects is fixed and equal to K. In that case,
          'poster.comp.prob2' is a matrix with K columns and n rows 
          with estimated posterior component probabilities - posterior
          mean over model parameters.

          These can also be used for possible clustering of the
          subjects 

summ.y.Mean: Posterior summary statistics based on chains stored in
          'y.Mean.*' columns of the 'data.frame' 'mixture'.

summ.y.SDCorr: Posterior summary statistics based on chains stored in
          'y.SD.*' and 'y.Corr.*.*' columns of the 'data.frame'
          'mixture'.

summ.z.Mean: Posterior summary statistics based on chains stored in
          'z.Mean.*' columns of the 'data.frame' 'mixture'.

summ.z.SDCorr: Posterior summary statistics based on chains stored in
          'z.SD.*' and 'z.Corr.*.*' columns of the 'data.frame'
          'mixture'.

poster.mean.w: a numeric vector with posterior means of mixture weights
          after re-labeling. It is computed only if K is fixed and even
          then I am not convinced that these are useful posterior
          summary statistics. In any case, they should be used with
          care.

poster.mean.mu: a matrix with posterior means of mixture means after
          re-labeling. It is computed only if K is fixed and even then
          I am not convinced that these are useful posterior summary
          statistics. In any case, they should be used with care.

poster.mean.Q: a list with posterior means of mixture inverse variances
          after re-labeling. It is computed only if K is fixed and even
          then I am not convinced that these are useful posterior
          summary statistics. In any case, they should be used with
          care.

poster.mean.Sigma: a list with posterior means of mixture variances
          after re-labeling. It is computed only if K is fixed and even
          then I am not convinced that these are useful posterior
          summary statistics. In any case, they should be used with
          care.

poster.mean.Li: a list with posterior means of Cholesky decompositions
          of mixture inverse variances after re-labeling. It is
          computed only if K is fixed and even then I am not convinced
          that these are useful posterior summary statistics. In any
          case, they should be used with care.

_O_b_j_e_c_t _o_f _c_l_a_s_s _N_M_i_x_M_C_M_C_l_i_s_t:

     Objects of class 'NMixMCMClist' is the list having two components
     of class 'NMixMCMC' representing two parallel chains and
     additionally the following components:

     _P_E_D values of penalized expected deviance and related quantities.
          It is a vector with five components: 'D.expect' = estimated
          expected deviance, where the estimate is based on two
          parallel chains; 'popt' = estimated penalty, where the
          estimate is based on simple MCMC average based on two
          parallel chains; 'PED' = estimated penalized expected
          deviance = 'D.expect' + 'popt'; 'wpopt' = estimated penalty,
          where the estimate is based on weighted MCMC average (through
          importance sampling) based on two parallel chains; 'wPED' =
          estimated penalized expected deviance = 'D.expect' + 'wpopt'.

     _p_o_p_t contributions to the unweighted penalty from each
          observation.

     _w_p_o_p_t contributions to the weighted penalty from each observation.

     _i_n_v._D for each observation, number of iterations (in both chains),
          where the deviance was in fact equal to infinity (when the
          corresponding density was lower than 'dens.zero') and was not
          taken into account when computing 'D.expect'.

     _i_n_v._p_o_p_t for each observation, number of iterations, where the
          penalty was in fact equal to infinity and was not taken into
          account when computing 'popt'.

     _i_n_v._w_p_o_p_t for each observation, number of iterations, where the
          importance sampling weight was in fact equal to infinity and
          was not taken into account when computing 'wpopt'.

     _s_u_m_I_S_w for each observation, sum of importance sampling weights.

_A_u_t_h_o_r(_s):

     Arno&#353t Kom&#225rek arnost.komarek[AT]mff.cuni.cz

_R_e_f_e_r_e_n_c_e_s:

     Celeux, G., Forbes, F., Robert, C. P., and Titterington, D. M.
     (2006). Deviance information criteria for missing data models.
     _Bayesian Analysis_, *1*, 651-674.

     Capp&#233, Robert and Ryd&#233n (2003). Reversible jump,
     birth-and-death and more general continuous time Markov chain
     Monte Carlo samplers. _Journal of the Royal Statistical Society,
     Series B_, *65*, 679-700.

     Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture
     distributions through Bayesian sampling. _Journal of the Royal
     Statistical Society, Series B_, *56*, 363-375.

     Kom&#225rek, A. (2009). A new R package for Bayesian estimation of
     multivariate normal mixtures allowing for selection  of the number
     of components and interval-censored data. _Computational
     Statistics and Data Analysis_, *53*, 3932-3947.

     Plummer, M. (2008). Penalized loss functions for Bayesian model
     comparison. _Biostatistics_, *9*, 523-539.

     Richardson, S. and Green, P. J. (1997). On Bayesian analysis of
     mixtures with unknown number of components (with Discussion).
     _Journal of the Royal Statistical Society, Series B_, *59*,
     731-792.

     Spiegelhalter, D. J.,Best, N. G., Carlin, B. P., and van der
     Linde, A. (2002). Bayesian measures of model complexity and fit
     (with Discussion). _Journal of the Royal Statistical Society,
     Series B_, *64*, 583-639.

_S_e_e _A_l_s_o:

     'NMixPredDensMarg', 'NMixPredDensJoint2'.

_E_x_a_m_p_l_e_s:

     ## See also additional material available in 
     ## YOUR_R_DIR/library/mixAK/doc/
     ## or YOUR_R_DIR/site-library/mixAK/doc/
     ## - files Galaxy.pdf, Faithful.pdf, Tandmob.pdf
     ## ==============================================

     ## Simple analysis of Anderson's iris data
     ## ==============================================
     library("colorspace")

     data(iris, package="datasets")
     summary(iris)
     VARS <- names(iris)[1:4]
     #COLS <- rainbow_hcl(3, start = 60, end = 240)
     COLS <- c("red", "darkblue", "darkgreen")
     names(COLS) <- levels(iris[, "Species"])

     ### Prior distribution and the length of MCMC
     Prior <- list(priorK = "fixed", Kmax = 3)
     nMCMC <- c(burn=5000, keep=10000, thin=5, info=1000)

     ### Run MCMC
     set.seed(20091230)
     fit <- NMixMCMC(y0 = iris[, VARS], prior = Prior, nMCMC = nMCMC)

     ### Basic posterior summary
     print(fit)

     ### Univariate marginal posterior predictive densities
     ### based on chain #1
     pdens1 <- NMixPredDensMarg(fit[[1]], lgrid=150)
     plot(pdens1)
     plot(pdens1, main=VARS, xlab=VARS)

     ### Bivariate (for each pair of margins) predictive densities
     ### based on chain #1
     pdens2a <- NMixPredDensJoint2(fit[[1]])
     plot(pdens2a)

     plot(pdens2a, xylab=VARS)
     plot(pdens2a, xylab=VARS, contour=TRUE)

     ### Determine the grid to compute bivariate densities
     grid <- list(Sepal.Length=seq(3.5, 8.5, length=75),
                  Sepal.Width=seq(1.8, 4.5, length=75),
                  Petal.Length=seq(0, 7, length=75),
                  Petal.Width=seq(-0.2, 3, length=75))
     pdens2b <- NMixPredDensJoint2(fit[[1]], grid=grid)
     plot(pdens2b, xylab=VARS)

     ### Plot with contours
     ICOL <- rev(heat_hcl(20, c=c(80, 30), l=c(30, 90), power=c(1/5, 2)))
     oldPar <- par(mfrow=c(2, 3), bty="n")
     for (i in 1:3){
       for (j in (i+1):4){
         NAME <- paste(i, "-", j, sep="")
         MAIN <- paste(VARS[i], "x", VARS[j])
         image(pdens2b$x[[i]], pdens2b$x[[j]], pdens2b$dens[[NAME]], col=ICOL,
               xlab=VARS[i], ylab=VARS[j], main=MAIN)
         contour(pdens2b$x[[i]], pdens2b$x[[j]], pdens2b$dens[[NAME]], add=TRUE, col="brown4")
       }  
     }  

     ### Plot with data
     for (i in 1:3){
       for (j in (i+1):4){
         NAME <- paste(i, "-", j, sep="")
         MAIN <- paste(VARS[i], "x", VARS[j])
         image(pdens2b$x[[i]], pdens2b$x[[j]], pdens2b$dens[[NAME]], col=ICOL,
               xlab=VARS[i], ylab=VARS[j], main=MAIN)
         for (spec in levels(iris[, "Species"])){
           Data <- subset(iris, Species==spec)
           points(Data[,i], Data[,j], pch=16, col=COLS[spec])
         }  
       }  
     }  

     ### Set the graphical parameters back to their original values
     par(oldPar)

     ### Clustering based on posterior summary statistics of component allocations
     ### or on the posterior distribution of component allocations
     ### (these are two equivalent estimators of probabilities of belonging
     ###  to each mixture components for each observation)
     p1 <- fit[[1]]$poster.comp.prob1
     p2 <- fit[[1]]$poster.comp.prob2

     ### Clustering based on posterior summary statistics of mixture weight, means, variances
     p3 <- NMixPlugDA(fit[[1]], iris[, VARS])
     p3 <- p3[, paste("prob", 1:3, sep="")]

       ### Observations from "setosa" species (all would be allocated in component 1)
     apply(p1[1:50,], 2, quantile, prob=seq(0, 1, by=0.1))
     apply(p2[1:50,], 2, quantile, prob=seq(0, 1, by=0.1))
     apply(p3[1:50,], 2, quantile, prob=seq(0, 1, by=0.1))

       ### Observations from "versicolor" species (almost all would be allocated in component 2)
     apply(p1[51:100,], 2, quantile, prob=seq(0, 1, by=0.1))
     apply(p2[51:100,], 2, quantile, prob=seq(0, 1, by=0.1))
     apply(p3[51:100,], 2, quantile, prob=seq(0, 1, by=0.1))

       ### Observations from "virginica" species (all would be allocated in component 3)
     apply(p1[101:150,], 2, quantile, prob=seq(0, 1, by=0.1))
     apply(p2[101:150,], 2, quantile, prob=seq(0, 1, by=0.1))
     apply(p3[101:150,], 2, quantile, prob=seq(0, 1, by=0.1))

