product Monte Carlo

A new, long, arXival about product Monte Carlo by Carlotti, Liu, and Yao. where the target is a product of densities \(\prod_k\pi_k\)(implicitely) assumed to be integrable. Read in the plane to B’ham and Warwick U. The starting point of the method is a collection of simulations from each term in the product, which imho is unnecessarily restrictive since gains could be made by coupling similar densities and exlploiting other ways of sharing strength. While PMC is already used for population Monte Carlo (Iwaki, 2000), the authors resort to this accronym for their approach, which is a form of Rao-Blackwellisation, integrating an estimator based on a “consensus” statistic conditional on a “discrepancy” statistic.

“In many of these applications, individual component densities \(\pi_k\) cannot be evaluated: local densities in privacy settings can leak data information; posterior predictive densities in expert pooling require additional integration; and generative-model-based samplers and simulation-based inference methods provide samples without tractable density evaluations.”

A consensus C is a statistic (transform) of the K realisations from the component densities \(\pi_k\) into a pseudo-observation that return the entry if all realisations are identical and that is equivariant by translation. While a discrepancy D is a statistic (of arbitrary dimension) of the difference between the “first” realisation and the \(K-1\) other ones that is null for a null input and bounded away from zero outside a neighbourhood of zero. (This seems to apply to a single realisation from each density. How does it extend to full samples from each density? What of the dependence on which density is labelled “first”? What of the impact of the choice of either statistic? Is there a minimal sufficiency or maximal ancilarity at play here?)

Judging from Section 2.2 when the algorithm is produced, the method appears to build consensus and discrepancy one output at a time, rather than postprocessing entire samples, which creates an unnecessary dependence on the (ancillary) order in which realizations are produced. They are used to construct a sample of \(C_s\) and \(D_s\) \((s=1,\ldots,S\) and to estimate
\[
f(D) = \mathbb E[h(C)|D]
\]
(which keeps the expectation exact) by \(\hat f_S(D)\), the integral estimator being \(\hat f_S(0)\). And biased. (Which reminds me of post-processing in ABC,à la Beaumont et al., 2003.) Further Bayesian processing can be found in modelling \(f(\cdot)\) as a Gaussian process.

Follows some theoretical developments like Cramèr-Rao bounds that fail to address the elephant in the room, namely the impact of the choices of C and D and the lack of differenciation between the components of the product. But the paper opens interesting prospects for privacy constrained inference.