​Shannon–type expected information gain can be used to evaluate the relevance of a proposed experiment subjected to uncertainty. The estimation of such gain, however, relies on a double-loop integration. More- over, its numerical integration in multidimensional cases, e.g., when using Monte Carlo sampling meth- ods, is therefore computationally intractable for real- istic physical models, especially those involving the solution of partial differential equations. In this pa- per, we present a new methodology, based on the Laplace approximation for the integration of the pos- terior probability density function (pdf), to accelerate the estimation of the expected information gains in the model parameters and predictive quantities of interest for both determined and underdetermined models. We obtain a closed–form approximation of the inner inte- gral and the corresponding dominant error term, such that only a single–loop integration is needed to carry out the estimation of the expected information gain.

Specifically when the parameters can not be deter- mined completely by the experimental data, we carry out the Laplace approximations in the directions or- thogonal to the null space of the corresponding Ja- cobian matrix, so that the information gain (K–L di- vergence) can be reduced to an integration against the marginal density of the transformed parameters which are not determined by the experiments. This in- tegration can be numerically estimated by projecting the prior samples or quadratures onto the underdeter- mined manifold via the gradient directions.

 

To deal with the issue of dimensionality in cer- tain complex problems, we can use a sparse quadra- ture for the integration over the prior pdf. We demon- strate the accuracy, efficiency and robustness of the proposed method via several nonlinear numerical ex- amples, including the designs of the scalar parameter in an one–dimensional cubic polynomial function, the design of the same scalar in a slightly modified func- tion with two indistinguishable parameters, the reso- lution width and measurement time for a blurred sin- gle peak spectrum, and the sensor optimization for a impedance tomography problem.