Skip to main content
Article
A Bayesian Interpretation of the Linearly-Constrained Cross-Entropy Minimization Problem
Engineering Optimization (1993)
  • Jacob Tsao, San Jose State University
  • Shu-Cherng Fang
  • D N Lee
Abstract

Both the linearly-constrained minimum cross-entropy (LCMXE) method and the Bayesian parameter estimation procedure have been widely used for solving various engineering problems. From the viewpoint of the information/decision theory, both approaches start with a prior distribution for a random variable, “absorb” new information, and finally produce a posterior distribution. In this paper, an equivalence relationship between these two approaches is established by identifying certain statistical experiments “embedded” in the LCMXE framework. Interestingly, the dual of the LCMXE problem actually “translates” the new information into its Bayesian counterpart. It is also shown that, while new information may come in stages, the identical final posterior can be obtained by applying the LCMXE method either stagewise or collectively. The equivalence further implies that the LCMXE method can help select a proper exponential family as the statistical model for the Bayesian experiments.

Keywords
  • Bayesian Interpretation,
  • Cross-Entropy,
  • Minimization Problem
Publication Date
1993
Citation Information
Jacob Tsao, Shu-Cherng Fang and D N Lee. "A Bayesian Interpretation of the Linearly-Constrained Cross-Entropy Minimization Problem" Engineering Optimization Vol. 22 Iss. 1 (1993)
Available at: http://works.bepress.com/jacob_tsao/45/