先验选择疑问：指数型无信息与有信息先验模型的似然差异

Hey there! Let’s unpack why your non-informative exponential prior is giving a higher likelihood value than the informative one—this is a super common question when getting hands-on with Bayesian models, so no worries if you didn’t dive into all the model specifics upfront.

Here are the key reasons this might be happening:

• You’re probably comparing likelihood at the MAP estimate (not the marginal likelihood)
  The maximum likelihood estimate (MLE) is the parameter value that makes your data look most probable, and it ignores priors entirely. A non-informative prior is "weak"—it barely nudges the MAP (maximum a posteriori) estimate away from the MLE. But an informative prior pulls the MAP toward its own mean (for exponential priors, that’s 1/λ, where λ is the prior’s rate parameter). If that prior mean is far from what the data actually supports, the shifted MAP will sit in a region where the likelihood is lower than at the MLE. That’s the most likely culprit here.

• If it’s the marginal likelihood, your informative prior might clash with the data
  The marginal likelihood (or "model evidence") is the average of the likelihood across all possible parameter values, weighted by the prior. If your informative exponential prior is concentrated on parameter values that don’t fit the data well, this weighted average will be smaller than when using a broad, non-informative prior that covers the data-friendly parameter range. Think of it like: your informative prior is putting most of its "weight" on bad fits, so the overall average likelihood ends up lower.

• The shape of your exponential priors matters a lot
  Exponential priors are defined by a rate parameter λ (their mean is 1/λ). A "non-informative" exponential prior usually has a tiny λ (super broad, mean approaching infinity), so it barely constrains the parameter. If your informative prior has a larger λ (narrower, mean tied to a specific value), and that value doesn’t match what the data is telling you, the prior will fight against the data—leading to lower likelihood metrics across the board.

Quick checks to confirm which case you’re in: #

• Pull up the parameter estimates from both runs: compare the MAP (or posterior mean) from the non-informative run to the informative one. If the informative run’s parameter is far from the non-informative one, that shift is almost certainly why the likelihood is lower.
• If you’re looking at marginal likelihood, check if your informative prior’s mean is in a region where the data’s likelihood is low. For example, if your data suggests a parameter value of 10, but your informative prior has a mean of 2, the prior is weighting the low-likelihood region heavily, dragging down the overall marginal likelihood.

内容的提问来源于stack exchange，提问作者SamPassmore