One relationship between the "MEL rule" and the "Bayes Rule" is that:
The Bayes rule is always well defined, but the MEL rule may not be
They will be the same as long as the Bayes risk is finite
The MEL rule is always well defined, but the Bayes rule may not be
They will be the same as long as the Bayes risk is positive
When constructing the prior p.d.f. for the parameters, a Bayesian econometrician must:
Never look at the data first
Do this before seeing the current sample of data, and must make sure that the prior is "proper"
Do this before seeing the current sample of data, and must make sure that the prior is adequately reflects ALL of the knowlege available at that time
Never look at the data first, and also make sure that the resulting posterior density is of a nice standard form that is easy to analyze
Bayes estimators are:
Admissible, consistent, and unbiased
Linear, unbiased and admissible as long as the prior is "proper"
Consistent, often biased, but admissible as long as the prior is "proper"
Admissible, linear, but possibly biased
The Bayes estimator for the mean of a normal population when the variance is known, and the natural-conjugate prior is used, is:
A precision-weighted average of the prior mean and the sample mean if we use a "zero-one" loss function
A precision-weighted average of the MLE of the population mean, and the mean of the prior density
Biased, but admissible, if the loss function is quadratic
All of the above, if the loss function is quadratic or "absolute error"
If we choose a "diffuse" improper prior then:
The posterior density will also be improper - that is, it will integrate to infinity, not unity
This will be the natural-conjugate prior if the data follow a proper uniform distribution
The posterior density will generally still be proper (i.e., it will integrate to unity)
None of the above
Under quadratric loss, the MEL estimator for the coefficient vector in a standard linear multiple regression model, with Normal errors, is:
Admisible, regardless of what prior density we use
Unbiased and consistent, regardless of what prior density we use
The same as the MLE when the priors for the coeffcient vector and the scale parameter for the error distribution are "diffuse"
The same as the MLE when we use a natural-conjugate prior
If the sample size is sufficiently large, then:
Then the Bayes estimator will always converge to the MLE
Then, as long as the prior density is not "degenerate", and has some variance, the posterior density converges to a normal density, centered at the MLE
Then, as long as the prior density is not "degenerate", and has some variance, the Bayes estimator will converge to the MLE
Then, if we are using a quadratic loss function, the Bayes estimator will converge to the MLE
Bayesians deal with "nuisance parameters" by:
Putting a "diffuse", non-informative prior on these parameters, so that they effectively play no role in the analysis
Putting a prior on them, retaining them in the analysis until the posterior density for ALL of the parameters has been constructed, and then marginalizing the posterior with respect to these nuisance parameters
Integrating them out of the prior density before combining this density with the likelihood function via Bayes' Theorem
Putting a prior on them, retaining them in the analysis until the posterior density for ALL of the parameters has been constructed, and then conditioning the posterior with respect to these nuisance parameters so that the focus is just on the remaining parameters of interest
A "natural-conjugate" prior is one with the property that:
For a given form of likelihood, the posterior density is from the same family of distributions as the prior
For a given form of likelihood, the posterior density differs from the prior density only by a factor of proportionality
For any type of likelihood, the posterior density is from the same family of distributions as the prior
It is from the same family of distributions as the likelhiod function
In order to implement the Gibbs Sampler to determine the marginal posterior densities for the parameters in our problem:
We need to recognize the forms of the various conditional posterior densities as being of "standard types"
We need to be able to sample randomly from each of the conditional posterior densities
We need to be able to numerically integrate the joint posterior to eliminate the parameters that are not of interest
We must limit ourselves to problems with a "manageable" number of parameters