If you are a member of The Church of Bayes (*) I recommend you read this.
If you are a physicist considering to become a member I recommend you read that.
(*) Wikipedia: Thomas Bayes (1702-1761), British mathematician, statistician and religious leader
I wanted to write about a nice illustrative example of Cosma's 1st exercise as the 2nd part to this post. But it has already been done before I could even begin with the typing (I was travelling) and I guess it is much better than what I would have achieved (but I would have left out the 'waterboarding', which is not that funny).
I recommend the comments to Brad DeLong's example if one is interested to see some members of the church argue (with each other). But then, perhaps you have something better to do...
The main lesson from all this is very simple. If the set of considered models does not contain the true model then Bayesian updating can go very wrong. But how does a Bayesian know that her process includes the true model without leaving the reference frame of her church?
Of course, we should not expect a true Bayesian to agree that there is a problem (e.g. in the comments to DeLong's post) - after all we know (now) that their procedure does not always converge on the truth...